Yes. This is done through the Feed Settings section of the PeepSo Configuration Settings (PeepSo → Configuration → CM Answers).
The Post Template can include some shortcodes to make the post content nice and relevant to the question.
To learn more about customizing activity posts see this Help article.

Is there any way to customize the Post that is automatically posted whenever a user asks a new question?