Knife makers use a smooth flat file to knock the peened end down to a better level and the stroke of the file also anneals the pin end to make it form more easily..
And...yes, I have done the same...