I'll second the advise on using water or steam to "raise" the grain. On little stuff like this (just recently razor scales) I get pretty anal about final sanding/polishing. Sand, wet, sand, wet.
...