I have not tried this but it seems like a good idea

http://straightrazorpalace.com/workshop/17007-handsanding-howto.html

He uses a wood jig with a leather piece on the end for sanding. Although...