In my opinion, sanding isn't the way to remove rust unless you want to sand out the pitting.

A little piece of pure copper is the way to go. It's harder than the rust and not as hard as the steel. Works like a charm. This method leaves more of the history on the blade, though the pitting will still be there.