A piece of copper sheet, roughly 11 gauge or a little thicker, is a great way to remove rust.
Another hobby of mine is restoring civil war era guns. I was fortunate to be able to learn some restoration techniques from a firearms museum curator.
Copper is softer than steel but harder than rust. It will leave copper color on the razor, though that will come off. A light touch is critical as you get closer to the edge of the blade, though a light touch is all that is ever needed. I've never had it scratch steel.