I would say to remove whatever rust that you can easily get off. 400 grit wet or dry or whatever won't matter too much if appearance is not important to you.
I don't know what Nortons you have...