At the risk of sounding dumb, why don't you take a piece of polished metal such as the spine of a knife or straight and start with the highest grit and work down to the lowest grit. In this way the previous scratch pattern will not interfere...or so it seems in my mind.