It sounds like what you want to do is what the Japanese sword polishers call “foundation polishing” using an “arato” stone (about 180 grit).

Most of the modern polishers use synthetic stones at this stage. It may be best to go to one of the knife/sword forums and ask the traditional guys what stones they use at this stage. Don Foggs forum and the British blade forum are the ones I use to visit and have a few traditional guys on them.

A course carborundum stone, used with water may be the best bet as it is hard enough not to easily dish and will be fairly fast if you use some pressure.

“The craft of the Japanese sword” and “The art of Japanese sword polishing” are good reads concerning the use of stones to shape and polish blades.