320 grit is rather coarse. I would lap it up to 1000 grit minimum.
To find out which side is finer just take a dull razor and perform 10-15 laps on each side and you will "feel" it right away. This...