They are constrained between the stone and the steel. This causes them to either roll or slide when the steel is moved over the stone, gouging out microscopic chunks as they go. The finer/smaller the particles are, the finer the finish produced. The filter the particles are, the more they polish or cut shallow more rounded grooves rather than cut deep sharp edged ones.

All of which is available in detail on Google if you really want to understand more. Search for slurry lapping. Most of the time nowadays it's done on powered machines, using diamond or other abrasive slurry, though back in the old days it was common to have a lapping plate in every machine shop for abrasive hand lapping to create high levels of flatness or smoothness.