Clay, the Sometimes Abrasive
Clay is derived from the mineral feldspar when granite is weathered and the little pieces get washed down to the ocean where they build up as layers of clay. Feldspar is hardness 4 on the Mohs Scale and steel is 5 - 5.5. The chemical formula for Feldspar is KALSi3O8 or a potassium aluminosilicate (the numbers should be subset). Clay won't abrade steel in its natural state. If you fire the clay into pottery however, everything changes because you have taken the soft sedimentary clay and reproduced the igneous process. Try taking a dull kitchen knife to the edge of a clay flower pot and you'll see the fired clay will brighten the edge.
The making of stones you're describing is called "vitrification" and the process is far from simple. Norton (now St. Gobain) has decades of experience doing it and they produce quality stones. Contrast that against the cheap Chinese stones that are on the market for a couple of dollars. They're warped, far from flat, and lack quality.
I, and other knifemakers, use the Chinese stones by soaking them in water for a few minutes and then clamping them in a vise and then forming the blade bevel. It's work but it's a welcome alternative to a noisy 1hp 2"x72" belt sander (mine is a Wilton Square D). It's fun to do from time to time. This process was shown to me in a class given by reknowned bladesmith, J.D. Smith who is known for his beautiful and tasteful damascus blades and swords.
So to sum up, clay is sometimes an abrasive but the binder use of it is inconsequential as an abrasive against the abrasive material that is held together by the fired clay.
Hope that helps answer your questions.