I've got a leather Cr2O3 paddle that was originally pasted with 3 micron diamond. I used saddle soap and a toothbrush to get it out. After it dried I used it as a daily strop for a couple of weeks with no apparent effect on the blade. There was probably a little bit of diamond left over, but not enough to do anything. Any abrasive embedded deeply enough to resist the soap and scrubbing is also probably embedded deeply enough that it's no longer at the surface.