No surface is "perfectly" smooth. There are always microscopic bumps and pitts. Even glass isn't perfectly smooth(just damn close)

I have polished knives on a leftover piece of 1-3/4" granite counter top with some oil on it.. the only grit would be the metal itself becoming slurry after being worn off by the microscopic roughness of the granite.