The grit from the sandpaper can become embedded in the stone. If you use sandpaper make sure to keep it thoroughly wet and free of anything. As for the holes I don't think they would cause any damage...