I've been doing towel wipes for a year and a half. No damage. I'd worry more about the towel, than about the razor.

Charles