I always clean up a few stray whiskers in just this manner. I wouldn't worry about dulling the blade at all, as your whiskers are still swelled up from water/soap/cream/whatever. You can easily get razor burn though. Best thing is to just avoid any long strokes with the razor. To date I've never got razor burn from cleaning up those 4-5 strays after shaving, and I do it every day. I have got a nick or two doing this, but it's pretty rare.