Yeah, I have that problem when I use oil.

Just wash your hands with soap n water, and do your blades at the same time.