Straight razors were designed to be using with a slicing or scything stroke...you shouldn't be trying to just plow through the hairs.
Just be careful not to slice too much in the stroke.