Assuming your prep and lather are good, it sounds like you may be trying to cut against the grain (or partly against the grain) when the whiskers are too long.

There's a tendency for the blade to hook up under long whiskers. Then, you try to apply more force and you lose control. Keeping the blade as sharp as possible minimizes the problem, but you can also try getting the whiskers as short as possible before you go against the grain. A fine stubble is best. Then you can actually reduce the pressure a lot, increase the angle of the blade (raise the spine) and very lightly and carefully cut into the grain.