I agree with the other posters of having a dull blade, I experienced this when first starting and that extra little bit of sharpness really makes or breaks a shave.

On another note, dont be afraid to try cross grain or against the grain once your blade is sharp enough to be comfortable. When I first started shaving with the grain was the most difficult for me and cross grain was the easiest. It is important (as you seem to realize) to hone your skills in all areas of the sahve, but its just as important to do what feels more natural so that you dont get to frustrated to fast.