yeah it's normal to have difficulty at this stage.
the razor has to be rather sharp, your beard well softened, and the angle low.

how about doing across the grain, would that work better?