What kind of razor and condition??

Other than this I would look to the difference in pressure you are putting on the side of the razor without the burr.

Have you tried circles??

Have fun,
...