My whiskers go in similar direction. I tackle the problem by going on an 45 degree angle from my chin towards the side of my neck. Not directly against the growth, but enough where I get the same...