If I go for a three-pass shave between my chin and lower lip, I know I'll end up with a little acne crop in that zone. One or two passes and I'm OK, and one pass there will get me out the door and presentable enough... so I've given up on getting that area as clean-shaven as it can be. So I'm just reaching an equilibrium between my razor and my skin.

You might consider a beard where the trouble is worst. A lot of classic beard shapes, I now realize, just match zones that are a pain to shave with a straight. A well-trimmed beard may be what your skin really wants there.