How long have you been using a straight? It sounds like your razor is sharper than most of mine, but I get smooth comfortable shaves every time. I think technique may be the issue. I have vague memories of shaving and not getting all of the stubble, but that was a long time ago.

I can think of a couple of suggestions off hand. First, are you stretching your skin? Second, I would make sure that your angle is about 30 degrees, and that may fix the problem. Also, I noticed that I lead with the point on my razor for most passes. I think this approach helps to slice the hairs rather than chop them.

I am by no means an expert with this stuff, and before long I'm sure that you will have many good (maybe better) ideas.