With your sensitive skin that is prone to ingrown hairs and breakouts, as you note, avoid going against the grain at all cost. Why make matters worst? You may not get a perfect shave, a marble smooth one, but a comfortable shave makes more sense.
The neck is difficult to shave for most people. It has always been for me. I do the best I can and opt for a comfortable shave that is irritation free rather than tear up my skin just to get a perfect shave.