Like yours, the grain on my neck also goes sideways (e.g. from Adam's apple to ear). To complicate things more, my beard is very course. Both these things mean it may take more time to learn to get a comfortable shave below the jawline. I'm now three months in to SR shaving and have literally just reached the point of being able to give myself a consistent, comfortable, close shave on my neck this past week...

5 things I discovered that may help you get there sooner:
  1. Even if you think you're already using light pressure trying going lighter
  2. The stretch is very important. You don't need to stretch so tight you can bounce a quarter off your neck, just enough so there's no 'play' in your skin (I find overstretching on my neck can cause just as much discomfort as not enough)
  3. Pay close attention to your blade angle. On curvy areas like the neck it's very easy for the blade too get a little too steep (e.g. perpendicular to the skin) when going over curves
  4. In your early/learning stages, if the best angle-of-attack you can manage on you neck is up/down -- in other words, XTG -- use several light, short strokes between 2 or 3 lathers to get the job done rather than one harsh XTG pass followed by another harsh ATG pass
  5. Once you're a little more experienced, try moving the blade in short, diagonal sweeps - kind of like a stropping stroke - from the direction of your chin to the direction of your shoulder,rather than straight up or down. This lets the slicing force pass through the hairs in a direction that's closer to WTG. Again, this is a more experienced move since a hasty motion or sudden hiccup can easily turn it into a pure horizontal movement and give you a little bit of a slice

And, as many others have already said, making sure to prep/warm/moisten/soften your beard as best you can will help a lot.

Good luck!