When you make lather, just make it a little watery.
For skin relief, a good moisturizing aftershave balm or even using lotion after that will make a bigger difference than the soap, IME.