+1 on Nun2Sharp's suggestion of using your fingers. Actually, a few old Barber texts from the turn of the century that I have copies of state that it is preferred that you rub the lather into your beard with the finger tips.
You can still use the brush to whip up the lather, but apply it with your finger tips.
Good luck!