If it's causing problems I wouldn't do it. Unless you are young and just starting out you have probably managed up until recently to get an acceptable shave without shaving ATG.
Some barbers advise...