I don't think it's such a big deal. I do it whenever needed. Admittedly, I haven't got many hairs which need trimming there.
I also shave my nose hairs. :eek:

X