I worked for several years in an inner-city YMCA-YWCA, which is to say that I have several years' experience working in an environment in which people of all backgrounds, orientations, persuasions, etc., were welcome. The number of issues brought to my attention as either front-desk staff (to whom 95% of all problems were reported) or as a director (to whom the other 5% were reported) stemming from trans people using the change room/bathroom someone else thought inappropriate: 1.

Number of issues completely resolved to everyone's satisfaction after a conversation with the complainant in which the situation was explained: 1.

Number of people harmed by trans people using the "wrong" facility: 0.

Just sayin'.

Trans people are not out to get anyone. They are just trying to live their lives. And I say this as a heterosexual cisgendered dude, but as a heterosexual cisgendered dude who has taken the opportunity to get to know many people whose sexuality/gender identity/whatever is very different from my own.