In today's time, we've come up with an enormous number of psychiatric diseases/ailments. I'm not saying they don't exist mind you, I have a few myself. But collections are not that big of a psychiatric condition-disorder as we see it.
I also collect swords. And want more. Always more. But that doesn't mean I intend to use them. Or ancient coins. Or knives, there are plenty for many purposes, but those I use, a kitchen must have sharp quality knives.

Another aspect of HAD or RAD or any X-ADs and collections is the size of your income. If you can buy them and live your life-your family's life with no ill effects/shortages of vital... stuff or decrease the quality of your life because of it, by all means, do so.
And, I've done this wrong, as, there were times where I gave up going vacations for buying a sword (I was about 19), or not eating for 3 days because I had to pay for a stone -funny thing, I can't remember which one- and that's unhealthy (kind of, I needed to lose a few pounds anyway ).
After that, I started using our beloved SRP classifieds (I'm extremely thankful for its existence, I love you guys!), I sell the stones I bought after testing them, and this way I've bought tested and sold almost every stone that was at some point in recent history commercially available without spending more that my base "investment".

My point is, like all psychiatric diseases, if your HAD interferes with the quality of your life, you are doing it wrong, and might even need help.
If not, keep doing it. Besides, the prices of naturals only rise, you can also see it as an investment with guaranteed profit. But if you are doing it for the profit, I'll find you!!!