It really depends on how interested you are in the historical aspect as opposed to just extending your hunting season. I started with muzzleloaders as a way to extend my hunting season, back before modern inlines. It has been an absolute game changer for me. I no longer own any modern firearms, and in fact, muzzleloading has sparked an interest in historical reenacting.
You can usually find me in the woods with my fowling piece. It's a 20-bore (.62 caliber) flintlock with a 42" barrel. Of course for deer I load with a round ball with flax tow for wadding. For birds or small game I load shot with the same tow wadding. It's only a 50-yard gun, but for NJ whitetails, that's all I need.
http://img.tapatalk.com/458efcdb-5860-9e00.jpg