If you're handy and frugal, it doesn't have to be expensive at all.

You can pickup decent blades with broken scales for 2 or 3 dollars on e-bay, if you watch for the right deal. If you have access to a few tools, rescaling a razor is not difficult at all.

A nice supple leather belt makes a great strop. Shop around any thrift store and you can probably come away with a belt that will work well as a strop for a couple bucks. I have a couple real strops but my favorite and the one I use daily, is a Calvin Klein woman's belt. It is real nice soft leather and it is wide enough to strop a blade without an X pattern. The best part is the fact that I picked it up at a surplus sale for $1.00.

I have a boars hair brush that I picked up for $2.99 at a budget online site. I rarely use it because I prefer my badger brush. But it does work and seems very durable.

You will probably want an old barber's hone at the very least. You should be able to find one for 10 - 20 dollars I would think.

So with a little effort, you could definitely have a good setup with a couple good razors for less than $50.