My setup:

ebay vintage straight razor: $32.
ebay hone: $11 - this is rather small but does work well.
shave brush: $5.
ebay strop: $11.
Shave soap: $5.

Comes to $64 in total. I'm wary to completely endorse this route since it requires learning to hone and assumes that you will source a good razor from ebay, which isn't always easy. With this said, learning to hone wasn't especially difficult (Gold Dollar razors are good for practice) and ebay does have a lot to chose from.

As the above poster commented, if you chose a shavette, your costs will be far lower and you will not require stropping or honing. Initial shaves will be tricky and it is very likely that you will make mistakes, although this improves in time. They're not the same as a proper straight but their low entry cost make be worth considering.