Just realized I didn't answer your question about other stuff.

Go to wal*mart and look for the VDH set. The logo reads VHD.

It includes a bowl, a puck of cheap but servicable shaving soap, and a cheap but serviceable brush. You will want to upgrade all of them later, but that will get you started for $9.

The only other thing you need is a strop. If you have access to a saddle shop, just go in and ask for a 30" long piece of 2" latigo leather. Punch a hole in one end and use a leather thong or some other heavy chord to tie it up. If the leather isn't as smooth as you'd like, just hand sand it a bit. Seriously. This is the cheapest way to get a strop, and will work great.

If there isn't a saddle shop available, I'd order a philly from... crud. I can't remember who sells the philly stop, but if you search the forums you'll find it quickly. That, or see if you can buy just a practice strop from Tony Miller.