Congratulations on your first shave.
Your skin feels less oily because with a straight razor you remove the oil and the top layer of dead skin cells from your skin.

You will quickly discover that your face feels much nicer when you don't use cartridges or canned soap anymore.

So for now: experiment, take your time, be careful and enjoy.