Go easy with the fabric side until you get the hang of it. For starters, keep it at hip level, just rotate on the spine (rolling like a pencil if you can), and push the razor back and forth with no downward pressure.
If you're moving from a strop with a heavier draw, you might be using too much pressure on the edge until you find your feel. Very unlikely you've done enough damage you need to go and rehone, more likely you just need to lighten up and you can recover the edge. Try that for 50 strokes or so, then move to leather, and see how it feels.