Sounds like the edge is crumbling after the initial shave. If the bevel wasn't set you wouldn't be able to get even one decent shave out of it. How many laps are you doing on each hone?

As well,...