I have this problem with some strops. I have ended up putting it down to pulling too hard and warping the leather through its natural grain. When you pull less hard it tends to flatten out.
Have you...