That's what I would have done too, I meant that the magnetic backing is designed to hold the pad to the handle.
Printable View
One of the felt pads separated on mine.. I just stuck it back together.. Maybe not enough glue used when they are making these things... seems like a common problem... easy fix... more glue!