I've never had any trouble with removing pins by filing and tapping since I started using a jewellers anvil. This one has a hole about the same size as most pin heads so only one side needs to be filed. Because the scales is supported all the way around the pin there is very little chance for the scales to crack or break. As for a punch I use a cheap jewellers screw driver filed to a point.
Using this method with these tools I don't have to rely on tapping, but can hit it a bit harder if need be.