I can only think of two ways, adding a washer like you said or by removing the scales and sanding the inside down so the scales are thinner there and don't make contact with the shank.