There's no reason for the pivot hole to enlarge. here's a tutorial.
Un-pinning and re-pinning a razor - Straight Razor Place Wiki