There are many ways to fill the hole and take out the slop. I usually choose the leave it alone option. Razors worked for years of daily use with the hole that size.

I fill the hole and take out the slop only when I am making a set of scales to a real close tolerance and the blade will be swinging very close to the wedge. Somtimes it is fun to make everything as precise as you can, sometimes it is fun to just put the thing together use it and dont worry about it.

BTW - My favorite method of filling the hole is a section of ball point pen tube,ahem, a low friction polyethylene bearing.

Charlie