I use a "belt dresser" that I get at sears. It's like a big rubber type eraser. When new, they are about 6" long & a little over 1" square... Found it by the routing bits at my local sears...

Edit: I only use this on the finer belts...coarse belts just chew up the eraser...