Reverse the bit, using the smooth portion of the shank as a bushing, or a piece of rod, file off the spurs, so just the flats are cutting and use by hand I think would work well.

Good idea Richard…