It took me a while but I finally understand your description. I pictured it as being the dimensions of 1/4 of a barber hone, so with two surfaces significantly wider that the other two sides. Given the wedge shape of the stone transition, I would assume it is a natural stone and not combination of a Thuringian and Tam O'Shanter.
I suppose I'd suggest using whatever side of the rubber that seems to most closely match the large stone. If one side of the rubber is scratching, you can fix that by lapping it. If you don't have a lapping plate, you can accomplish this easily by rubbing the scratchy side against the narrow, unused side of the large stone. However, if you use the strategy shown in the figure provided by Jimmy, then the scratching probably won't be an issue, but I'd still lap it smooth anyway.