"Has anyone tried to shave with only the against-the-grain
pass? ."

Like Chuck Yeager famously said, "It can be done, but
I don't recommend it."