I would just leave the mug without rinsing it. The leftover lather will just convert back to the soap so you can use it when you shave next.