I imagine the answer is YMMV but I always rinse off the soap.