If it isn't the soap, you might want to try using painting strokes for face lathering. It tends to be gentler on skin.