I've had similar results to the OP. I just couldn't get a good shave with cold water.
I am still wanting to experiment with cold water, as I've read some posts explaining the rationale for it, and I believe it has some merit.
I think when I shave tonight I'm going to try 1st and 2nd pass with hot lather, then switch to cold lather for a final pass. (I'm going to pour the hot water from my scuttle, then refill with ice water... Hopefully the temperature change won't cause it to crack or anything)