Start over with some pressure on the 4K and see what happens.

220 to 4000 is a huge jump - you need to do a lot of strokes to smooth out the scratches from the 220 grit.