When the scratch pattern from the 2k has been largely replaced by a 4k or 8k pattern. Often visible in good light without magnification. Or when the TPT and/or HHT gets noticeably better. Or all 3.
I use the visual method until it looks close finish-wise. Then I TPT the edge. If the TPT confirms my visual assessment, then I'm done on that grit.