I think the problem stems from not starting with a flat blank. If the horn was 5mm and you cut it in half you are already near the minimum width - and that's before you start to flatten and sand.

If the blank is not completely flat then when you drill pivot and wedge holes you are taking a chance in terms of alignment - if that's out then the scales can easily pull to one side - hence your problem.

On the other hand it might be the blade.