Yea, you really can't be 100% sure unless you have the razor in it's original sealed box. The scales can be changed the same way they were put in, you just need the same pins and washers if those were used and a very basic skill.
Of course when you see modern pins/scales on old razor, or mismatched pins chances are that's not the original job. With some other razors you can tell if the scales look from the period, or not.

BTW the same thing is true of the blades as well - quite often just looking at the lines resulting from the grinding process you can tell that there was regrinding at a later point. And then there is the question of when were the thumbnotches made, esp. the double ones from the 1800s. I've seen people with big egos and small brains claim originality of those with absolutely no evidence to base it on.