Hm, I don’t expect the Scaling having an influence on the Baking result, but I can’t be sure.
I suspect it has something to do with the UDIMs, but I haven’t used them myself yet, so it is also just guessing.
Maybe it’s a bug, or maybe you’re doing something wrong (Baking is very error-prone…I am not a religious person, but I always pray before I click the Bake button).
If you decide to upload it, I’ll be happy to take a look, I don’t think screenshots will be enough to troubleshoot in this case.