I think I see what you´re saying, the shift of the uv seam is nothing more than the smoothing of the geometry itself, and how far it shifts depends on how even /close the surrounding geometry/loops are.
And if you really don´t want the uv to shift at all, you can´t use subdivision surface, cuz that´s what it does, it subdivies and evens it.
Thinking back the cases where it magically worked, I either had relatively evenly laid out geometry, or really close constraining loops…
Well I feel a bit smarter now, thank you Omar and Martin!
