Displacement is dependent on object scale with respect to the world. So it sounds like the objects you’re applying the shader to are significantly different scale than the totem.
Of course you can just adjust the displacement strength via the math node that I dialed way down. Just raise that value till it displaces the way you want.