It’s a great question Marius, but tricky to explain; when you Scale in Object Mode, you are scaling the axes and not the object.
So, Scaling the default Cube on the X-axis by a factor of 2, means that “taking a step in the X-direction” gets you ‘visually’ twice as far as in Y-, or Z- direction because the X-axis is ‘stretched’:
As a general rule, you always want to Apply the Scale, to avoid unexpected behavior, like with Beveling as @jlampel pointed out. But in the case you are referring t, it probably has to do with an Array Modifier with Object Offset and then you get somethingv like this if you do not Apply the Scale:
but it is really hard to explain why this is happening…