Hi Szymon @luklum ,
With this method, it is probably easiest to slightly tweak the Constant Offset (Z Value), trying to get the overlap as small as you can. When you enable Wireframe in the Viewport Overlays, you can see it better. Then Add a Generate > Weld Modifier at the bottom of the Modifier Stack and tweak the Merge Distance, to get completely rid of the overlap.