Yeah it’s a classic issue and a bit counterintuitive, but you have to smooth the object that is making the cut for the hole to be influence. Even though your brain would tell you that since the hole is in that object, it must be in the object itself, but nop, it’s the cutter one.
