Hm, that surprises me too, that without Portal, it Renders faster…well, Portals are more for quality, than for speed it seems. And that actually makes sense. In an enclosed scene, with light coming through a relatively small window, for instance, rays, shot from the Camera have only a small chance to ‘find the light source’ (window). A Portal ‘tells’ Cycles, where to ‘look for the light’ and that means more Samples will find the Light Source and contribute to the Lighting. It doesn’t necessarily mean that Blender uses less Samples (although it might for some Pixels, when using ‘Adaptive Sampling’, now enabled, when using the Noise Threshold Threshold)…Then there is also Path Guiding, which helps Cycles to clean up the Noise faster on certain Scenes, but that only works on CPU at the moment.
Like you know by now, there is a lot to it; there are no simple rules, what to use when.