diagonal edges are visible in the upper left and lower right corner of the cap outline. These edges are obviously not part of the original rectangular grid cell mesh
Actually they are! I just edge-slid those verts a bit to show that you can change the shape of the cap a bit, even after you created the target mesh.
As for the complex model: that is not how this was intended (did I forget to mention that you should start out with a clean topology?).
As this is a Subsurf method, I wouldn’t use such an already rounded shape with 153(!) verts. make it as simple as you can to still get that shape when subdivided and then maybe subdivide the Sphere one more time and the move ( Merge) the verts of the sphere to make that shape. Or just create that (simplified!) shape directly on the Sphere by sliding some verts. That ensures that you keep the all-quad topology (at least in that area).
