If I rotate the camera around “Global X” in my scene, the horizon aligns well and the camera is pointing slightly upwards (around 96 degrees) as I would have estimated from the footage where a house seems to be standing on a hill with a slight slope. I accidentally first grabbed the plane and aligned the horizion with the footage which led to a strange result:

So, I think, for a manual alignment you should best know the real location well (since you’ve filmed it) because there’re otherwise several possible settings that allow an alignment without correctly reconstructing the real life location and camera settings.