Backface culling works by only rendering one side of a polygon, and it chooses which side based on the direction of the face’s normal. This only works in the viewport though and does not affect Cycles, so the walls closest to the camera are set to be ignored by the camera in Cycles.
You could definitely just remove those faces if you want, but I wanted to be able to light the scene with an environment light and only have that light spilling through the windows and bouncing off those invisible walls.
