Hm,
I am not 100% sure, but I think Occlusion Culling like you mean already happens automatically in Cycles. That is, for the Russian Doll example.
If an Object is behind an other, it might have some contribution, via Indirect Bounces, to the Lighting, or be visible in Reflections.
You’d probably be better off using EEVEE, if you want something like that.
