I have encountered the exact same problem!
However changing from GPU to CPU did do something, it still shows a very odd result.
When using the “Alpha Over” node, it only shows the layer that is plugged into the bottom input of the node.
Now upon checking it seems that all alpha layers are value 1.. And I have no idea why. (followed all the exact same steps as Sean did).
I have also tried combining everything using the “mix color” as node set to “mix” @mackdadd suggested (and tried other options as well) without usable result.
I am running on a M2 Ultra. (Blender 4.2).
So I guess I have to wait for a patch in a new update?
UPDATE: I made it work.
I also needed to switch to CPU, but during the process of figuring out why it wasn’t working, I also disabled the “transparent” in film settings. This resulted in fully white alphas. Enabling this resulted in usable alphas 🙂
