I am having trouble making a flag with grommets :

The grommet (rigid body) always ends up falling through the flag (cloth).
I have tried various set ups, including parenting the grommet to the flag vertices surrounding the hole, hooks, parenting to an empty…
Well, here i am, defeated. Any idea about how i can make this particular configuration work ?