I am going to do some more messing with it tonight and see what I come up with. I think that somehow the cloth will clip through, then it won’t correctly get unstuck because of self collision. From what I can tell, the underside will collide through the top, then it is a going slower after the initial impact, so it won’t collide back through the mesh because it detects the other vertices and respects the collision.
The blend file can be found here: http://pasteall.org/blend/index.php?id=49776
I have already applied the modifiers to the socks, but the shirt shows what is going on.