I’d say it’s all about how close you’re going to have the camera to something. A dirt floor with all the crumbs in the dirt can come through nicely if you have lots of geometry and a displacement map raising all that detail. Sometimes you have a displacement map for bricks in a brick wall, because you want to make a close up shot of the brick wall and you want all that detail to pop, which would get lost or pixelated with just normal or bump. But sometimes you just need lots and lots of details to accomplish other things that you are after. First thing that comes to mind is landscapes. You need a lot of geometry to be displaced and have all that sweet detail in the terrain or a mountain.
