Checking the “Caching” box for the “Audio Strip” in the “Video Sequence Editor” seemed to solve the problem:

But the more I’m scrubbing back and forth the more sounds are simply skipped randomly and later played again. If I then let Blender do a full replay (pressing “Spacebar”) or wait two or three seconds before continuing scrubbing, it’s working again.