TIP: If selecting INSIDE the 3D Viewport doesn’t work (well) go, think OUTSIDE it.
Open an extra window and transform it into the Blender Outliner (see image and link below)

Here you can always (de)select everything, the way you want it:
You can read more about the Outliner over here:
https://docs.blender.org/manual/en/dev/editors/outliner.html