Box select should work in both object and edit mode on most workspaces. It sounds like you just activated some tool you didn’t intend to.
W key on the keyboard cycles the different selection styles typically. (I used left mouse button for selection and this is the behavior I experience)
W key cycles through Box Select, Circle Select, Lasso Select, and Move Selection tools. It is also the top left icon on the toolbar that can be activated with hitting T on the keyboard.
it should be the mouse icon. You can click and hold on it and pick the different types of selection tools. Box Selecting objects should work in Layout or any other Workspace.