I’m not quite sure I follow. That’s how it goes really, when you move your mouse, the view moves. Once you’re on orthographic view, you should only zoom in and out, because if you orbit, that will take you out of orthographic. Maybe you want to have different views at the same time? There’s the quad view option:
