When adding a keyframe, you are adding a start position and and end position, every frame in between is based on the interpolation method.
Rotating around a pivot point in the viewport from the start keyframe to the next keyframe will not be the same in the animation.
If the interpolation is linear it will just travel in a straight line from one keyframe to the next.
You need to learn more about the graph editor. The Fundamentals of Animation is what you need.
