Generally, when they are two separate Objects, add the second one in Object Mode. For example. if you have a table and you want to add a plate, add a Circle, or an other Primitive, in Object Mode. When you have a tabletop, and want to add a leg, add a Circle or Cylinder or… to the tabletop in Edit Mode.
But don’t stress about it; know that you can join two Objects, with CTRL+J (see the answer to your other question) and separate part of an Object (like the plate on the table), by, in Edit Mode press P > Seperate by Selection or by Loose Parts.
Sometimes you want to seperate the arms of a character for instance and do some changes to them and then combine (join) them again to the body.
There are no strict rules (and I still, after more than 10 years, make the wrong decision sometimes) but it is easy to ‘fix’.
.
