An atomic operation is a sequence of steps that are indivisible in their effects. That's a return to the original definition of "atom." Atoms as constituents of molecules are the exception from the trend and a misnomer.
There are two major uses of the term "atomic" in CS. The sense is either that of database transactions, in which either everything happens together or nothing happens at all, or that of concurrency, in which... really, it's the same thing, but with a focus on whether an intermediate state can be visible from another thread/process, as opposed to whether an intermediate state can be written to persistent media. An atomic operation is one in which, whatever method you use to observe the affected state, you will only see the state from before the operation begins or the state from after it completes.
It doesn't matter whether the CPU does the operation by magic or by some physical process involving the passage of time; that's not part of the concept. The operation is not divisible; it isn't possible for part of it to occur.
Mostly, I agree. However, in a multitasking environment anything that is multiple instructions can be significantly split up in time. While not the primary effect of the op, the op itself is still divided which can have other non-negligible secondary effects on a large system.
