Why, at the lowest level of the hardware performing operations and the general underlying operations involved (i.e.: things general to all programming languages' actual implementations when running code), is vectorization typically so dramatically faster than looping?

What does the computer do when looping that it doesn't do when using vectorization (I'm talking about the actual computations that the computer performs, not what the programmer writes), or what does it do differently?

I have been unable to convince myself why the difference should be so significant. I could probably be persuaded that vectorized code shaves off some looping overhead somewhere, but the computer still has to perform the same number of operations, doesn't it? For example, if we're multiplying a vector of size N by a scalar, we'll have N multiplications to perform either way, won't we?