Difference Between Cold Storage and "No Storage"?

This is probably a dumb question but...

If you move Bitcoin to a cold storage wallet then I understand that it is not on your computer anymore and that this is more secure as you can't get hacked unless somebody gets the cold storage drive and knows how to get in.

But if you move Bitcoin to, say, an Electrum wallet and save the seed phrase (on paper somewhere or in your head), and then you delete the Electrum app completely from your computer then also you would not be able to get hacked via your computer if somebody gained access to it.

So what really is the difference or pros/cons of the 2 approaches? Is there anything wrong with just deleting the wallet app and not spending extra money on a cold storage device since you can always recover the wallet with the seed whenever you want access to the Bitcoin by reinstalling the wallet software?