Hardware Wallets

At this point I am a bit confused on the insane emphasis for a hardware wallet. As far as I can tell, and please correct me if I'm wrong. The hardware wallet is really there for device specific usage. If you have your keys, you can still access your btc, regardless of hardware wallet or not..with the exception of using the same wallet on the same device used to originally generate your keys. So wouldn't your best bet is to generate your own key, and never have it online or in a device?

That being said I do if using one, an open source and audited device with your own 100 dice roll, is optimal if using one ie not Ledger.