Risks of Hardware Wallets
While hardware wallets are certainly safer than most options (so long as they are ordered directly from the manufacturer and have never been used or tampered with) there are still some worrying aspects about them which can make them potentially vulnerable.
Most wallets out there require you to plug them in physically with a computer or phone when you use them. While this is reasonable, considering the technology of today, it does expose the device to potential unknown vulnerabilities of the computer/phone and the hardware wallet itself. For example, there could be undiscovered vulnerabilities in the USB hardware or the wallet. By plugging the wallet in directly, these vulnerabilities are now presented to potential malware on the computer or phone.
A better, but somewhat harder to use approach, is an air-gapped hardware wallet. Here, the wallet is never plugging into a phone or computer. Instead, the device scans QR code data off a phone or computer and produces QR code data (after confirmation) to be scanned by a computer or phone. This air-gapping ensures that the device is less vulnerable to hardware or software level attacks, made possible by a direct physical connection, such as a USB cable.
Most hardware wallets are also vulnerable to a slew of spoofing techniques, some more recently discovered. Malware can intervene between the device and your computer and show you invalid receiving addresses (i.e. accounts the attacker owns and not you). An unaware user would then send funds to the attackers accounts not owned by the user. In order to thwart these kinds of attacks, you should derive your accounts on multiple computers and notate or know your commonly used receiving addresses before regularly using your hardware wallets. This way, if malware is in between your hardware and computer you can be alerted to it. For more information and recommendations from the Ledger team, see the references below.
While there are some air-gapped hardware wallets coming onto the market, they are less popular and harder to use on a regular basis.