Coinbase is not actually an exchange, and in fact markets themselves as a wallet:
"Coinbase is an international digital wallet that allows you
to securely buy, use, and accept bitcoin currency"
It really comes down to whether you trust Coinbase (equivalent of a bank, albeit a startup bank) or yourself (equivalent of putting cash in your mattress) to securely store your money.
It's not really anything like putting cash in your mattress. With Bitcoin, you can even store it in your brain, not to mention any place where you can store a piece of paper or a string of digits.
You can even trust a regular bank, by putting the key in their safe. And if it's encrypted, you don't even need to trust them to not abuse it.
And using something like ssss[1], you could also store it in a lot of places at once, without any single one having enough info to use your coins.
Ok, it's like keeping cash in your mattress, safe, safety deposit box, wallet, personal vault, whatever.
The point is it's your responsibility to secure it, and if you think you can do that properly then great. I can do it, but I don't think it's easy enough that most "normal" people can do it properly yet.
You being responsible for the security of your assets is the price you pay for having total control and ownership, just like with everything else. Putting money in the bank or PayPal reduces the risk of theft but increases the risk of bail-ins and account freeze. There is always a tradeoff.
Bitcoin offers a lot of security mechanism that are not available with traditional asserts, like m-of-n-keys etc. Insured online wallets will become available as well.
"Coinbase is an international digital wallet that allows you to securely buy, use, and accept bitcoin currency"
It really comes down to whether you trust Coinbase (equivalent of a bank, albeit a startup bank) or yourself (equivalent of putting cash in your mattress) to securely store your money.