SSH keys work exactly like self-signed certificates. On first connection you get the "whoah, this isn't trusted, do you want to proceed" warning, and if you accept, you are not warned in the future unless the key changes.
If browsers would make it easier to "permanently accept" a self-signed certificate (right now it's usually a multi-step process with blood-red warning messages at every step) we'd have the same situation as SSH keys.
HTTPS involves multiple servers that are interchanged regularly and commonly use different keys. The keys change regularly. There'd be no way to know if the key changing is part of regular operation or a man in the middle.
It would be unwise to do so. In any case, phasing out old servers and certs for new ones is common practice. Using one cert everywhere (which would mean using one set of keys everywhere, which is a horrible idea) would require more downtime for maintenance of the certificates. It's not gonna happen.
If browsers would make it easier to "permanently accept" a self-signed certificate (right now it's usually a multi-step process with blood-red warning messages at every step) we'd have the same situation as SSH keys.