I think you'd be surprised at just what you're allowed to omit in HTML. For example, the following document is valid HTML 4.01 Strict:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<title></title>
<p>
The 'html', 'head' and 'body' elements -- opening and closing -- are inferred at the correct points by a compliant parser, as is the closing of the lone 'p' element.
Once upon a time on my personal blog, I used minimal HTML 4.01 Strict (omitting tags and other bits of syntax whenever and wherever possible -- I had a Markdown fork that knew how to do this within the constraints of the spec); my one concession was a longer DOCTYPE declaration to deal with anything that didn't know how to resolve the public identifier. Browsers actually handled it just fine, but some other systems (in particular, OpenID autodiscovery) failed, because they weren't actually doing HTML parsing.
You're right, I'd forgotten about omitting the closing </p>, but I think that's the only one (omitting a whole element - both open and close - is something different).
Of course, in practice, most HTML parsers will do a wonderful job of interpreting malformed HTML.
It's not the only one. You can omit closing tags for td, tr, th, li, and probably others I don't remember. I actually do this; it doesn't make the resulting document any less clear for humans, and so if I'm typing it myself, there's little point in inserting them.
Once upon a time on my personal blog, I used minimal HTML 4.01 Strict (omitting tags and other bits of syntax whenever and wherever possible -- I had a Markdown fork that knew how to do this within the constraints of the spec); my one concession was a longer DOCTYPE declaration to deal with anything that didn't know how to resolve the public identifier. Browsers actually handled it just fine, but some other systems (in particular, OpenID autodiscovery) failed, because they weren't actually doing HTML parsing.