The OS and application would know the dpi and hence could convert pixels to pts I thought.

That wouldn't help. Without wanting to go too deep into OT-territory:

Even if the DPI could be known (and afaik normally it isn't, it is just set to a value) and it would be possible to convert in into real inches (btw, I have no idea how big that is in real units) to be displayed on a monitor, how would a designer know how big any element, even text, should be? It depends on the available space, which depends on the size of the monitor/the size of the window.

To think about how absurd that would be, try to find a situation where someone would say "this heading should be 2 cm tall".

px as size lead to the same problem with the available space, but they at least correspond to the resolution.