I don't think points is a good fit for that. A point has a defined length, 1/72 of an inch. If you don't want your text to be 12/72 of an inch, don't specify the size in points.
Since the invention of desktop publishing, PostScript points have been 72 to the inch. Resolution-independent rendering was taken into consideration in the 80's. If a "point" is not 1/72 of an inch then it's rendered wrong. https://en.wikipedia.org/wiki/Point_%28typography%29
