That's only true when forwarding IPv4x -> IPv4. When you're going the reverse direction and you need to forward IPv4 -> IPv4x, well, still need a table then.