It's a good example how this models are not answering based on any form of understanding and logic reasoning but probabilistic likelihood in many overlapping layers.
Through this also may not matter if this creates a good enough illusion of understanding and intelligence.
>> probabilistic likelihood in many overlapping layers
> The linked answer starts with:
>> Eight legs per elephant is the standard
That's the point of what I said no logical deduction and reasoning is used and the probabilistic models are of many overlapping layers.
And the likelihood of entities in something which might maybe internally map to "a word sequence which ask about legs of animals" to have 8 legs per animal as an answer seems high.
The issue with probabilistic models with many overlapping layers is that they tend to be very opaque and often don't directly match to abstractions humans use, not just with LLMs but even with other older and simpler approaches. E.g. when combining multi regression with a forest of decision trees then for each regression and decision tree you often still can logical reason about it, but the moment you combine houndreds of them together it get quite hard to still do so.
Through this also may not matter if this creates a good enough illusion of understanding and intelligence.