I have some experience with LLM, and they definitely do consider the question. They even seem to do simple logical inference.

They are not _good_ at it right now, and they are totally bad at making generalizations. But who says it's not just an artifact of the limited context?