We don't actually know if it is SOTA, the previous SOTA solution also got around... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

traject_ on June 17, 2024 | parent | context | favorite | on: Getting 50% (SoTA) on Arc-AGI with GPT-4o

We don't actually know if it is SOTA, the previous SOTA solution also got around the same on the evaluation set.

cma on June 17, 2024 [–]

Yeah and GPT4o was potentially trained on this test set and if the tried to hold it out it was still likely trained on discussions of the problems.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact