According to @emollick, the new ARC-AGI-3 benchmark is “human winnable,” and he needed a few tries to solve it, raising questions about whether frontier models’ very low initial scores stem from the evaluation harness, vision and tools integration, or inherent LLM limits. As reported by Ethan Mollick on Twitter, this highlights a crucial AI industry focus: distinguishing capability gaps in reasoning from setup issues like agent tool use and multimodal perception, which will shape how labs invest in tool augmentation, vision pipelines, and benchmark design for trustworthy AGI progress tracking.