Measuring Progress Toward AGI - Cognitive Abilities
Design high-quality benchmarks that go beyond recall to evaluate how frontier models truly reason, act, and judge.
Official page previewCaptured Oct 19 · not live
Open the competition on Kaggle↗
Open the competition on Kaggle↗- Effort
- 25–80 hours
- Difficulty
- Research
- Compute
- No compute
- Scoring
- Not verified
At a glance
Full rules ↗- Teams
- Up to 5 per team
- Submitting
- Up to 1 a day
- Host
- Google DeepMind
Is it for you?
Prepare before joining
Beginner fit16/100
Learning value72/100
Who it suits
Researchers, benchmark designers, and advanced LLM practitioners interested in evaluating cognitive abilities beyond memorization or simple benchmark recall.
What makes it hard
The core work is benchmark design and evaluation methodology. It requires understanding what a high-quality AGI-oriented task measures, how to avoid leakage, and how to communicate judging criteria clearly.
Getting started
Skills to bring
- benchmark design basics
- LLM evaluation literacy
- technical writing
- awareness of data leakage and contamination risks
We haven't pulled tutorials, data or baselines from the official page yet; the task and the data are there.
First competition? Read No GPU? Pick Competitions That Still Teach You Good Habits