LLMDates unconfirmedWhat's Up, Docs? Document Summarization with LLMs
Looking for a great way to start working with LLMs? See if you can summarize research papers from an open archive of the social sciences.
Official page previewCaptured Oct 19 · not live
Open the competition on DrivenData↗
Open the competition on DrivenData↗- Effort
- 3–12 hours
- Difficulty
- Beginner
- Compute
- Single GPU
- Scoring
- Automatic
At a glance
Full rules ↗- Scoring
- “For scoring how well your summaries align with the author-written abstracts…” The official page's words
Is it for you?
Strong first competition
Beginner fit85/100
Learning value90/100
Who it suits
Beginners interested in LLM-based text generation who want to practice prompt engineering and document summarization with automated evaluation.
What makes it hard
While the task is approachable with modern LLMs, optimizing for the combined ROUGE and semantic similarity metric requires careful prompt design, and balancing brevity with completeness in clinical document summarization is non-trivial.
Getting started
Skills to bring
- Python
- basic understanding of LLMs and prompt engineering
- text processing basics
- familiarity with NLP evaluation metrics
From the organizer
- DataSocArXiv social-science document data source ↗
- TutorialBenchmark blog post to help entrants get started ↗
- HelpCompetition forum for questions about rules
- HelpEmail support for questions about rules
First competition? Read How to Choose Your First AI Competition