Perplexity AI Releases WANDR: An Open Benchmark Evaluating Research Agents That Must Search Wide And Deep

Perplexity AI has introduced WANDR, a comprehensive open benchmark designed to evaluate the capabilities of AI research agents in conducting thorough, evidence-based investigations. The platform features 500 evidence-heavy tasks that challenge AI systems to discover multiple qualifying entities while providing cited, re-verifiable evidence for each finding. This development marks a significant step forward in standardizing how researchers assess the quality and reliability of autonomous research tools powered by artificial intelligence.

WANDR establishes a rigorous evaluation framework that tests whether research agents can effectively balance breadth and depth in their investigations. The benchmark measures performance using two distinct metrics: soft F1 scores and hard F1 scores, which differentiate between partial credit and strict accuracy requirements. Perplexity's own Search as Code system currently leads the benchmark with a soft F1 score of 0.363 and a hard F1 score of 0.133, demonstrating both the benchmark's discriminative power and the considerable challenges that remain in this domain.

The benchmark's design prioritizes evidence verification, requiring that all discovered entities be backed by citations that can be independently validated. This emphasis on reproducibility addresses a critical concern in AI research: ensuring that agent-generated findings can be audited and verified by human researchers.

• Standardization of research agent evaluation: WANDR provides the first comprehensive open standard for assessing research agent performance across diverse, evidence-heavy tasks
• Transparency in AI capabilities: The benchmark's focus on verifiable evidence sets a new baseline for accountability in autonomous research systems
• Competitive benchmarking: The open framework enables other AI developers to compare their research agents against established performance metrics
• Advancement of search technology: Results highlight significant room for improvement, driving innovation in how AI systems discover and validate information

As AI systems increasingly assist with research, information discovery, and fact-finding tasks, establishing reliable evaluation benchmarks becomes essential. WANDR addresses a critical gap in the AI evaluation landscape by providing an open, evidence-centric framework that prioritizes accuracy and verifiability. By releasing this benchmark publicly, Perplexity is fostering industry-wide improvements in research agent capabilities while promoting transparency and reproducibility in AI-assisted research methodologies.