Editor's Synopsis
A systematic review and meta-analysis published in Nature Human Behaviour examines whether combining humans and AI yields better performance than either alone. Researchers analyzed 106 experimental studies with 370 effect sizes published between January 2020 and June 2023, finding that on average, human–AI systems performed significantly worse than the best of humans or AI alone (Hedges' g = −0.23). However, these combinations did outperform humans working alone, demonstrating what the authors call human augmentation (g = 0.64). The findings challenge the widespread assumption that pairing humans with AI automatically produces superior results.
The study identifies critical moderating factors. Task type matters significantly: decision tasks (choosing among finite options) showed performance losses, while creation tasks (generating open-response content) showed performance gains from human–AI collaboration. The relative performance gap between human and AI also proved crucial — when humans outperformed AI alone, the combination achieved genuine synergy (g = 0.46), but when AI outperformed humans, the combination suffered losses (g = −0.54). The authors hypothesize that when humans are more skilled at a task, they are also better at judging when to trust their own judgment versus the AI's suggestions.
Notably, factors that have received significant research attention — such as providing AI explanations or confidence scores — did not significantly affect human–AI system performance. The authors suggest researchers should shift focus toward less-studied but more impactful factors like task type, baseline performance differences, and division of labor between humans and AI. They also note that only 3 of 100+ experiments explored predetermined delegation of subtasks to humans and AI, pointing to a largely unexplored but promising design approach for achieving genuine human–AI complementarity.
