Editor's Synopsis
A controlled experiment involving 680 U.S. participants investigates how four distinct generative AI tools—Chat assistance, Conversation Starters, Feedback on drafts, and reply Suggestions—affect both content producers and consumers in a realistic social media environment. Participants were organized into groups of five and discussed three topics (dogs vs. cats, health benefits of oats, and universal basic income) across different experimental conditions. The study reveals a critical duality: while AI tools increase user engagement, content volume, and participation equality, they simultaneously decrease perceived quality and authenticity of discussions. Consumers rated AI-assisted content as more "robotic" and "generic," and all AI treatments significantly increased dislike reactions compared to the control group.
The research uncovers nuanced usage patterns across tools and topics. Chat was widely adopted (94.4%) with users adapting their prompts to context—casual queries for lighter topics, fact-checking for scientific discussions, and political engagement for divisive subjects. Conversation Starters improved participation equality but were often misaligned with user intent, with 27.5% of users diverging entirely from AI suggestions. Feedback was most valued in high-stakes discussions requiring argumentation and credibility, while Suggestions showed a notable bias toward agreeing stances in controversial topics. Critically, no single AI tool improved both producer and consumer experiences simultaneously.
Based on these findings, the authors propose four design principles for integrating AI into social media: transparent disclosure of AI-generated content, user-focused personalization, context-sensitivity accounting for topic and intent, and intuitive user interfaces. The study represents one of the first direct experimental tests of whether generative AI can genuinely elevate online discussions, with evidence suggesting it more likely amplifies low-quality interaction and creates negative spillover effects on conversations—even affecting users who don't use the AI tools themselves.
