AI Hallucinations Drive Customers to Ignore Human Experts, Triggering Allergy Risks and Fake Orders
Updated
Updated · The Verge · Oct 2
AI Hallucinations Drive Customers to Ignore Human Experts, Triggering Allergy Risks and Fake Orders
3 articles · Updated · The Verge · Oct 2
Summary
Restaurant, park and childcare workers say customers are increasingly overruling staff with ChatGPT answers, creating safety risks such as diners disputing shellfish warnings and ordering dishes that contain allergens.
Workers describe AI-generated errors ranging from nonexistent wines and products to impossible river itineraries and flawed child sleep schedules, yet some customers still trust the chatbot over sommeliers, rangers and teachers.
In food service, the problem has widened from guests asking AI what to order to agents making reservations and sending last-minute requests for equipment or menu changes that staff suspect no human client actually made.
Meta’s new Muse agent could intensify the trend, adding to concerns that generative AI is eroding routine human judgment in customer-service interactions where direct expert advice once settled disputes.