A Chinese farmer had spent months learning to trust AI with his farm, until one recommendation wiped out nearly 25 acres of his sesame crop.
The 67-year-old farmer, identified as Wu, had reportedly been using an AI application for agricultural advice, including questions about fertilizers and crop protection, and had grown more confident in its recommendations after earlier advice appeared to work.
That confidence eventually led him to follow an AI-generated pesticide treatment for weeds and pests across roughly 150 mu of sesame.
The problem is not that AI will always be wrong; it is that a system can repeatedly give useful answers, build a user’s confidence, and then produce a bad recommendation with the same level of confidence. That makes human verification especially important when the cost of an error is high.
Details of the incident
Several outlets reporting on the incident confirmed that the AI generated multi-chemical treatments. But the exact number of chemicals used varies by publication.
Per Tom’s Hardware, the chatbot’s advice included mixing flupyrimethalin and flusulfasulfaether with thiamethoxazine and methyl salt.
ZME Science’s reporting on the incident, meanwhile, noted that the disastrous advice included mixing haloxyfop-P-methyl and fomesafen, both herbicides with insecticides, suggesting that the farmer wanted to get rid of both weeds and pests.
Wu followed the recommendation across 24.7 acres of farmland without cross-checking it against other sources or agricultural experts.
By the next day, the treatment had killed not only the weeds but also the sesame seedlings, turning what was supposed to be a crop-protection measure into a field-wide crop failure.
When Wu returned to the AI for an explanation, the system reportedly did what many AI chatbots do after giving confidently wrong advice. It pointed out that flusulfasulfaether is the likely cause.
That matters because the problem was not simply that the chemical could kill plants. They are, in fact, legitimate chemicals used for plant protection, particularly for soybean crops. The problem, however, might have stemmed from the combined use of all chemicals, according to ZME Science.
Repeated trust does not mean repeated reliability
Wu’s experience shows why outright trust in AI can be dangerous, particularly when its answers move beyond information and into real-world decisions.
Using AI for issues like crop protection can be especially tricky because they involve facts. And AI systems are known to be more likely to hallucinate, conflate scientific terminology, and omit important information, making their output better suited as a starting point than as a final authority.
Wu’s incident may be related to agriculture, but it’s not an outlier. According to The Independent, last year, a 60-year-old man used ChatGPT for health advice, and as a result, developed a rare condition after the chatbot told him to replace table salt with sodium bromide.
Even with the popular AI mistake labels, incidents like this indicate that many people keep trusting AI tools blindly when they should, in fact, verify.
The broader lesson is simple: AI hallucinates, and does so confidently, and maybe, more frequently than we may notice. As a result, anything of consequence should pass through human expert vetting if it comes from AI tools, or, where possible, skip the AI altogether.
Other News: OpenAI says GPT-5 is showing stronger performance on medical benchmarks as the company introduces new mental health guidelines aimed at making AI responses safer in sensitive conversations.
Read the full article here
