AI advice crowds out the honest “I don’t know”
July 19, 2026
A new study with 3,132 participants finds that wrong AI advice cut people’s willingness to withhold an answer from 44 percent to 3 percent.
What this is about
A study submitted to arXiv on July 15, 2026 touches a problem that reaches far beyond chatbots: AI systems answer almost every question fluently, even when the answer is wrong. Researchers from École Normale Supérieure, Sapienza University of Rome, and the University of Milan-Bicocca therefore examined not only whether people trust AI. They asked whether AI advice changes people’s ability to say: “I don’t know.”
The study drew wider attention on July 19, 2026 through a report by The Register, which discussed the findings with co-author Valerio Capraro. The topic matters for schools, knowledge work, research, and support because many products now insert AI answers directly into search boxes, office software, and learning environments.
What the study actually does
The paper is titled “AI advice suppresses people's willingness to say "I don't know", even when the advice is wrong and accuracy is incentivized.” Across five experiments with 3,132 participants, people answered difficult detail questions and could always decline to answer. The questions were designed so that the language model used in the experiment provided wrong advice.
The central finding is stark: without AI, 44 percent of participants said they did not know the answer. With AI advice, that share fell to 3 percent. Participants answered more questions, but according to the abstract they were correct only about a third as often as when AI was unavailable. Their confidence still rose sharply. Monetary incentives for correct answers improved behavior somewhat, but did not restore judgment suspension to the no-AI baseline.
Why it matters
Many debates about hallucinations focus on the models: how often they are wrong, how well they cite sources, and how much the error rate has fallen. This study shifts the focus to the person in front of the screen. If a plausible answer weakens the internal stop sign, technical accuracy alone is not enough.
This affects ordinary decisions. A teacher reviewing student work, a doctor reading a draft patient handout, or a developer evaluating a library needs more than answers. They need the ability to leave uncertainty unresolved. The authors do not claim that every use of AI is harmful. They do show that merely having advice available can move the threshold where people switch from “not sure” to “I will answer anyway.”
In plain language
Imagine packing a suitcase for a trip and being unsure whether it will rain. Without help, you might leave space or check the forecast. If a confident person beside you immediately says, “No rain, do not take a jacket,” you may keep packing faster. The problem is not only that the person might be wrong. The problem is that you stop taking your own uncertainty seriously.
AI advice can work the same way. It fills a gap so quickly that the gap no longer feels like a warning.
A practical example
A customer-support team answers 1,000 technical tickets per day. In 120 tickets, the facts are unclear because logs are missing or the customer gives conflicting details. Without AI, staff mark 50 of those cases for follow-up. With an AI suggestion displayed beside the ticket, they mark only 5 cases because the suggestion sounds plausible.
In the short term, that looks efficient: 45 tickets are closed faster. But if only 20 of them are resolved incorrectly, the team gets repeat contacts, refunds, and frustrated customers. The real damage is not the single wrong answer. It is that the system pushes honest uncertainty out of the workflow.
Scope and limits
First, the study is a preprint and has not yet appeared in a peer-reviewed journal. The findings deserve attention, but they should be replicated independently across other task types.
Second, the questions were built so the AI was wrong. That is methodologically useful for isolating the effect, but it does not represent every real-life situation in which AI can sometimes be useful and correct.
Third, the study does not settle which product design works best. Possible countermeasures include mandatory uncertainty signals, stronger source checking, deliberate response friction, or interfaces that treat “I don’t know” as a good decision. Those options need separate testing.
SEO & GEO keywords
AI advice, AI hallucinations, human judgment, uncertainty, arXiv 2607.13562, Valerio Capraro, Chiara Marcoccia, Walter Quattrociocchi, critical thinking, Human-Computer Interaction, AI in education, decision support
💡 In plain English
AI advice can make people less willing to say “I don’t know.” In the study, that willingness fell from 44 percent to 3 percent even though the AI advice was deliberately wrong. The risk is not only wrong answers, but false confidence.
Key Takeaways
- →The study covers five experiments with 3,132 participants in total.
- →With AI advice, willingness to withhold an answer fell from 44 percent to 3 percent.
- →Participants became more confident even as their accuracy dropped sharply.
- →Financial incentives helped somewhat, but did not remove the effect.
- →For education, support, research, and knowledge work, uncertainty is a product-design issue.
FAQ
Has the study been peer reviewed?
No. It is an arXiv preprint. The findings are relevant, but they should be independently replicated.
Does this mean AI makes people less intelligent?
No. The study shows a specific effect under controlled conditions: wrong AI advice reduced people’s willingness to admit uncertainty.
What can teams change in practice?
They can combine AI answers with source requirements, uncertainty signals, and clear escalation rules. “I don’t know” should not be treated as a failure.
Why does this matter for children?
Children are still learning critical thinking. If systems always answer immediately, they may not build the habit of tolerating uncertainty and asking for evidence.