AI: On the Confidence of Machines

There is a particular kind of confidence that troubles me more than uncertainty ever could: the confidence of an answer that sounds thoroughly authoritative but has never once looked you in the eye. It is a confidence without consequence, offered by something that will never sit with you through a difficult result, never notice that your hands have started to shake, and never once say, quietly, “I’m not sure – let’s find out together.” And yet it is precisely this kind of confidence, dispensed at two in the morning by a chatbot with impeccable grammar and no medical license whatsoever, that a growing number of my patients now bring with them into my practice.

They arrive, more often than not, having already diagnosed themselves. Somewhere between midnight and dawn, worried and unable to sleep, they have typed their symptoms into a box with something approaching the trust once reserved for a physician who had actually trained for a decade to earn it. I understand the appeal entirely, and I would be a sorry sort of naturopathic doctor if I pretended otherwise. The chatbot is immediate. It is free. It never sighs, never glances at the clock, never seems the least bit impatient with a third follow-up question at an ungodly hour. When you are frightened about your health in the wee hours of the night, these are not trivial virtues. They are, in fact, rather seductive ones.

But seduction and soundness are not the same thing, and here the evidence is sobering. A study published this year in *BMJ Open*, which examined five of the most widely used AI chatbots, found that very nearly half of their health-related answers were, in the researchers’ own careful phrasing, problematic; that one in five was problematic to a serious degree. What struck me most, reading it, was not the error rate itself, dispiriting though that is, but the manner in which the errors were delivered: fluently, confidently, almost without exception, and entirely regardless of whether the underlying evidence actually supported what was being said. Of two hundred and fifty questions put to these systems, a chatbot declined to answer on precisely two occasions. Two. The machine, in other words, very rarely admits to not knowing. It simply proceeds, in polished sentences, as though it does.

This, I think, is the crux of the matter, and it is worth dwelling on rather than rushing past. An algorithm cannot distinguish between a symptom that is trivial and one that is the herald of something graver, because it has never taken a proper history from you. It has never watched you cross a room, never noticed the wince you tried to disguise as a shrug, never caught the particular pallor that a trained eye recognizes instantly and a screen cannot register at all. It has not asked the follow-up question that, in a real consultation, so often turns out to be the one that changes everything: the offhand remark about a new medication, or the detail about your mother’s health at your age that you nearly didn’t think to mention. It has no access to your bloodwork, your genetics, your sleep, your accumulated stress, or the quiet, cumulative story that your body has been telling for months before you finally sat down to type a question into a box. What it offers instead is pattern-matching, dressed very convincingly in the clothes of insight.

This matters to me particularly, because the entire premise of functional and naturopathic medicine is to look beneath the presenting symptom toward its root: hormonal, nutritional, structural, or otherwise, and to attend to the whole person rather than simply to the label attached to their complaint. That is a slow, individualized, occasionally untidy sort of reasoning, built from years of sitting opposite real people and learning to notice what they do not say as much as what they do. It is precisely the kind of reasoning that a generalized model, trained above all to produce a plausible-sounding paragraph on demand, was never designed to perform. Self-diagnosis by algorithm does not merely risk being wrong, which would be bad enough. Even worse, it risks reassuring you, confidently and fluently, in exactly the wrong direction, and thereby delaying the proper investigation you actually needed.

None of this is an argument against curiosity. Quite the opposite. I encourage it, and I would far rather sit across from a patient who has done some reading than one who has done none. There is a meaningful and, I think, an important difference between researching a topic in order to bring sharper, better-informed questions to your appointment, and outsourcing the diagnosis itself to a system that has no stake whatsoever in getting it right, and no notion of who you actually are. The first makes you a better collaborator in your own care. The second asks a stranger with no face and no memory to make a decision that ought, by rights, to involve both.

There is an old and rather unfashionable idea in medicine that the consultation itself is part of the treatment; that being properly listened to, examined, and understood is not merely a preamble to the real work but a form of care in its own right. This is something I firmly believe, and it happens to be the one thing a chatbot, however articulate, can never provide. It can generate an answer. It cannot generate attention.

Therefore, if something in your body has been troubling you, quietly, persistently, in a way you cannot quite identify, please do not let a chatbot have the final word on it. Bring your questions, your worries, and your history to a conversation that will actually come to know you as a whole person rather than a string of symptoms. Unlike the algorithm, it will look you in the eye.

Book your free 15-minute discovery call — by phone or video

Sources

“Generative artificial intelligence-driven chatbots and medical misinformation: an accuracy, referencing and readability audit,” *BMJ Open*, 2026.