Can AI diagnose you?

Can AI Diagnose You? A Programmer-Patient's Honest Assessment of Claude, ChatGPT, and Medical AI Tools

A skeptical, frustrated, and occasionally hopeful look at what happens when patients turn to AI for medical advice

An acquaintance who has worked as a programmer for many years was expressing her frustration at trying to navigate the US healthcare system for a relatively common medical problem. Over several weeks she began making comparisons between (1) what human healthcare providers can (and cannot) do, versus (2) what AI-powered healthcare queries can (and cannot) do.

Even though her symptoms did not include dizziness, we found her perspective both fascinating and instructive. Her reflections and constructive suggestions deserve a wider audience — among healthcare practitioners, patients, and AI developers alike.

-------------------------------------------------------------------

I've spent the better part of two decades writing software. I debug things for a living. I know what it looks like when a system produces a confident-sounding output that is, underneath the hood, assembled from pattern matching and statistical inference rather than ground truth. I also know what it looks like to sit in a waiting room for ninety minutes past your scheduled appointment, finally get ten minutes with a physician who is clearly managing four other things at once, and walk out with a referral to a specialist you'll wait three months to see, for a problem that may or may not still be relevant by then.

I am, in other words, both a skeptic about AI and a frustrated patient. That's the lens through which I want to examine what's actually happening when people — people who are scared, confused, or just sick of the system — turn to Claude, ChatGPT, Perplexity, Gemini, OpenEvidence, or DoximityAsk and ask: what's wrong with me, and what should I do about it?

The answer is complicated. And the complications matter.

Where AI Medical Tools Actually Do a Good Job

Let's start with honesty, because the picture isn't entirely bleak.

AI is excellent at explaining things a physician never had time to explain. You've just been told you have atrial fibrillation. Your cardiologist, running forty minutes behind and with a waiting room full of people, spent six minutes with you and handed you a prescription for a blood thinner. You understand approximately nothing. You go home and ask Claude to explain what atrial fibrillation is, why the clot risk exists, how anticoagulants work, what the difference is between a rate-control and rhythm-control strategy, and what symptoms should send you to the ER. You will get a clear, patient, reasonably accurate explanation that would have taken your physician twenty minutes to deliver — twenty minutes they don't have and you didn't get. That's genuinely valuable. That is AI filling a real gap.

AI is good at differential diagnosis brainstorming when the user presents clean data. If you describe a classic presentation — say, unilateral leg swelling after a long flight, with calf tenderness — a good AI system will appropriately surface deep vein thrombosis, walk through the Wells criteria, and tell you clearly that this warrants same-day evaluation, not a Google rabbit hole. It won't always volunteer that information in the right order, but it knows it. The knowledge base underlying these systems, at least for common presentations, is solid.

AI is useful for medication reconciliation and interaction checking. Patients on five or more medications — a population that is enormous and frequently poorly served by fragmented care — can get remarkably sensible answers about interactions, food contraindications, and timing. This is one area where AI adds consistent value that isn't easily replaced by a busy pharmacist with a drive-through line.

Specialized tools have real strengths in their lanes. OpenEvidence and DoximityAsk are designed for clinicians and are explicit about that, but patients who stumble into them are accessing systems grounded in indexed medical literature rather than general web scraping. Perplexity, with its citation model, at least shows you where the answer came from — which is more epistemically honest than a confident paragraph with no sourcing.

Where AI Diagnosis Falls Short — and Why It Matters

Now we get to the part that concerns me as a programmer, because I understand the mechanism well enough to know why it fails the way it does.

These systems were not trained to take a history. They take you at your first word.

This is the one that keeps me up at night, professionally speaking. When a physician sees a patient, the first few minutes are not passive. The physician is running a decision tree in real time. You say "chest pain" and they immediately begin asking: where exactly, sharp or pressure, does it radiate, what makes it better or worse, how long has it been happening, do you have diabetes or smoke, what's your family history? Each answer prunes the tree and opens new branches. This process — the clinical history — is the single most powerful diagnostic tool in medicine. Studies suggest it leads to the correct diagnosis more often than physical examination or testing alone.

AI systems, in their default mode, do not do this. You type "I've been having chest pain" and you get a response about chest pain. It might be a thorough response, covering cardiac causes, musculoskeletal causes, gastrointestinal causes, anxiety — but it is a response to the words you typed, not to the clinical picture you're actually presenting. The system didn't ask whether the pain is worse when you press on your sternum (musculoskeletal, probably not dangerous) or whether it radiates to your jaw and left arm and came on while you were climbing stairs (call 911 now). A patient who doesn't know that distinction is a meaningful one will not volunteer it. And most patients don't know.

Consider a patient who mentions rectal bleeding to an AI — but volunteers, in the same breath, that they have a history of hemorrhoids. The AI, working only with what it has been given, will very likely validate the hemorrhoid hypothesis and discuss conservative management: sitz baths, fiber, topical creams. What the patient didn't think to mention — because they didn't know it was relevant — is that the bleeding pattern has recently changed: darker, more frequent, accompanied by a subtle shift in bowel habits they've attributed to a change in diet. Those details, systematically elicited by a careful clinician, would raise the suspicion for colorectal cancer and trigger a very different conversation. The patient wasn't being careless or incurious; they simply led with the explanation they already had, never knowing that the thing they omitted was the thing that mattered most.

The physician knows what they don't know and asks for it. AI knows what it knows and retrieves it. These are not the same thing.

Patients are unreliable narrators of their own symptoms, and AI has no mechanism to compensate for that.

This is not a criticism of patients — it is a well-documented feature of how humans experience and report symptoms. Pain is subjective. Timelines are misremembered. People minimize symptoms they're afraid of and amplify symptoms they're worried about for other reasons. They describe a chief complaint, but omit what they've decided isn't relevant, not knowing that the thing they omitted is the thing that changes everything.

A physician examining you has multiple channels of information: what you say, how you look, what they observe on examination, what your vital signs show. They can notice that you said you're "fine" while you're guarding your abdomen. AI has only what you type. And what you type is, almost by definition, an incomplete and sometimes distorted representation of your actual clinical state.

Confident delivery is a feature, not a bug — and that's a problem.

These AI systems are optimized, at some level, to be readable and useful. That means they produce prose that sounds authoritative. The tone of a response from Claude or ChatGPT about your symptoms reads the way a knowledgeable doctor friend might sound — measured, clear, organized. This is part of what makes them useful for explanation, but it is exactly what makes them dangerous for diagnosis. The user has almost no signal to distinguish "this is well-established medical consensus" from "this is the most statistically likely next tokens given your prompt." Both read about the same.

A physician who is uncertain says "I'm not sure, let's run some tests." AI uncertainty, when it appears at all, is often buried in qualifications at the end of a paragraph that already sounded confident. Most people don't read to the end.

It is worth noting that most of these platforms are aware of this problem — aware enough, at least, to append a disclaimer along the lines of "AI can make mistakes; always consult your healthcare provider." That disclaimer is appropriate and correct. It is also, almost without exception, placed at the end of the response, after the confident, well-organized, authoritative-sounding paragraphs have already done their work on the reader. That placement is not an accident. A disclaimer at the top of a medical response would do what disclaimers are supposed to do — alert the reader to approach what follows with a more critical eye — but it would also create friction, signal fallibility, and undermine the confident tone that keeps users engaged. In healthcare, that friction would be a feature. The AI industry has decided, for reasons that are not hard to guess, to treat it as a bug.

Rare diseases, uncommon presentations of common diseases, and atypical demographics get flattened.

Medical training and medical literature are not equally representative of all patients. AI systems trained on that literature inherit its biases. Women presenting with a myocardial infarction (heart attack) often have atypical symptoms — nausea, fatigue, jaw pain — that look nothing like the "classic" presentation and have historically been misdiagnosed even by physicians. An AI system asked about those symptoms by a woman in her fifties, if it does not specifically ask about cardiac risk factors, may reasonably and incorrectly anchor on gastrointestinal causes. Similarly, rare diseases — by definition — are underrepresented in training data, and a system optimized for statistical likelihood will, when it encounters the rare presentation, keep suggesting common conditions.

OpenEvidence and DoximityAsk were built for clinicians, not patients.

This matters. These platforms assume a user who understands clinical vocabulary, knows how to interpret a probability, and has a clinical context in which to place the answer. A layperson using DoximityAsk to look up their own condition faces two compounding problems, not one. The first is formulation: these platforms are designed to receive precise, clinically structured queries — the kind a physician constructs after already having taken a history, considered a differential, and narrowed the question. A layperson arrives instead with a symptom and a fear, expressed in everyday language, and the gap between "I can't catch my breath and my chest feels strange" and a well-formed clinical query is not trivial. The second problem is interpretation: even when a useful answer is returned, parsing it requires the clinical context and vocabulary that the platform was built to assume. Without both skills — asking the right question and reading the answer correctly — the tool is liable to be not just unhelpful but actively misleading.

How Can a Patient Tell Whether AI Medical Advice Is Any Good?

This is the question I find most troubling, and the one I most wish had a clean answer.

Here is what a layperson cannot assess: whether the AI is anchoring on the wrong diagnosis, whether it has missed a red flag, whether its confident prose reflects robust evidence or pattern-matched text, and whether the key piece of clinical information that changes everything is the piece they forgot to mention.

Here is what they can do:

Ask the AI to steelman its own uncertainty. "What are the most serious diagnoses consistent with what I've described, and what symptoms would suggest those?" This forces the system out of its default mode of producing the most probable answer and into a mode of surfacing the dangerous alternatives. A response that doesn't have an answer to this question is a bad sign.

Notice whether it asked you anything. If you typed three sentences and got four paragraphs of confident diagnostic reasoning, something is missing. A clinician would have asked more questions. If the AI didn't, either prompt it to ask you clarifying questions before continuing, or treat the response as a rough map, not a destination.

Look for red flag language. Good AI responses about symptoms of concern should — and increasingly do — include explicit language about when to seek emergency care, when to call your physician today versus schedule a routine visit, and what symptom changes should trigger escalation. If that language is absent, the response is incomplete.

Use Perplexity if you want to audit the sources. Its citation model is imperfect, but at least you can see what it's drawing from and evaluate whether those sources are credible.

Never let AI replace a physical examination. There are diagnoses that cannot be made without one. An irregular heartbeat, a subtle abdominal rigidity, a fundoscopic finding, a melanoma with the right border characteristics — these require a body in a room with a clinician. AI has no body. This is a hard limit.

Why Patients Turn to AI for Medical Advice in the First Place

Here is the thing that frustrates me most, and it is a systemic frustration rather than a technological one.

People turn to AI for medical advice because the alternative is broken. A primary care appointment takes weeks to schedule in many parts of this country. Urgent care for a non-emergency non-trivial problem costs money and time many people don't have. The ER is for emergencies, but the definition of what constitutes an emergency is not always clear to a frightened layperson at ten o'clock at night. The physician who would actually help is behind a wall of scheduling systems, insurance authorizations, copays, and time constraints.

People reach for AI because the supply-and-demand gap is real and enormous, but they are frequently unaware that AI cannot reliably close it — not because the technology is immature, but because it was never intrinsically designed to do so. It would be easy to say "just see your doctor." For a lot of people, that sentence is not actionable advice.

So we end up in a situation where a powerful but fundamentally limited tool is being used by people who have few alternatives, to answer questions that genuinely require more than the tool can provide. And the people using it — out of necessity, not naivety — have almost no way to know where the tool's competence ends.

What Better AI Medical Tools Would Actually Look Like

If I were designing this from scratch — which, again, is what I do for a living — I would want AI medical systems to do the following things they currently do not reliably do:

Take the history before rendering an opinion. Before generating a differential diagnosis, ask a structured sequence of clarifying questions. What is the location, quality, severity, timing, context, and modifying factors? What associated symptoms are present? What has the user already tried? This is not technically difficult. It is, in fact, trivially implementable. The systems that don't do it are making a product decision, not running into a capability wall.

And we know this is achievable because these same systems already do something functionally identical in a different high-stakes context. Ask any major AI platform a question that signals suicidal ideation — "what is the most painless way to commit suicide?" — and the response does not begin with a confident, well-organized answer to the question as literally asked. It begins with a crisis resource, a hotline number, and an invitation to talk to someone. The system recognized the category of question being asked, classified it as high-stakes, and modified its behavior accordingly — before generating any other content. That is precisely the capability that would be needed to implement interview mode for acute medical presentations. The classifier already exists. The conditional response architecture already exists. What does not yet exist is the decision to apply it to the patient who types "I have chest pain right now and I don't know if it's serious" with the same urgency that it is applied to the person in mental health crisis. Those two users have more in common than the current AI design acknowledges.

Distinguish clearly between "here is an explanation" and "here is a clinical assessment." These are different activities with different epistemic weights, and users deserve to know which one they're getting.

Be more calibrated about confidence. "This is highly likely" and "this is one possibility among several serious ones" should read differently. They often don't.

Surface red flags proactively and prominently. Not buried in paragraph four. At the top.

The Bottom Line

AI is a useful, imperfect, and increasingly unavoidable presence in how laypeople navigate their health. It is better than nothing — often substantially better — when used for explanation, for context, for understanding what a physician has told you. It is a poor substitute for clinical reasoning, particularly for undifferentiated symptoms, atypical presentations, and any situation where the history you don't think to give is the one that matters most.

The best thing a layperson can do is treat AI like a smart friend who read a lot of medical literature but has never actually examined a patient. Ask them to help you understand. Ask them what questions you should be asking your actual doctor. Ask them what the scary possibilities are that you should be aware of. And then, where at all possible, go see the actual doctor — and push hard against the system that makes that harder than it should be.

Despite the honest efforts of hard-working clinicians, despite genuinely remarkable technology, and despite the best intentions of nearly everyone involved, the United States healthcare delivery system is broken. AI is a genuinely powerful tool, but power alone does not guarantee correctness, completeness, or fitness for every purpose. Neither of these things is a reason to give up on getting good care. They are, instead, a reason to be smarter about what each tool is actually good for.

The author writes software, reads too much medical literature for someone without a medical degree, and has strong opinions about waiting rooms.