Module 4 Handout — Critical Appraisal, Bias & Clinical Autonomy

How this module works. Take the short Preview Quiz first — five minutes, ungraded. Then work through this handout (or the companion slide deck) at your own pace. Section B asks you to pull up real outputs you generated in Modules 2 and 3 — have those handy. Section E is a worked simulation case — form your own answer before reading the AI's. Finish with the Post-Test.

By the end of this module you will be able to:

Identify hallucination, overconfidence, and demographic or algorithmic bias in real AI outputs.
Apply a structured routine for interrogating AI outputs — especially when they confirm your own initial impression.
Articulate the legal and professional boundary of physician accountability when using AI-assisted tools.

Section A — Why This Module Exists: The Confirmation-Bias Trap (KNOW)

Every module so far has ended the same way: with a warning that scrutiny drops exactly when an AI's output matches what you already believed. Module 1 named it. Module 3's case made it concrete — a differential that felt plausible enough to accept without a second look. This module is where that warning becomes a trained habit instead of a recurring footnote.

There's a reason this comes after Modules 2 and 3 rather than before them: by now you have real AI outputs of your own — a documentation note, a differential — to examine, instead of a hypothetical example. Critiquing your own work is a different exercise than critiquing someone else's, and it's the one that actually changes behavior.

Two ideas from outside clinical medicine describe this trap precisely, and it's worth having language for them:

The "sovereignty trap"

An AI's confident, fluent tone tempts a person to defer judgment to it — and the temptation is strongest exactly when the tool's answer already matches the person's own gut feeling. The agreement feels like confirmation. It's actually the moment least scrutiny gets applied.

The "hollowed mind"

When an answer arrives with no friction — instantly, fluently, confidently — the effortful cognitive work of actually reasoning through a problem gets quietly skipped. Repeated over years, the opportunities to refine clinical judgment narrow, and "reviewing" the AI's output becomes performative — a glance, not a real check — rather than the substantive second look it's supposed to be.

Neither of these is about AI being unreliable in some abstract sense. They're about a specific, human, entirely predictable pattern: the more convenient and confident an answer feels, the less most people naturally interrogate it. This module is built to counter that pattern directly, through effortful practice rather than passive reading — which is why Sections B and E ask you to do something, not just read something.

Section B — Spotting Hallucination and Overconfidence in Your Own Outputs (THINK)

This section is a self-audit, not a reading exercise. Pull up two things you already produced in this course: the AI-drafted note from Module 2 Section C (the hypertension follow-up visit), and the AI-generated differential from Module 3 Section D (the Dili fever case). You already found the planted errors in both once, working through them the first time. This time, apply a more general lens — not "what was wrong here" but "what pattern of wrongness was this."

1. In the Module 2 note: which errors were fabrications (something invented that wasn't said), and which were omissions (something true that got dropped)? Did one type feel easier to catch on a first read than the other?

2. In the Module 3 differential: the AI's top-ranked answer was fluent and specific ("most likely, self-limiting"). Did that fluency and specificity make it feel more trustworthy than it deserved, even in hindsight?

3. In both cases, was there a moment where the AI's output was almost right — close enough to be genuinely persuasive — rather than obviously wrong? What made the near-miss more dangerous than an obvious error would have been?

4. Think of one AI output from your own practice, outside this course. Using the same lens, would you catch a fabrication or an omission in it today — and how confident are you, honestly, that you would?

In the live seminar, you'll compare notes with colleagues on what each of you found — the differences in what people caught and missed are usually the most instructive part of this exercise.

Section C — Algorithmic and Demographic Bias: How It Gets Into Clinical Tools (KNOW)

Module 3 introduced validation gaps — a tool underperforming because its training population differs from yours. Bias is the same underlying mechanism, viewed from a different angle: not just "this tool wasn't tested on people like my patients," but "this tool's outputs are systematically skewed in a particular direction, for particular groups, because of how it was built." Three sources account for most of it:

None of these announce themselves. A biased tool doesn't output a warning — it outputs a confident number or ranking, identical in tone and format to an unbiased one. This is the same "fails silently" pattern from Module 3, applied to systematic unfairness rather than random error.

Source	How it shows up in a clinical tool
Training data imbalance	A group is underrepresented in the data the tool learned from, so the tool is systematically less accurate for that group — not randomly wrong, but wrong in a consistent direction.
Historical bias baked into labels	If the data used to "teach" the tool reflects past unequal care (a group historically under-diagnosed or under-treated), the tool can learn to reproduce that same pattern as if it were correct.
Proxy variables	A tool can use a variable that correlates with a protected characteristic (postcode, insurance type, prior healthcare utilization) as an indirect stand-in for it — encoding bias without ever using the characteristic directly.

Why this matters here specifically Most widely available AI tools were built and validated on populations that look very little like Timor-Leste's — different disease prevalence, different nutritional baselines, different access histories. That's not a hypothetical equity concern; it's the concrete, practical reason Module 3's validation-gap habit and this module's bias-awareness habit have to run together, not separately.

Section D — The "Open the Black Box" Routine (THINK)

Every habit this course has built — form your own opinion first, distrust unexplained specificity, check the validation population, watch for the moment of agreement — is really one routine wearing different clothes in each module. This section names it once, as a single checklist you can actually run, on anything, in under a minute.

The Black Box Routine

State your own answer first. Before you read the AI's output, or independent of it — what would you have said?
Ask what it's actually based on. Do you know what data or reasoning underlies this answer, or are you assuming it's sound because it sounds sound?
If it agrees with you, look harder, not less. Agreement is the moment scrutiny naturally drops — which is exactly why it needs to rise instead.
Treat unexplained precision as a flag, not reassurance. An exact percentage, a specific dose, a confident decimal — verify it against a real source rather than letting its specificity stand in for accuracy.
Name one fact the tool couldn't have known. Local context, an exposure history, something from the physical exam — and ask whether it would change the answer.
Decide, document, and own the call. The record should show your reasoning, not "AI recommended X" — because the accountability sits with you either way (Section F).

In the live workshop, you'll run this routine live, out loud, on a case a colleague brings — the goal is to make it fast enough that it fits inside a real clinical encounter, not just a training exercise.

Section E — Simulation: A High-Stakes Case Where the AI Is Confidently Wrong (THINK)

Case — Step 1: the situation It's the end of a long shift. An 18 kg, 6-year-old with known stage 3 chronic kidney disease presents with a febrile urinary tract infection. Local protocol calls for an aminoglycoside as part of the regimen, and aminoglycosides are cleared renally and carry real nephrotoxicity and ototoxicity risk — dosing has to account for the reduced renal clearance. You're tired, the pharmacy is closed, and you ask a general-purpose AI chatbot to confirm the dosing.

Before reading on — what do you already know has to happen before you can safely answer this question yourself?

Case — Step 2: the AI's response

The chatbot replies fluently and specifically: it states the standard pediatric weight-based dose for the antibiotic, adds "a dose reduction of approximately 10% is generally sufficient to account for stage 3 renal impairment in pediatric patients," and confirms the regimen is "safe to proceed with at this adjusted dose." The tone is textbook-confident. No source is cited. No uncertainty is expressed.

Would you have given that dose? What, specifically, made the answer feel trustworthy — and what should have made you stop?

Discussion — what this case is built to show

The specific figure is the tell, not the reassurance. A precise-sounding "10% reduction" is exactly the kind of plausible-value filling flagged back in Module 1 — a number invented to sound authoritative, not one drawn from a verified pediatric renal-dosing reference.
Renal dosing for nephrotoxic drugs isn't a simple percentage adjustment. It typically depends on the drug, the degree of renal impairment, and interval as well as dose — the kind of detail a real protocol or pharmacist consult addresses and a generic chatbot answer glossed over entirely.
No source, no uncertainty, full confidence — at the end of a long shift, that combination is precisely when the sovereignty trap from Section A is strongest. The tiredness doesn't make the AI more reliable; it makes the clinician less likely to apply the Black Box Routine.
The correct move was never "trust or don't trust the number." It was to treat a nephrotoxic pediatric drug dose in renal impairment as something that gets verified against a real pediatric renal-dosing reference or a pharmacist — every time, regardless of how confident the source sounds, AI or otherwise.

If your instinct was to verify before administering, that's the routine working. If the confident tone nearly carried you past that step, that's exactly the gap this exercise exists to close — and precisely the scenario the live simulation session will put you through again, live, under time pressure.

Section F — Legal and Professional Accountability: Where the Line Actually Sits (KNOW)

One principle has been implicit since Module 2 and is worth stating outright: using an AI tool does not shift clinical or legal responsibility away from the physician who acts on its output. The tool assists; it does not absorb accountability. That holds whether the AI drafted a note, ranked a differential, flagged an image, or suggested a dose — the clinician who signs, orders, or acts remains the one accountable for that decision.

Layer	What this module put there
KNOW	Why confirmation bias is the throughline risk across every module so far; the "sovereignty trap" and "hollowed mind" as named patterns; how algorithmic and demographic bias gets built into clinical tools; the baseline principle of physician accountability.
THINK	A self-audit of your own AI outputs from Modules 2–3; the Black Box Routine as one repeatable checklist; a high-stakes simulation practicing all of it under realistic pressure.

Critical Appraisal, Bias & Clinical Autonomy