The Stakes Are High
AI tools like ChatGPT and Claude can reduce burnout and improve workflows. They can also destroy your career if you cross the wrong line. The boundary between "innovative efficiency" and "malpractice risk" is clearer than you think.
This guide shows you exactly where that line is.
The Absolute Don'ts (Red Lines)
The Non-Negotiable Rule
Never enter protected health information into any public AI tool. Not names, not MRNs, not dates of birth, not any combination of identifiers. This is not a gray area.
DON'T: Input Protected Health Information (PHI)
Unless you're using a BAA-signed (Business Associate Agreement) enterprise instance provided by your hospital, never paste patient identifiers into a chatbot.
What counts as PHI: Names, MRNs, dates (birth, admission, discharge), detailed geographic data, phone numbers, email addresses, or any combination of diagnosis + age + location that could identify someone.
Why this matters: Public LLMs may use your inputs to train future models. Your patient's case history could be memorized and regurgitated later. Even if the company says they don't train on your data, policies change. Don't bet your license on a privacy policy.
DON'T: Input Sensitive Research or Institutional Data
PHI isn't the only protected content. When you use a public LLM, the company potentially has access to everything you input.
| Data Type | Why It Matters | Consequence |
|---|---|---|
| Unpublished research data | Your novel findings before publication | Lose competitive advantage, risk being scooped |
| Grant proposals | Innovative ideas and methodologies | IP exposure before submission |
| Peer review manuscripts | Someone else's unpublished work | Violate confidentiality, breach publisher policy |
| Institutional IP | Proprietary methods, strategic plans | Legal liability, employment breach |
The solution for sensitive work: Use local LLMs that run entirely on your computer.
- Ollama (free, open source): Download models like Llama 3 or Mistral. Zero data leaves your machine.
- LM Studio (free): User-friendly interface for running local models.
These aren't as capable as GPT-4 or Claude, but they're sufficient for drafting, editing, and brainstorming when confidentiality matters. Use public LLMs for generic tasks. Use local LLMs when confidentiality matters.
DON'T: Use AI for Unsupervised Clinical Decisions
LLMs are prediction engines, not truth engines. They hallucinate — they make up facts, guidelines, drug interactions.
The risk: If an AI makes an error and you act on it, you are liable for malpractice. Not the AI company. You.
Do not ask "What is the chemotherapy dosage for a 12kg child with Ewing Sarcoma?" and act on the answer without verifying it in a formulary.
DON'T: Rely on AI for Recent Guidelines
Most models have a knowledge cutoff date. They may not know about a landmark trial published last month that changed the standard of care.
Use tools with web-browsing (Perplexity, Gemini) that can check live data, but still verify the primary source.
The Dos (Green Lights)
DO: Use AI for Grunt Work and Administration
This is the sweet spot. AI excels at formatting, summarizing, translating language.
Examples: Drafting appeal letters, summarizing meeting notes, converting bullet points into narrative text, formatting citations for a paper.
DO: Use AI as a Thinking Partner
AI is excellent at broadening your differential. It acts as a "second opinion" generator to ensure you haven't developed tunnel vision.
Workflow: "I have a patient with symptoms X, Y, and Z. I'm thinking it's A or B. What other rare etiologies should I consider? List 5 possibilities ranked by probability."
Benefit: It helps you pause and think, "Did I rule out Lupus?"
DO: Use AI for Patient Education
Physicians notoriously overestimate patient health literacy. AI is a master translator.
Workflow: Paste a complex procedure description and ask: "Rewrite this at a 5th-grade reading level using simple analogies."
Benefit: Improves adherence and patient understanding.
DO: Verify Everything
Treat AI output like a note drafted by a sub-intern. It might be brilliant, or it might be dangerously wrong. Your job is to read it, correct it, and sign it. Once you sign it, it becomes your work.
The Maybe Zones (Proceed with Caution)
Drafting Clinical Notes
Risk: Temptation to leave in hallucinations or generic fluff that creates medico-legal bloating.
Guideline: If you use AI to draft a note (using de-identified input), rigorously edit it. Ensure physical exam findings match what you actually did. Do not let AI document "PERRLA" if you didn't check the eyes.
Research and Data Analysis
Risk: AI can write code to analyze data, but it can also hallucinate statistical methods or misinterpret your dataset.
Guideline: You need to understand the logic of the code it writes. If you can't explain the statistical test used, don't publish the result.
Peer Review (Mostly Forbidden)
Major publishers (Nature, Science, NEJM, JAMA, Elsevier) explicitly prohibit using AI to review manuscripts. If you paste someone else's unpublished manuscript into ChatGPT, you're violating both the journal's policy and the author's confidentiality.
The responsible way: Dictate your thoughts as you read the paper. Record voice notes with your critique. Then use AI to structure your ideas into coherent reviewer comments. Never paste the manuscript itself.
Why this matters: Peer review depends on confidentiality. Manuscripts under review contain unpublished data that could be scooped. Respecting this boundary is non-negotiable.
Summary: The Pre-Prompt Checklist
Before hitting "Enter" on any prompt:
The Three Questions
1. Is there PHI in this? (If yes → Delete it)
2. Am I asking for a fact or medical decision? (If yes → Verify the output externally)
3. Would I be comfortable defending this output in M&M conference? (If no → Don't use it)
AI is a bicycle for the mind. It makes you faster and more efficient, but you still have to steer and pedal.