Medical Advice Audit — Steer Document¶

Date: 2026-04-08 Author: Srikanth Donthi (CPO/CTO) + Claude Code session 34 Status: Audit complete — remediation implemented (PR #84). Ongoing via CI guard. Companion spec: medical-advice-remediation-feature.md

1. Why this exists¶

Curaway is a cross-border medical travel coordination platform, not a medical practice. The platform must never give medical advice. This is both a regulatory requirement (FDA / CE / UAE Ministry of Health all draw the line at "is the system telling the patient what to do clinically?") and a brand requirement (we are an intake coordinator, not a doctor).

This document is an honest audit of where the platform currently sits on the medical-advice line — what's safe, what's gray, what's borderline. The accompanying feature spec captures the proposed remediation but is not implemented yet because clinical wording should never be merged without review.

2. What's enforced (the safe surface)¶

The platform has six layers of protection against medical advice in the chat interface:

Verdict on the chat conversation layer: safe. A patient messaging the agent should never receive "you have X, you should do Y" output. The guardrails are real, tested, and enforced at runtime + CI.

3. Where it gets gray (the audit findings)¶

Five features ship clinical content to patient-visible surfaces (not just internal LLM reasoning) and contain text that reads like a clinical recommendation even though it's framed as "what providers need".

Finding 1 — Risk assessor mitigation field 🚨 HIGHEST CONCERN¶

Source: app/services/risk_assessor.py (Session 33, PR #68) Visible at: EHR drawer Risk Assessment section, EHR side panel top-3 risks Patient sees: Yes, in the Health Record drawer

The rule set has ~20 mitigation strings that read as imperative clinical instructions:

Why this is the highest concern: - The verbs are imperative: "Optimize", "Hold", "Avoid", "Bring", "Correct", "Coordinate" - A reasonable patient reads this as "Curaway is telling me to lower my HbA1c / hold my warfarin / get a cardiology clearance" - This is the closest the platform comes to actually giving medical advice - The text was written by me (Claude) during the risk assessor build (PR #68) — no clinical review

Defence (weak): The text is intended as "what providers consider" and the EHR Risk Assessment section header says "Risk Assessment". But the language doesn't reflect that intent.

Finding 2 — Auto-detected comorbidities labels 🚨 DIAGNOSTIC LABELING¶

Source: app/agents/lab_analyzer.py (Session 21) Visible at: EHR drawer comorbidities section, summary panel, side panel Patient sees: Yes

lab_analyzer.py reads uploaded lab results and produces entries like:

• "Prediabetes — detected from HbA1c 6.3%"
• "Mild anemia — detected from Hemoglobin 11.2"
• "Reduced kidney function — detected from eGFR 58"
• "Iron deficiency — detected from Ferritin 12"
• "Hyperlipidemia — detected from LDL 142"

Why this is a concern: - These are diagnostic labels assigned to a patient based on a single lab value, displayed in the patient's health record - A clinician would never label a patient with "prediabetes" off a single HbA1c without context (was the patient fasting? on steroids? hydrated?) - The framing reads as a diagnosis the system is assigning to the patient

Defence (medium): The thresholds are conservative and clinically defensible (HbA1c 5.7–6.4% is the standard prediabetes range). But the verb "detected" makes it sound diagnostic.

Finding 3 — Records-first required tests reply¶

Source: Session 34 records-first wiring (PR #67), Neo4j REQUIRES_TEST data Visible at: Conversation chat reply, document checklist card Patient sees: Yes (in chat)

After identifying TKR, the conversation agent now produces replies like:

"You'll need a CBC from the last 30 days, a Basic Metabolic Panel within 30 days, an HbA1c within 90 days, an ECG within 90 days, and a chest X-ray within 180 days."

Why this is borderline: - Telling a patient "you'll need a CBC" is procedural-sounding but prescriptive in form - The information is sourced from Neo4j REQUIRES_TEST (which is provider-supplied, not invented by Curaway), so the content is safe - Only the framing is gray

Defence (strong): This is closer to "package contents" than medical advice. Hospitals require these tests for case review. The fix is purely linguistic — change "you'll need" to "providers typically request".

Finding 4 — CoT clinical reasoning output¶

Source: Session 34 (PR #76), CLINICAL_ANALYSIS_SYSTEM_PROMPT_COT and ICD_MAPPING_SYSTEM_PROMPT_COT Visible at: Internal only — events table + Langfuse + Match Agent input Patient sees: No

The new CoT prompts produce reasoning like:

{
  "pathway": "TKR right, pre-op optimization (target HbA1c <7.0)",
  "contraindications": "none absolute",
  "comorbidity_interactions": [
    {
      "conditions": ["type_2_diabetes", "obesity_class_1"],
      "interaction": "compound_wound_infection_risk",
      "severity": "moderate",
      "detail": "HbA1c 6.8 + BMI 31 elevates infection risk but both below surgical thresholds"
    }
  ]
}

Why this is fine for now: The CoT output is consumed only by the Match Agent's reranker and stored in events for future LangSmith eval datasets. No frontend renders it.

Why it's fragile: There is no test enforcing that this stays internal-only. If a future PR wires comorbidity_interactions into a frontend rich card, we'd be giving overt clinical reasoning to patients. The CoT output contains exactly the kind of language patients should not see ("target HbA1c <7.0").

Finding 5 — On-site tests notice (#64, this session)¶

Source: Session 34 (PR #64), _build_on_site_tests_notice in case_orchestrator.py Visible at: Confirmation rich card after consent + forwarding Patient sees: Yes

"A few tests will be performed at the hospital when you arrive — even if you've already shared results. This is standard pre-op protocol."

Verdict: safe. This is procedural information about what the receiving hospital will do, not advice about what the patient should do. It's telling the patient what to expect, not what to do.

4. Risk classification¶

5. The honest summary¶

Intended: No medical advice. Multiple guardrails in the chat interface, all real and enforced.

In practice: The chat interface is clean. But several rich-card surfaces — the EHR Risk Assessment, the auto-detected comorbidities, the records-first reply — show patient-facing clinical content that a reasonable observer (or regulator) could call "soft medical advice".

The two highest-risk surfaces (risk assessor mitigation field and auto-detected comorbidity labels) were both written without clinical review. The text is plausible but it crosses the line in tone if not in letter.

This is fixable with linguistic changes plus a CI guard, not a fundamental redesign. See the companion feature spec for the remediation plan.

6. Out of scope for this audit¶

• Match explanations ("Apollo Chennai performs 450 TKRs/year with 94% success rate") — these are factual provider data, not clinical advice. Safe.
• Cost ranges and recovery timelines — sourced from Neo4j metadata, not Curaway clinical judgment. Safe.
• Provider selection — pure data ranking, no clinical recommendation language. Safe.
• Document validity windows — sourced from REQUIRES_TEST.validity_days in Neo4j, no Curaway clinical judgment. Safe.
• Clinical Context Agent extraction (ICD codes, FHIR resources) — these go into structured records that flow to providers, not into patient-facing chat. Safe.

7. What needs clinical review (not Claude review)¶

Three categories of strings need a medical professional to look at the proposed remediation wording before merge:

1. The 14 risk assessor mitigation strings (Finding 1)
2. The 6 lab_analyzer comorbidity detection labels (Finding 2)
3. Any wording change to the records-first reply that changes meaning (Finding 3 — pure framing change, lower risk)

Suggested reviewer: Dr. Shrikanth Naidu (Clinical Advisor, listed in CLAUDE.md). Pre-Series A: this is the right level of clinical review for the volume of patient-facing clinical text we have. Post-Series A: this needs to become a formal sign-off process per release.

8. References¶

• Sessions that introduced patient-facing clinical content:
• Session 21 — lab_analyzer.py auto-detected comorbidities
• Session 33 (#68) — risk_assessor.py mitigation strings
• Session 34 (#67) — records-first required tests reply
• Session 34 (#76) — CoT prompts (internal only — no patient surface)
• Existing safety infrastructure:
• Session 24 — Agent guardrails + file upload validation
• config/guardrails.yaml — output validation rules
• config/voice_rules.yaml — forbidden phrases
• tests/test_voice_compliance.py — CI scanner
• Companion docs:
• medical-advice-remediation-feature.md — implementation spec
• ADR-0013: Rule-Based Pre-Operative Risk Assessor — risk assessor design rationale