Clinical Image Analysis Pathway — Design Spec¶

Issue: curaway-ai/curaway-backend#347 Date: 2026-04-23 Scope: Distinguish clinical photos (wounds, X-rays) from document scans in the OCR pipeline. Add observational analysis for clinical images.

Problem Statement¶

When patients upload clinical photos (wound images, X-rays, ultrasound screenshots), the OCR pipeline treats them as documents to read text from. A wound photo has no text — OCR finds nothing — observation_count: 0. The agent then claims to have "reviewed reports" with no data.

Real case: CRW-2026-00310 — patient uploaded 12 WhatsApp images including wound photos of venous leg ulcers. All yielded 0 observations. The agent fabricated laterality ("right iliac vein" when patient had left).

What Clinical Image Observations Look Like¶

For a wound photo like the one from CRW-2026-00310, the system should capture:

Clinical Photo Observations:
- Body region: Lower leg / ankle
- Laterality: Left (based on patient context)
- Image type: Clinical photo (wound)
- Observations:
  • Severe skin breakdown with multiple open wounds
  • Discoloration consistent with venous insufficiency (dark/purplish skin changes)
  • Active wound discharge visible
  • Tissue maceration and peeling
  • Clinical setting visible (hospital bed, IV equipment)
- Image quality: Clear

These observations are descriptive, not diagnostic. They describe what is visually present. The triage agent can reference them: "I can see the wound photos showing skin breakdown on your left leg." A provider reviewing the case gets visual context alongside the medical reports.

Design Decisions¶

1. Detection: How to distinguish clinical photos from document scans¶

After OCR returns insufficient text from an image (< 10 words), classify the image: - Document scan (lab report, prescription, doctor's note photographed) → re-attempt with enhanced OCR / different tier - Clinical photo (wound, X-ray, scan, physical exam) → route to clinical observation pathway

Detection method: Claude Vision with a classification prompt. The same vision call that observes the image can also classify it. One LLM call, not two.

2. Observation prompt: What language to use¶

CRITICAL — Curaway does NOT practice medicine. The observation must be: - Descriptive: "visible wound with discharge" — NOT "infected venous ulcer" - Observational: "skin discoloration" — NOT "cyanosis" or "venous stasis" - Neutral: "multiple open areas" — NOT "Stage 3 pressure injury"

However, the observations must be clinically useful for providers. Purely lay descriptions like "owie on leg" are useless. The sweet spot is informed observation without diagnosis:

3. EXIF stripping: Privacy protection¶

Patient photos from phones contain EXIF metadata: GPS coordinates, device model, timestamp, sometimes user name. Must strip before storage.

• Strip on upload, before R2 storage
• Use Pillow if available, graceful fallback if not
• Log that stripping occurred (audit trail)

4. Camera capture on mobile¶

Current state: <input type="file"> with no capture attribute opens file picker (gallery, camera, or files). This is correct as the default.

For wound/clinical photos specifically: Consider adding a separate "Take photo" button alongside "Attach file" in a future iteration. NOT in this spec — this is backend + pipeline work only.

5. How observations flow to the triage agent¶

Observations stored as clinical_image_observation entities in extracted_data. The existing _get_extracted_findings_summary() (in intake.py) reads extracted_entities from documents — clinical observations flow through the same channel.

The triage agent receives:

[Clinical findings from uploaded documents]
--- wound_photo.jpg (clinical photo) ---
  - Clinical photo: lower leg / ankle (left)
    • Severe skin breakdown with multiple open wounds
    • Discoloration consistent with venous changes
    • Active wound discharge visible
    • Tissue maceration and peeling
    • Clinical setting (hospital bed, IV equipment)

6. Feature flag¶

clinical_image_analysis — default: enabled. Disable via Flagsmith or env var FF_CLINICAL_IMAGE_ANALYSIS=false.

Architecture¶

New Components¶

Modified Components¶

Processing Flow¶

Image uploaded
  ↓
Strip EXIF metadata
  ↓
Image OCR (Claude Vision — extract text)
  ↓
Sufficient text? (≥100 chars + ≥10 words)
  ├── YES → Regular post-OCR pipeline (Clinical Context Agent → FHIR)
  └── NO → Is clinical_image_analysis flag enabled?
        ├── NO → Mark as low_quality (existing behavior)
        └── YES → Claude Vision clinical observation prompt
              ↓
            Returns image_type?
              ├── "other" (not medical) → Mark as low_quality
              └── Clinical type → Store observations as extracted_entities
                    ↓
                  Set analysis_status = "completed"
                  Set ocr_method = "clinical_photo_analysis"
                  Emit SSE progress event
                  Skip Clinical Context Agent (no text to analyze)

Clinical Observation Output Schema¶

{
  "analysis_type": "clinical_image",
  "extracted_entities": [{
    "type": "clinical_image_observation",
    "body_region": "lower leg / ankle",
    "laterality": "left | right | bilateral | not_determinable",
    "image_type": "clinical_photo | xray | ct_scan | mri | ultrasound | other",
    "observations": [
      "Severe skin breakdown with multiple open wounds",
      "Discoloration consistent with venous changes (dark/purplish)",
      "Active wound discharge visible",
      "Tissue maceration and peeling",
      "Clinical setting visible (hospital bed, IV equipment)"
    ],
    "image_quality": "clear | acceptable | poor | unusable",
    "source": "clinical_photo_analysis"
  }],
  "observations": [],
  "demographics": {}
}

Claude Vision Prompt¶

You are reviewing a medical image uploaded by a patient seeking cross-border medical care. 
Describe ONLY what is visually observable — use informed observational language, not diagnostic terms.

Return a JSON object:
{
  "body_region": "the body part or anatomical area visible",
  "image_type": "clinical_photo | xray | ct_scan | mri | ultrasound | lab_report_photo | other",
  "laterality": "left | right | bilateral | not_determinable",
  "observations": ["list of descriptive observations — what is visually present"],
  "image_quality": "clear | acceptable | poor | unusable",
  "clinical_setting": true/false
}

RULES:
- DESCRIPTIVE: "visible wound with discharge" — NOT "infected ulcer"
- OBSERVATIONAL: "skin discoloration, dark purplish changes" — NOT "venous stasis dermatitis"
- NEUTRAL: "multiple open areas on skin surface" — NOT "Stage 3 pressure injury"
- Be specific about location, size approximation, color, texture, number of affected areas
- Note if clinical setting is visible (hospital, clinic, home)
- If you cannot determine laterality from the image alone, say "not_determinable"
- If the image is not medical/clinical, return {"image_type": "other"}

Compliance Notes¶

Edge Cases¶

• Non-medical image (selfie, landscape) → image_type = "other" → mark as low_quality, not clinical
• Blurry/unusable image → image_quality = "unusable" → mark as low_quality with message "Image too unclear for analysis"
• X-ray / CT scan photo → different observation focus (bone structure, organ contours) — prompt handles via image_type
• Multiple body regions in one image → observations cover all visible regions, body_region notes "multiple"
• Image with text overlay (WhatsApp annotations, date stamps) → OCR may extract some text → if sufficient, goes through normal pipeline; if not, clinical analysis
• Pillow not installed → EXIF stripping skips gracefully, logs warning — image processed with metadata intact
• Claude Vision API failure → falls back to low_quality status (existing behavior)
• Very large image (>10MB) → should be resized before sending to Claude Vision to reduce cost
• Feature flag disabled → existing low_quality behavior unchanged

What This Does NOT Include¶

• Camera capture button on mobile (future UX iteration)
• Wound measurement / size estimation (requires calibration reference)
• Image comparison over time (before/after)
• Automated wound scoring (BWAT, PUSH scores) — this would be practicing medicine
• Provider-side image annotation tools