Skip to content

Test-Data Tagging Runbook

Overview

QA and development sessions sometimes create patient records in production (tenant-curaway-patients). These synthetic patients pollute:

  • Conversion-funnel analytics — inflated abandonment rates when cases never leave intake
  • Clinical-quality audits — low signal from 1-turn / repeated-condition sessions
  • LLM cost dashboards — test traffic counted against real case spend
  • Future Mem0 / PostHog cohort analysis — synthetic journeys skew cohort shapes

The remedy is to set patients.metadata.is_test = true on each synthetic patient. Every downstream Metabase query, BE analytics route, and cohort filter should include the exclusion clause described below.

Tagged Patients

patient_id tenant_id date tagged tagged by reason
(none yet — apply after merge using the command in the next section)

Add a row here every time you run tag_test_patient.py in production.

How to Tag a New Test Patient

Prerequisites

  • DATABASE_URL_ADMIN must be set (superuser connection, used by Alembic migrations).
  • railway CLI installed and authenticated.

One-liner (production via Railway)

railway run -s curaway --environment production -- \
  python scripts/tag_test_patient.py \
  --patient-id <patient-uuid>

The --tenant-id flag defaults to tenant-curaway-patients. Override if needed:

railway run -s curaway --environment production -- \
  python scripts/tag_test_patient.py \
  --patient-id <patient-uuid> \
  --tenant-id tenant-some-other

Omit --apply (or pass --dry-run) to preview without writing:

railway run -s curaway --environment production -- \
  python scripts/tag_test_patient.py \
  --patient-id db17f171-09ba-4b6c-bbc4-068ea0cd9fdf

This prints the before/after metadata and the planned audit row without touching the database.

Apply

railway run -s curaway --environment production -- \
  python scripts/tag_test_patient.py \
  --patient-id db17f171-09ba-4b6c-bbc4-068ea0cd9fdf \
  --apply

The script is idempotent: if is_test is already true, it logs "already tagged" and exits 0 without writing.

After tagging

  1. Add the patient to the Tagged Patients table above.
  2. Verify the row in audit_logs (action = 'admin.patient.tagged_test').
  3. Update Metabase dashboards that show patient / case counts (add the SQL filter below).

SQL Filter Pattern

Use this clause in every Metabase query, BE analytics route, and cohort filter to exclude test patients:

WHERE COALESCE(metadata->>'is_test', 'false') != 'true'

Example — cases from real patients only:

SELECT c.*
  FROM cases c
  JOIN patients p ON p.id = c.patient_id
 WHERE COALESCE(p.metadata->>'is_test', 'false') != 'true'
   AND c.tenant_id = 'tenant-curaway-patients';

Reverting a Tag

If a patient was incorrectly tagged, remove the is_test key and audit-log the reversion:

-- Run inside a transaction:
BEGIN;

UPDATE patients
   SET metadata = metadata - 'is_test'
 WHERE id = '<patient-uuid>'
   AND tenant_id = 'tenant-curaway-patients';

INSERT INTO audit_logs
    (id, actor_id, actor_type, actor_role, action,
     resource_type, resource_id, tenant_id, changes, created_at)
VALUES (
    gen_random_uuid(),
    'system:manual-revert',
    'system',
    'super_admin',
    'admin.patient.untagged_test',
    'patient',
    '<patient-uuid>',
    'tenant-curaway-patients',
    '{"before": {"metadata": {"is_test": true}}, "after": {"metadata": {}}}'::jsonb,
    now()
);

COMMIT;

Update the Tagged Patients table above (add a note / strike through the row).

Cross-References