ADR-0026 — Matching framework architecture: parameter registry, store contract, projection sync¶

• Date: 2026-05-07
• Status: Accepted
• Author: Curaway engineering (SD + Claude Opus)
• Context: Audit of the 147-parameter matching algorithm spec (Curaway_Matching_Algorithm_Parameters.docx) against the live implementation revealed three architectural drifts that this ADR corrects.

Context¶

The matching algorithm spec describes 147 parameters across 12 domains, with the headline novelty being a 7-hop graph traversal Patient → Condition → Procedure → Provider → Outcome → Cost → Location. An audit on 2026-05-07 found:

1. Engine coverage is shallow. WeightedScoringV1 consumes 14 of 36 Provider fields and touches 7 of 12 spec domains. Missing fields are substituted with neutral defaults (0.5 / 2.5 / 1.0) without renormalization, so a sparse provider scores identically to a populated one in practice.
2. Stores are out of sync. 172 doctors are seeded in Postgres via seed_doctors_full.py but never written to Neo4j. Helper functions graph_service.create_doctor_node and link_doctor_procedure exist but are never called. The 7-hop traversal is therefore impossible today.
3. No mechanism to "graduate" parameters. The spec is aspirational — 147 parameters with no signal for which are seeded, which are reference-only, and which are not yet implemented. Any new parameter requires manual code in the scorer.

The cause is implicit. Postgres and Neo4j have been treated as parallel sources of truth: each is written from a different code path (repositories vs. seed scripts), with no contract for which writes propagate where. Provider/doctor onboarding on admin (planned, ~2 weeks out) makes this drift production-visible: every admin form submit will need to land in both stores, and the current code has no pattern for that.

This ADR locks in three decisions that have been pushed off implicitly until now.

Decision¶

1. Postgres is the canonical source of truth. Neo4j is a projection.¶

Postgres earns its keep on three things Neo4j is materially weaker at: multi-tenant RLS enforcement, ACID + Alembic migrations, and audit-log immutability. Neo4j earns its keep on graph traversals (matching engine 7-hop reasoning) and the data flywheel (COMPLETED_JOURNEY_AT similar-patient analysis).

Anything in Neo4j must be reconstructible from Postgres + reference YAML. If a field exists only in Neo4j, that's a bug.

2. Parameter registry: config/matching/parameters/<domain>.yaml¶

The 147 parameters live in version-controlled YAML, split per-domain (clinical_context.yaml, provider_capability.yaml, visa_regulatory.yaml, etc.). Every parameter has the same schema:

- id: jci_accreditation
  domain: provider_capability
  domain_weight_share: 0.06          # share within domain (sums to 1.0 per domain)
  source: postgres                    # postgres | neo4j | yaml | computed | api
  source_path: provider.accreditations  # ORM path / Cypher attribute / YAML file
  status: active                      # active | seeded | unseeded | not_implemented
  enable_when:
    min_provider_coverage: 0.5        # don't activate until ≥50% of providers have data
  patent_claim: true                  # included in patent novelty disclosure

The matching engine reads the registry, applies only status: active parameters that pass enable_when, computes per-domain scores using only present data, and renormalizes weights across domains where data exists. This delivers four properties at once:

1. Patent claim is satisfied — the registry references all 147 parameters even when most are dormant.
2. Completeness scoring is free — match_confidence is the share of active parameters with non-null data per provider.
3. Per-tenant + per-patient activation is Flagsmith-controllable — flag a single id to override status.
4. Graduation is observable — moving a parameter from unseeded → active is a single registry edit, tracked in git.

A parameter with status: not_implemented exists in the registry for spec/patent completeness but is never read by the engine. It graduates when (a) a data source lands, (b) status is changed to seeded or active, and (c) coverage thresholds are met.

3. Projection sync: QStash event-driven, async, with bulk-rebuild fallback¶

Every Postgres write that affects a graph entity (provider, doctor, procedure offering, completed journey) emits a QStash event. A projection worker consumes the event and upserts the matching Neo4j subgraph idempotently. Two layers of consistency:

Admin provider onboarding follows the submit-now / matchable-in-30s pattern: the form returns 200 as soon as Postgres commits; the projection lands within seconds; the admin sees a "matchable" badge update via SSE or refresh. The form does not block on the projection — that would couple admin form latency to graph writes.

Reference data (visa corridors, country tier, AQI, climate) is loaded into Neo4j by the same bulk-rebuild path, sourced from config/reference/*.yaml. Refresh is operator-triggered via the admin UI (/admin/matching parameter dashboard, see Decision 4 below).

4. /admin/matching becomes a parameter-registry-aware dashboard¶

The current /matching page surfaces 5 category-level weight sliders and a static reference list of 18 parameters. It is replaced by a dashboard that reads the registry directly:

• Per-domain coverage panel: % of active parameters with data per provider, aggregated.
• Per-parameter status panel: 147 rows showing id, domain, status, source, coverage, last refresh.
• Per-domain weight tuning: edit domain_weight_share for any domain (writes a Flagsmith override, not the YAML).
• "Rebuild graph" button: triggers POST /admin/graph/rebuild.
• "Refresh reference data" button: re-reads config/reference/*.yaml and updates Neo4j.

The 5-slider model is preserved as a "domain-level view" toggle.

Consequences¶

Positive¶

• New parameters can be added by registry edit + seed without engine changes.
• Sparse providers score honestly — completeness is observable, not hidden.
• Admin onboarding has a clear sync contract: write to Postgres, projection follows.
• The patent narrative ("147 parameters across 12 domains") is no longer aspirational; it's executable, with a graduation path.

Negative¶

• Two layers of indirection. Engineers reading the matching code now read the registry first, then the engine, then the projection worker. Onboarding is steeper.
• Eventual consistency. A provider edited via admin appears in Neo4j ~30s later. Tests that race admin write → matching read need explicit waits.
• Registry drift risk. A parameter with source_path: provider.foo whose underlying ORM field is renamed silently breaks scoring until the next coverage check. Mitigation: a CI guard that walks the registry and validates every source_path resolves.

Migration impact¶

Phase 0 (doctor-graph standalone PR) is a one-time backfill: writes existing 172 Postgres doctors into Neo4j, then enables the projection worker for new writes. Post-Phase-0, any Postgres state can be reconstructed in Neo4j by hitting /admin/graph/rebuild.

Alternatives considered¶

• Postgres-only. Drop Neo4j; emulate graph traversals with recursive CTEs. Rejected: kills the patent claim's "graph-based reasoning" novelty and the data-flywheel architecture, neither of which Postgres can replicate at parity.
• Neo4j-only. Make Neo4j source of truth; demote Postgres to audit storage. Rejected: weak multi-tenant story, no Alembic equivalent, no existing tooling for GDPR cascade delete.
• Synchronous projection inside admin form. Form blocks on Postgres write + Neo4j upsert in one transaction. Rejected: couples admin UX latency to a second store; admin doesn't need instant matchability.
• Parameter registry in Python instead of YAML. Considered briefly. Rejected: YAML is reviewable by non-engineers (Bhaskar, Dr. Naidu) and editable without redeploys when paired with on-demand reference refresh.

References¶

• Curaway_Matching_Algorithm_Parameters.docx (2026-03, root of repo) — original 147-parameter spec
• ADR-0016 — Repository / DAO pattern (PG write contract)
• ADR-0018 — Multi-tenancy + 7-actor model (constrains where tenant_id flows)
• feedback_alembic_admin_connection.md — DDL admin role vs runtime app role distinction
• Audit transcript 2026-05-07 — three-subagent audit of PG schema, Neo4j seed, engine partial-data behavior