Langfuse Prompt Management — Implementation Recommendation¶

Status: Recommended for post-demo implementation Effort: 2-3 hours (dedicated session) Priority: Medium — do after demo stabilizes, before real patient tuning begins

Problem Statement¶

All 8 LLM prompts are hardcoded as Python string constants across 3 files. Changing any prompt requires a code change, commit, PR, deploy, and Railway restart. This is fine for development but becomes a bottleneck when:

• Tuning brand voice based on patient feedback
• Adjusting guardrail boundaries (too strict or too lenient)
• A/B testing different intake approaches
• CPO/CTO wants to iterate on prompts without engineering deploys

Proposed Solution¶

Migrate all LLM prompts to Langfuse Prompt Management (already deployed on the HIPAA instance at hipaa.cloud.langfuse.com). Prompts become editable in the Langfuse dashboard with versioning, rollback, and A/B testing. Current hardcoded prompts become the fallback when Langfuse is unavailable.

Approach¶

Phase 1: Create Prompts in Langfuse Dashboard¶

Create 8 prompts in the Langfuse UI with template variables:

Phase 2: Add Prompt Fetching Helper¶

# app/agents/prompt_loader.py

from langfuse import get_client

_FALLBACKS = {}  # Populated from current hardcoded prompts

def get_prompt(name: str, variables: dict = None, fallback: str = "") -> str:
    """Fetch a prompt from Langfuse with automatic fallback.

    - Cache: Langfuse SDK caches prompts (configurable TTL, default 60s)
    - Fallback: if Langfuse unavailable, use hardcoded default
    - Logging: logs when fallback is used (indicates Langfuse issue)
    """
    try:
        client = get_client()
        prompt = client.get_prompt(name)
        return prompt.compile(**(variables or {}))
    except Exception as e:
        logger.warning("Langfuse prompt '%s' unavailable: %s — using fallback", name, e)
        template = _FALLBACKS.get(name, fallback)
        if variables:
            return template.format(**variables)
        return template

Phase 3: Refactor LLM Calls¶

Before:

system = CURAWAY_SYSTEM_PROMPT.format(
    phase_context=phase_context,
    patient_context=patient_context,
)

After:

system = get_prompt("curaway_system_v1", {
    "phase_context": phase_context,
    "patient_context": patient_context,
})

Phase 4: Register Fallbacks¶

Move current hardcoded prompts into a _FALLBACKS dict in prompt_loader.py. These are the safety net — if Langfuse is down, the agent still works with the last-known-good prompts from the codebase.

Advantages¶

Risks¶

Tradeoffs¶

Cost Implications¶

Total incremental cost: $0/month

Files Affected¶

Decision Criteria¶

Do it when: - You're actively tuning prompts based on patient/demo feedback - You want to A/B test different intake approaches - Non-engineers need to edit prompts - You need audit trail for prompt changes (compliance)

Don't do it yet if: - Prompts are stable and rarely change - Only engineers touch prompts - Demo is imminent and stability is priority

Recommended Timeline¶

1. Now: Keep current inline prompts. Session 24 guardrails.yaml covers configurable rules.
2. Post-demo (1-2 weeks): Migrate prompts to Langfuse. Current prompts become fallbacks.
3. With real patients: Use Langfuse A/B testing to optimize intake flow, brand voice, and guardrail sensitivity.