Admin Hardening

Added by Troy Pastoral Troy P. June 8, 2026 12:37pm
Column
Done June 12, 2026 2:14pm
Assigned to
Troy Pastoral Troy P.
Notes

QA READY — Daryle AI DevOps Harness


What changed: The bot had no automated way to detect when it
stops responding — failures only appeared via user complaints.
Now a daily cron actually tests the bot, model errors get
logged, and models can be swapped from the admin panel without
a deploy.


What to test:


  1. 4-provider status cards
    Before: Only OpenAI and Pinecone had status cards
    After: Anthropic and Gemini cards also visible
    Steps: Admin → Service Health. Count cards at top — 4
    expected. Click "Run Check Now" — all 4 update.
  2. LLM Failure Events panel
    Before: No visibility into AI model errors
    After: Panel shows errors grouped by type; empty state
    when system is healthy
    Steps: Admin → Service Health → scroll below donut charts.
    "LLM Failure Events — Last 24h" section visible.
  3. Model Configuration panel
    Before: Changing AI model = code deploy
    After: Change model from admin, live within 60 seconds
    Steps: Admin → Service Health → scroll to bottom. Six rows
    (grounded/fast/claude/pro/chatgpt/gemini) with
    editable fields and Save buttons.
  4. Chat modal model info
    Before: Chat modal showed messages only
    After: Footer shows model name, response time, fallback
    Steps: Admin → Chats → open any chat. Footer line shows
    model and latency (only for chats with log data).
  5. Daily canary (deploy-gated)
    Before: No automated health check
    After: GitHub Actions fires 6am + 6pm UTC, tests each
    route, emails alert on 2nd consecutive failure
    Steps: GitHub → Actions → Health Canary → Run workflow.
    Edge case: Requires SUPABASE_FUNCTIONS_URL +
    SUPABASE_SERVICE_ROLE_KEY in repo secrets first.