Feedback: Multiple Tweaks from Malina (7/10/25)

Added by Justin Sheehan Justin S.
Column
Done
Assigned to
Troy Pastoral Troy P.
Subtasks
Review tweaks Troy Pastoral Troy P. Jul 24, 2025
🔄 General Functionality
📚 Knowledge Access via Project SMART
📄 Page-by-Page QA from Attached Notes
Justin Sheehan
Justin Sheehan Chief Journey Officer
Troy you may have made these tweaks already, but please take a look at this email from Malina - Feedback from Malina - 🟣 Doden AI | AE

Make any remaining tweaks (if they make sense and are merited).
Troy Pastoral
Troy Pastoral AI Whisperer
[SPRINT 1]

✅ Fixed Issues Summary


1. Sub-prompt Override Logic - RESOLVED

Problem: Sub-prompts completely overrode mode personalities, making modes appear non-functional Fix:

  • Modified the logic in openai.ts to combine mode prompts with sub-prompt styles instead of replacing them
  • Sub-prompts now enhance the base mode personality rather than override it
  • Added clear logging to track when sub-prompt modifiers are applied

2. Mode Validation & Error Handling - RESOLVED

Problem: No validation for invalid modes, poor fallback handling Fix:

  • Added mode validation in the edge function with explicit fallback to 'coach' mode
  • Added warning logs for invalid modes
  • Improved error messages and logging throughout the pipeline

3. Multi-Style Consistency - RESOLVED

Problem: Mode changes didn't clear sub-prompts, causing style conflicts Fix:

  • Added automatic sub-prompt clearing when changing modes in ChatInterface.tsx
  • This prevents previous style modifiers from interfering with new mode personalities

4. Model Compatibility - RESOLVED

Problem: Edge function was using invalid OpenAI model Fix:

  • Changed from gpt-4.1-2025-04-14 to gpt-4o-mini for proper compatibility

🎯 Expected Improvements

  1. ✅ Coach Mode (Alpha): Now guaranteed to work consistently with proper mode validation
  2. ✅ Default Style Response: Better error handling and fallback logic prevents glitches
  3. ✅ Multi-Style Consistency: Modes now maintain distinct personalities even with sub-prompt modifiers
  4. ✅ Style Switching: No more cached behavior when switching between modes
Troy Pastoral
Troy Pastoral AI Whisperer
[SPRINT 2]

🔍 Project SMART Article Recognition:

  • Enhanced Threshold Logic: Adaptive scoring (0.25-0.4) based on match quality and distribution
  • Improved Fallback: When no high-quality matches found, uses top 2-3 with relaxed threshold instead of failing
  • Quality Monitoring: Added comprehensive logging for article coverage, relevance scores, and namespace status
  • Alert System: Warns when <5 sources found (target: ~15 articles) or relevance <0.3

🧠 Conversational Memory Optimization:

  • Memory Management: For conversations >8 messages, keeps last 6 + summarizes older context
  • Context Compression: Prevents token overflow while preserving conversation continuity
  • Smart Summarization: Extracts key topics from older messages for context retention

📊 Monitoring & Diagnostics:

  • Real-time Metrics: Tracks total matches, high-quality matches (>0.5), and top scores
  • Performance Alerts: Identifies low coverage and relevance issues
  • Debug Enhancement: Better logging for troubleshooting article recognition problems
Troy Pastoral
Troy Pastoral AI Whisperer
[SPRINT 3]

✅ Vague feedback handling - Refined responses for short prompts like "great" to avoid default disclaimers ✅ Source prioritization - Enhanced recognition for high-use articles like 5Cs framework
✅ Formatting cleanup - Removed unnecessary [a] bullet formatting and "knowledge base" phrases ✅ Thread continuity - Improved follow-up question handling and context retention ✅ Error handling - Better confidence scoring for specific sub-questions