How Numan Built an Eval-First AI Health Coach with LangGraph and LangSmith

Numan, a UK digital health company, developed Nu, an AI health coach for managing long-term conditions, focusing on high-quality conversational coaching while avoiding regulated medical advice. They found standard evaluation methods inadequate for conversational coaching and improved their evaluation criteria to better assess multi-turn interactions and maintain safety boundaries.