Run, Monitor, and Maintain

NCP-AAI topic area — exam weight: 5%

Ongoing operation, monitoring, and maintenance of agentic systems post-deployment.

Ingested Material

• AI Agent Evaluation — Summary
• Log, Trace, and Monitor Portkey Integrations
• Time-Weighted Retrieval Integration
• Observability Concepts (LangSmith)
• Structured Output (LangChain Agents)
• How to Handle Model Rate Limits
• A Guide to Monitoring Machine Learning Models in Production (cross-section)
• Monitoring ML Models: Data Quality and Integrity (cross-section)