Monitoring

Monitoring tracks your AI system's health in real-time: response times, success rates, error rates, and custom metrics. Alerts fire when thresholds are breached, giving you early warning of problems before users notice.

Monitoring architecture

Response latency (p95 and p99), success rate, error rate, and token usage. These four metrics tell you if your agent is fast, reliable, and cost-efficient. Add custom metrics for your specific use case as needed.

patterns/19b_monitoring.py
python
class PerformanceMetrics:
    def __init__(self):
        self.response_times = []
        self.success_count = 0
        self.error_count = 0

    def add_response(self, duration, success=True, error_type=None):
        self.response_times.append(duration)
        if success:
            self.success_count += 1
        else:
            self.error_count += 1

    def check_alerts(self):
        alerts = []
        avg_time = sum(self.response_times) / len(self.response_times)
        if avg_time > 5.0:
            alerts.append("High response time: {:.1f}s".format(avg_time))

        total = self.success_count + self.error_count
        success_rate = self.success_count / max(1, total) * 100
        if success_rate < 90:
            alerts.append(f"Low success rate: {success_rate:.0f}%")
        return alerts

class AgentMonitor:
    def get_health_status(self):
        # HEALTHY / WARNING / CRITICAL based on metrics
        alerts = self.metrics.check_alerts()
        if len(alerts) == 0: return "HEALTHY"
        elif len(alerts) <= 2: return "WARNING"
        else: return "CRITICAL"

PerformanceMetrics tracks response times and alerts on anomalies.

Quiz: Quiz

Loading practice…

Flashcards: Flashcards

Loading practice…

You now have the tools to monitor AI systems in production. Next, we will learn how to prioritize tasks so your agents handle the most important work first.