CULTIVA IA — Observabilidad platform-api

Golden Signals & SLO Status

◈Service Status

HEALTHY

99.94%

Availability (30d)

↑ +0.04% vs last month

142ms

P50 Latency

287ms

P99 Latency

◎SLO Achievement (30d)

SLO	Target	Current	Error Budget	Status
Availability	99.9%	99.94%	40.3 min restantes 94% restante43.2 min total	OK
Request Latency P95	< 200ms	178ms	62% bajo umbral 78% margenthreshold: 200ms	OK
Error Rate	< 0.5%	0.31%	37.8 min restantes 62% restanteconsumo acelerado	WARN
User Journey Success	99.9%	99.97%	97% budget intacto 97% restante43.2 min total	OK

◑Error Budget Remaining

40.3 min left

37.8 min left

0.94x

Burn Rate

14.4x

Alert Threshold

Request Latency & Traffic

◷Request Latency (p50 / p95 / p99)

histogram_quantile • 5m rate

p50

p95

p99

SLO 200ms

-6h-5h-4h-3h-2h-1hnow

Current: 142ms p50

p95: 178ms

p99: 287ms

Prometheus Query

histogram_quantile(0.95, rate(http_request_duration_seconds_bucket{service="platform-api"}[5m]))

⟳Request Rate (req/s)

rate • 1m window

Total RPS

2xx

5xx

4xx

-6h-5h-4h-3h-2h-1hnow

Total: 247 rps

2xx: 245.4 rps

5xx: 0.8 rps

4xx: 0.8 rps

Prometheus Query

sum(rate(http_requests_total{service="platform-api"}[1m])) by (status_code)

Error Rate & Resource Saturation

!Error Rate (%)

0.31% — WARN (SLO: <0.5%)

5xx Rate

4xx Rate

SLO 0.5%

-6h-5h-4h-3h-2h-1hnow

Prometheus Query

sum(rate(http_requests_total{service="platform-api",code=~"5.."}[5m])) / sum(rate(http_requests_total{service="platform-api"}[5m]))

⊙Saturation Metrics

USE Method

CPU

Memory

DB Connections

Worker Queue

-6h-5h-4h-3h-2h-1hnow

CPU: 43%

Mem: 61%

DB Conn: 68%

Queue: 34%

Infrastructure Resources

◈CPU Usage

3 pods · GKE n2-standard-4

◈Memory Usage

4.9 GB / 8 GB limit

⇅Network I/O

↑ 45 MB/s

↓ 128 MB/s

⊡Disk I/O

Read: 12 MB/s

Write: 3.4 MB/s

Endpoint Performance & Distribution

▤Top Slowest Endpoints (p95 · last 1h)

Endpoint	Method	p50	p95	p99	RPS	Error%	Status
/api/v1/ai/generate	POST	312ms	845ms	1.2s	42	0.8%	WARN
/api/v1/skills/execute	POST	198ms	425ms	680ms	89	0.2%	OK
/api/v1/billing/usage	GET	45ms	120ms	245ms	18	0.0%	OK
/api/v1/agencies/report	GET	280ms	520ms	940ms	7	0.1%	SLOW
/api/v1/auth/token	POST	35ms	88ms	145ms	156	0.1%	OK
/api/v1/jobs/submit	POST	89ms	195ms	420ms	34	0.1%	OK

▦Request Latency Heatmap (24h)

bucket distribution · ms

1000 500 200 100 50 10

00h03h06h09h 12h15h18h21h24h

Density: low high

Alert Optimizer — Resultados del Analisis

⚠Alertas Ruidosas Detectadas

5 alertas · 60% falsos positivos

HighErrorRate

Noise score: 4/5 · window: 1m

Fix: Anadir 'for: 5m' + aumentar ventana a 5m

CRITICAL

HighLatency

Noise score: 4/5 · sin clausula 'for'

Fix: for: 5m + p95 en lugar de p99

WARNING

HighCPU

Noise score: 4/5 · threshold fijo 80%

Fix: histeresis 75%/85% + for: 10m

WARNING

HighMemory

Noise score: 4/5 · sin runbook

Fix: agregar runbook + for: 15m

WARNING

Reduccion estimada de ruido

-58%

60% → 25%
falsos positivos

◎Burn Rate Alerts Generadas (SLO-Based)

Multi-window · Availability SLO

Sustituyen las alertas threshold simples con alertas basadas en consumo de error budget (metodologia Google SRE):

Alerta	Ventana Corta	Ventana Larga	Burn Rate	Budget Consumido	Severity
Availability Burn Rate	5m	1h	14.4x	2%	CRITICAL
Availability Burn Rate	30m	6h	6x	5%	WARNING
Availability Burn Rate	2h	1d	3x	10%	INFO
Error Rate Burn Rate	5m	1h	14.4x	2%	CRITICAL
Error Rate Burn Rate	30m	6h	6x	5%	WARNING

Ejemplo — Prometheus alert rule (Availability Burn Rate 2%)

- alert: AvailabilityBurnRate2pct
expr: >
((1 - (sum(rate(http_requests_total{service='platform-api',code!~'5..'}[5m])) /
sum(rate(http_requests_total{service='platform-api'}[5m]))) > 14.4) and
(1 - (sum(rate(http_requests_total{service='platform-api',code!~'5..'}[1h])) /
sum(rate(http_requests_total{service='platform-api'}[1h]))) > 14.4))
for: 0m
labels:
severity: critical
annotations:
summary: "Error budget burn rate 14.4x — 2% budget consumed"

SLA Commitments & Implementation Roadmap

◇SLA Recomendados para Agencias Cliente

Metrica	SLO interno	SLA cliente	Ventana	Penalizacion
Availability	99.9%	99.8%	monthly	10% credit <99.9%
Feature Availability	99.9%	99.8%	monthly	10% credit <99.9%
API Response Time	<200ms p95	<500ms p95	monthly	Reportado en dashboard
Error Rate	<0.5%	<1%	monthly	5% credit <99.5%

Nota SRE

Buffer interno: SLO 99.9% → SLA 99.8% garantiza margen para resolver incidencias antes de impactar creditos a clientes.

▶Implementation Roadmap

Generado por disenador-observabilidad-produccion

Instrumentar con OpenTelemetry

Agregar auto-instrumentation a FastAPI + Celery. Exportar a Prometheus + Tempo.

Importar dashboard.json en Grafana Cloud

13 paneles listos: golden signals, SLO gauges, top endpoints, heatmap de latencia.

Sustituir alertas simples por burn rate alerts

Eliminar 5 alertas ruidosas. Desplegar 10 reglas multi-ventana. Objetivo: -58% falsos positivos.

Configurar runbooks en PagerDuty

Enlazar cada alerta a su runbook. Escalacion: dev → SRE → CTO segun severidad.

Revision post-guardia (1 rotacion on-call)

Re-ejecutar alert_optimizer --analyze-only contra config live. Iterar si actionable ratio no mejoro.