CultivaML — Configuración Istio Traffic Management

Microservicios

Recursos Istio

Canary Inicial

Errores max CB

10s

Timeout global

Arquitectura de tráfico

🌐

GatewayHTTPS :443

→

↔

VirtualServiceRouting rules

→

⚙

DestinationRulePolicies + CB

→

📦

model-serving v1stable :8501

⇢ 5%

🆕

model-serving v2canary GPU :8501

Microservicios configurados

api-gateway

Punto de entrada HTTPS con TLS terminado

:443 / :8080

Ingress

model-serving

Inferencia ML — v1 stable + v2 GPU (canary)

:8501

Canary 5%Mirror

feature-store

Features en tiempo real — alta carga bajo picos

:7070

Circuit Breaker

auth-service

Autenticación JWT con sticky sessions

:9000

LB Hash

webhook-dispatcher

Envío de eventos a clientes externos

:8090

Retry

Distribución de tráfico — model-serving (fase canary actual)

v1-stable

95%

v2-gpu (canary)

v2-gpu (mirror)

100% espejo

100%

Plan de rollout progresivo

Fase 1

0% canary

Solo mirroring silencioso al v2-gpu

Completado

Fase 2

5% canary

Tráfico real mínimo + mirroring 100%

En curso

Fase 3

25% canary

Ampliar si error rate v2 < 0.1%

Pendiente

Fase 4

50% canary

Validar latencia p99 ≤ 200ms

Pendiente

Fase 5

100% stable

Promover v2 → stable, retirar v1

Pendiente

Configuraciones YAML generadas

Ingress Gateway — api.cultivaml.io (HTTPS)

Gateway + VirtualService

# Gateway: terminar TLS en el borde del mesh
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: cultivaml-gateway
  namespace: cultivaml-prod
spec:
  selector:
    istio: ingressgateway
  servers:
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: cultivaml-tls-secret
      hosts:
        - "api.cultivaml.io"
---
# VirtualService: enrutar tráfico entrante a api-gateway service
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cultivaml-ingress-vs
  namespace: cultivaml-prod
spec:
  hosts:
    - "api.cultivaml.io"
  gateways:
    - cultivaml-gateway
  http:
    - match:
        - uri:
            prefix: /api/v1/predict
      route:
        - destination:
            host: model-serving
            port:
              number: 8501
    - match:
        - uri:
            prefix: /api/v1
      route:
        - destination:
            host: api-gateway
            port:
              number: 8080

Canary Deployment + Traffic Mirroring — model-serving

VirtualService + DestinationRule

# VirtualService: 95% stable / 5% canary + mirror 100% a v2-gpu
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: model-serving-canary
  namespace: cultivaml-prod
spec:
  hosts:
    - model-serving
  http:
    - route:
        - destination:
            host: model-serving
            subset: stable
          weight: 95
        - destination:
            host: model-serving
            subset: canary
          weight: 5
      mirror:                    # duplicar tráfico silencioso
        host: model-serving
        subset: canary
      mirrorPercentage:
        value: 100.0
      timeout: 10s
      retries:
        attempts: 2
        perTryTimeout: 5s
        retryOn: connect-failure,refused-stream,503
---
# DestinationRule: subsets stable vs canary-gpu + connection pool
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: model-serving-dr
  namespace: cultivaml-prod
spec:
  host: model-serving
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 200
      http:
        h2UpgradePolicy: UPGRADE
        http2MaxRequests: 2000
  subsets:
    - name: stable
      labels:
        version: v1-stable
    - name: canary
      labels:
        version: v2-gpu

Circuit Breaker — feature-store (alta carga)

DestinationRule

# Circuit breaker: expulsar pods tras 5 errores 5xx consecutivos
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: feature-store-cb
  namespace: cultivaml-prod
spec:
  host: feature-store
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 150
      http:
        http1MaxPendingRequests: 200
        http2MaxRequests: 1500
        maxRequestsPerConnection: 20
        maxRetries: 3
    outlierDetection:           # circuit breaker
      consecutive5xxErrors: 5   # umbral de errores
      interval: 30s              # ventana de análisis
      baseEjectionTime: 30s     # tiempo fuera del pool
      maxEjectionPercent: 50    # máx pods expulsados
      minHealthPercent: 30      # garantía de disponibilidad

Retry & Timeout — webhook-dispatcher

VirtualService

# Reintentos con backoff para llamadas a clientes externos (503 frecuentes)
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: webhook-retry
  namespace: cultivaml-prod
spec:
  hosts:
    - webhook-dispatcher
  http:
    - route:
        - destination:
            host: webhook-dispatcher
      timeout: 10s
      retries:
        attempts: 3
        perTryTimeout: 3s
        retryOn: connect-failure,refused-stream,unavailable,cancelled,retriable-4xx,503
        retryRemoteLocalities: true

Sticky Sessions (Consistent Hash) — auth-service

DestinationRule

# Hash consistente por header x-tenant-id para stickiness de sesión JWT
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: auth-sticky
  namespace: cultivaml-prod
spec:
  host: auth-service
  trafficPolicy:
    loadBalancer:
      consistentHash:
        httpHeaderName: x-tenant-id  # mismo tenant → mismo pod

Comandos de diagnóstico

# Validar toda la config del namespace
$ istioctl analyze -n cultivaml-prod

# Ver rutas efectivas del model-serving
$ istioctl proxy-config routes deploy/model-serving -n cultivaml-prod -o json

# Comprobar endpoints y health del feature-store (CB)
$ istioctl proxy-config endpoints deploy/feature-store -n cultivaml-prod | grep -E "HEALTHY|UNHEALTHY"

# Ver distribución real de tráfico en el mesh (Kiali)
$ kubectl port-forward svc/kiali -n istio-system 20001:20001
# → http://localhost:20001 → Graph → cultivaml-prod

# Debug logs del webhook-dispatcher (retry visibility)
$ istioctl proxy-config log deploy/webhook-dispatcher -n cultivaml-prod --level debug

# Aplicar todos los manifiestos
$ kubectl apply -f 01-cultivaml-gateway.yaml \
    -f 02-model-serving-canary.yaml \
    -f 03-feature-store-circuit-breaker.yaml \
    -f 04-webhook-retry.yaml \
    -f 05-auth-sticky-sessions.yaml

Mejores practicas aplicadas

✅

Do's — aplicados en CultivaML

Canary progresivo: empieza en 5% y escala con métricas
Timeouts en todos los VS: 10s global, 3s por intento
Subsets con labels explícitas: v1-stable / v2-gpu
Circuit breaker habilitado en feature-store (alta carga)
Mirror a v2 antes de enviar tráfico real (fase 1)
Sticky sessions por x-tenant-id en auth-service

⛔

Don'ts — evitados

No hacer rollout al 100% sin pasar por canary gradual
No ignorar outlierDetection en servicios con picos
No configurar retryOn sin límite de intentos (cascada)
No activar mirror a producción hasta tener v2 estable
No saltarse istioctl analyze antes de aplicar configs
No sobrecargar connectionPool sin benchmarking previo

CultivaML — Istio Traffic Management