pos-system/docs/en/skills/deployment-kubernetes.md

name, description

name	description
deployment-kubernetes	Kubernetes deployment patterns for GoodGo microservices. Use when deploying to staging/production, creating K8s manifests, configuring HPA, setting up ingress, or troubleshooting K8s deployments.

Kubernetes Deployment Patterns

When to Use This Skill

Use this skill when:

• Deploying services to staging/production environments
• Creating or updating Kubernetes manifests
• Configuring autoscaling (HPA/VPA)
• Setting up ingress and load balancing
• Managing secrets and configmaps
• Troubleshooting deployment issues
• Implementing health checks and probes
• Setting up monitoring and logging

Core Concepts

Deployment Strategy

• Rolling updates for zero-downtime deployments
• Resource limits and requests for stability
• Health checks (liveness/readiness probes)
• Horizontal Pod Autoscaler (HPA) for auto-scaling
• ConfigMaps for configuration
• Secrets for sensitive data

Service Deployment Manifest

# kubernetes/auth-service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-service
  namespace: goodgo
  labels:
    app: auth-service
    version: v1
spec:
  replicas: 3
  selector:
    matchLabels:
      app: auth-service
  template:
    metadata:
      labels:
        app: auth-service
        version: v1
    spec:
      containers:
      - name: auth-service
        image: goodgo/auth-service:latest
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 3000
          name: http
        env:
        - name: NODE_ENV
          value: "production"
        - name: PORT
          value: "3000"
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: database-secrets
              key: url
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: auth-secrets
              key: jwt-secret
        - name: REDIS_URL
          valueFrom:
            configMapKeyRef:
              name: redis-config
              key: url
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: auth-service
  namespace: goodgo
spec:
  type: ClusterIP
  selector:
    app: auth-service
  ports:
  - port: 80
    targetPort: 3000
    protocol: TCP

Horizontal Pod Autoscaler

# kubernetes/hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: auth-service-hpa
  namespace: goodgo
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: auth-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15

ConfigMap & Secrets

# kubernetes/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: goodgo
data:
  NODE_ENV: "production"
  LOG_LEVEL: "info"
  REDIS_URL: "redis://redis-service:6379"
  METRICS_ENABLED: "true"

---
# kubernetes/secrets.yaml (example - use sealed-secrets in production)
apiVersion: v1
kind: Secret
metadata:
  name: database-secrets
  namespace: goodgo
type: Opaque
stringData:
  url: "postgresql://user:pass@postgres:5432/db"
  
---
apiVersion: v1
kind: Secret
metadata:
  name: auth-secrets
  namespace: goodgo
type: Opaque
stringData:
  jwt-secret: "your-secret-key"
  refresh-secret: "your-refresh-secret"

Ingress Configuration

# kubernetes/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: goodgo
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rate-limit: "100"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - api.goodgo.com
    secretName: api-tls-secret
  rules:
  - host: api.goodgo.com
    http:
      paths:
      - path: /auth
        pathType: Prefix
        backend:
          service:
            name: auth-service
            port:
              number: 80
      - path: /users
        pathType: Prefix
        backend:
          service:
            name: user-service
            port:
              number: 80

Database Deployment (Development Only)

# kubernetes/postgres.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: goodgo
spec:
  serviceName: postgres
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:14-alpine
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_DB
          value: goodgo
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secret
              key: password
        volumeMounts:
        - name: postgres-storage
          mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
  - metadata:
      name: postgres-storage
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi

Deployment Scripts

#!/bin/bash
# scripts/deploy-k8s.sh

# Set namespace
NAMESPACE="goodgo"
ENVIRONMENT="${1:-staging}"

# Create namespace if not exists
kubectl create namespace $NAMESPACE --dry-run=client -o yaml | kubectl apply -f -

# Apply configurations
echo "Applying ConfigMaps..."
kubectl apply -f kubernetes/configmap-$ENVIRONMENT.yaml

echo "Applying Secrets..."
kubectl apply -f kubernetes/secrets-$ENVIRONMENT.yaml

echo "Deploying services..."
kubectl apply -f kubernetes/auth-service.yaml
kubectl apply -f kubernetes/user-service.yaml

echo "Configuring autoscaling..."
kubectl apply -f kubernetes/hpa.yaml

echo "Setting up ingress..."
kubectl apply -f kubernetes/ingress.yaml

# Wait for rollout
kubectl rollout status deployment/auth-service -n $NAMESPACE
kubectl rollout status deployment/user-service -n $NAMESPACE

echo "Deployment complete!"

Health Check Implementation

// src/modules/health/health.controller.ts
export class HealthController {
  constructor(
    private prisma: PrismaClient,
    private redis: Redis
  ) {}

  // Liveness probe - is the service alive?
  async liveness(req: Request, res: Response) {
    res.status(200).json({ status: 'ok' });
  }

  // Readiness probe - is the service ready to accept traffic?
  async readiness(req: Request, res: Response) {
    try {
      // Check database connection
      await this.prisma.$queryRaw`SELECT 1`;
      
      // Check Redis connection
      await this.redis.ping();
      
      res.status(200).json({
        status: 'ready',
        checks: {
          database: 'ok',
          redis: 'ok'
        }
      });
    } catch (error) {
      res.status(503).json({
        status: 'not ready',
        error: error.message
      });
    }
  }
}

Monitoring with Prometheus

# kubernetes/servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: auth-service-monitor
  namespace: goodgo
spec:
  selector:
    matchLabels:
      app: auth-service
  endpoints:
  - port: http
    path: /metrics
    interval: 30s

Common Commands

# Deploy to staging
kubectl apply -f kubernetes/ -n goodgo-staging

# Check deployment status
kubectl get deployments -n goodgo
kubectl get pods -n goodgo
kubectl get svc -n goodgo

# View logs
kubectl logs -f deployment/auth-service -n goodgo
kubectl logs -f pod-name -n goodgo --tail=100

# Scale manually
kubectl scale deployment auth-service --replicas=5 -n goodgo

# Update image
kubectl set image deployment/auth-service auth-service=goodgo/auth-service:v1.2.3 -n goodgo

# Rollback
kubectl rollout undo deployment/auth-service -n goodgo

# Port forward for debugging
kubectl port-forward service/auth-service 3000:80 -n goodgo

# Execute command in pod
kubectl exec -it pod-name -n goodgo -- /bin/sh

# View HPA status
kubectl get hpa -n goodgo
kubectl describe hpa auth-service-hpa -n goodgo

# View resource usage
kubectl top nodes
kubectl top pods -n goodgo

Troubleshooting

Pod Not Starting

# Check pod status
kubectl describe pod pod-name -n goodgo

# Check events
kubectl get events -n goodgo --sort-by='.lastTimestamp'

# Check logs
kubectl logs pod-name -n goodgo --previous

ImagePullBackOff

# Check image name and tag
kubectl describe pod pod-name -n goodgo | grep -i image

# Check image pull secrets
kubectl get secrets -n goodgo

CrashLoopBackOff

# Check logs of crashed container
kubectl logs pod-name -n goodgo --previous

# Check resource limits
kubectl describe pod pod-name -n goodgo | grep -A 5 Limits

Best Practices

1. Resource Management

   • Always set resource requests and limits
   • Monitor actual usage and adjust accordingly
   • Use HPA for automatic scaling

2. Configuration

   • Use ConfigMaps for non-sensitive config
   • Use Secrets for sensitive data
   • Never hardcode configuration in images

3. Health Checks

   • Implement both liveness and readiness probes
   • Set appropriate timeouts and thresholds
   • Include dependency checks in readiness probe

4. Deployment

   • Use rolling updates for zero-downtime
   • Set maxSurge and maxUnavailable appropriately
   • Test deployments in staging first

5. Security

   • Run containers as non-root user
   • Use network policies to restrict traffic
   • Regularly update base images
   • Use sealed-secrets or external secret manager

6. Monitoring

   • Expose metrics endpoint
   • Set up alerts for critical issues
   • Monitor resource usage and performance