diff --git a/.cursor/plans/event-driven_architecture_skill_f267a3f0.plan.md b/.cursor/plans/event-driven_architecture_skill_f267a3f0.plan.md new file mode 100644 index 00000000..066174be --- /dev/null +++ b/.cursor/plans/event-driven_architecture_skill_f267a3f0.plan.md @@ -0,0 +1,405 @@ +--- +name: Event-Driven Architecture Skill +overview: Tạo skill documentation chi tiết cho Event-Driven Architecture với Apache Kafka, bao gồm patterns cho event publishing/consuming, schema versioning, tích hợp Traefik để expose events qua HTTP (SSE/WebSocket), error handling, observability, và testing patterns phù hợp với GoodGo platform standards. +todos: + - id: create-skill-file + content: Tạo file .cursor/skills/event-driven-architecture/SKILL.md với structure cơ bản (header, When to Use, Core Concepts) + status: completed + - id: kafka-setup-section + content: Viết section Kafka Setup & Configuration (infrastructure, client config, connection management) + status: completed + - id: publishing-patterns + content: Viết Event Publishing Patterns section (Publisher class, outbox pattern, best practices) + status: completed + - id: consuming-patterns + content: Viết Event Consuming Patterns section (Consumer class, DLQ, processing strategies) + status: completed + - id: schema-versioning + content: Viết Schema Versioning section (Schema Registry, Avro, evolution strategies) + status: completed + - id: traefik-integration + content: Viết Traefik Integration section (SSE endpoints, WebSocket, routing config) + status: completed + - id: error-resilience + content: Viết Error Handling & Resilience section (error patterns, retries, DLQ, replay) + status: completed + - id: observability + content: Viết Observability section (logging events, metrics, tracing) + status: completed + - id: testing-patterns + content: Viết Testing Patterns section (unit, integration, E2E với test containers) + status: completed + - id: best-practices-examples + content: Viết Best Practices section và Complete Examples (end-to-end flows) + status: completed + - id: review-integration + content: Review và ensure consistency với existing skills, add references, final polish + status: completed +--- + +# Event-Driven Architecture Skill Implementation Plan + +## Overview + +Tạo comprehensive skill documentation cho Event-Driven Architecture patterns sử dụng Apache Kafka làm message broker. Skill này sẽ cover event publishing, consuming, schema versioning, integration với Traefik (SSE/WebSocket), error handling, observability, và testing patterns theo chuẩn GoodGo. + +## Architecture Context + +**Tech Stack:** + +- **Message Broker**: Apache Kafka (high-throughput, streaming) +- **API Gateway**: Traefik (existing) - expose events qua HTTP/SSE/WebSocket +- **Schema Registry**: Confluent Schema Registry (Avro schemas) +- **Services**: Node.js/TypeScript microservices với Express +- **Libraries**: kafkajs (Node.js Kafka client), @goodgo/logger, @goodgo/tracing + +**Traefik Integration:** + +- Traefik vẫn giữ vai trò API Gateway cho synchronous HTTP/REST +- Thêm support expose event streams qua HTTP endpoints (Server-Sent Events / WebSocket) +- Services publish events vào Kafka, Traefik routes SSE/WebSocket requests tới event streaming endpoints + +## Files to Create/Update + +### Primary File + +- `.cursor/skills/event-driven-architecture/SKILL.md` - Main skill documentation (new) + +### Potential Updates (if needed) + +- `.cursor/skills/project-rules/SKILL.md` - Update infrastructure section to include Kafka +- `docs/en/architecture/service-communication.md` - Add event-driven communication section +- `docs/vi/architecture/service-communication.md` - Add event-driven communication section + +## Detailed Implementation Plan + +### Phase 1: Skill File Structure + +Tạo file `.cursor/skills/event-driven-architecture/SKILL.md` với structure: + +1. **Header Metadata** + + - name: event-driven-architecture + - description: Event-driven architecture patterns với Kafka + +2. **When to Use This Skill** section + + - Khi nào nên dùng event-driven patterns + - Use cases: async processing, decoupling services, event sourcing, CQRS + +3. **Core Concepts** + + - Event-driven vs request-response + - Kafka fundamentals (topics, partitions, consumer groups) + - Event schema versioning + - Traefik integration for event streaming + +### Phase 2: Kafka Setup & Configuration + +**Topics Covered:** + +- Kafka infrastructure setup (docker-compose, Kubernetes) +- KafkaJS client configuration +- Connection management và pooling +- Environment variables (KAFKA_BROKERS, KAFKA_CLIENT_ID, etc.) + +**Code Examples:** + +```typescript +// Kafka client setup +// Producer configuration +// Consumer configuration +// Connection error handling +``` + +### Phase 3: Event Publishing Patterns + +**Topics:** + +- Event structure và naming conventions +- Publisher service pattern +- Transactional publishing (outbox pattern) +- Idempotency keys +- Event correlation IDs +- Publishing best practices + +**Patterns:** + +- Simple event publisher +- Transactional publisher với outbox pattern +- Batch publishing +- Error handling và retries + +**Code Examples:** + +```typescript +// EventPublisher class +// Publishing events from services +// Outbox pattern implementation +// Event metadata (correlationId, traceId, timestamp) +``` + +### Phase 4: Event Consuming Patterns + +**Topics:** + +- Consumer service patterns +- Consumer groups và partition assignment +- Processing strategies (at-least-once, exactly-once) +- Commit strategies +- Dead letter queue (DLQ) pattern +- Consumer error handling + +**Patterns:** + +- Simple consumer +- Batch consumer +- Transactional consumer +- DLQ handling +- Retry patterns với exponential backoff + +**Code Examples:** + +```typescript +// EventConsumer class +// Consumer group management +// Message processing với error handling +// DLQ implementation +``` + +### Phase 5: Schema Versioning & Registry + +**Topics:** + +- Schema Registry setup (Confluent) +- Avro schema definitions +- Schema evolution (backward/forward compatible) +- Schema versioning strategy +- Schema validation + +**Code Examples:** + +```typescript +// Schema definitions +// Schema registry integration +// Schema evolution examples +// Validation patterns +``` + +### Phase 6: Traefik Integration (SSE/WebSocket) + +**Topics:** + +- Exposing events qua HTTP endpoints +- Server-Sent Events (SSE) pattern +- WebSocket support (optional) +- Traefik routing configuration +- Authentication/authorization cho event streams +- Rate limiting cho event streams + +**Patterns:** + +- SSE endpoint implementation +- Event filtering và subscription +- Connection management +- Traefik labels configuration + +**Code Examples:** + +```typescript +// SSE endpoint implementation +// Event stream service +// Traefik routing config +// Client subscription patterns +``` + +### Phase 7: Error Handling & Resilience + +**Topics:** + +- Event publishing failures +- Consumer error handling +- Retry strategies +- Circuit breaker cho Kafka operations +- Dead letter queue management +- Event replay patterns + +**Integration với existing skills:** + +- Reuse resilience-patterns (circuit breaker, retry) +- Reuse error-handling-patterns (custom errors) + +**Code Examples:** + +```typescript +// Error handling trong publisher +// Error handling trong consumer +// DLQ processing +// Event replay service +``` + +### Phase 8: Observability + +**Topics:** + +- Logging events (structured logging) +- Metrics (events published/consumed, lag, throughput) +- Distributed tracing với OpenTelemetry +- Health checks cho Kafka connectivity +- Monitoring consumer lag + +**Integration:** + +- Use @goodgo/logger +- Use @goodgo/tracing +- Prometheus metrics + +**Code Examples:** + +```typescript +// Event logging +// Metrics collection +// Tracing events +// Health check endpoints +``` + +### Phase 9: Testing Patterns + +**Topics:** + +- Unit testing publishers/consumers +- Integration testing với Kafka +- Test containers cho local testing +- Mocking Kafka trong tests +- E2E event flow testing + +**Code Examples:** + +```typescript +// Unit test examples +// Integration test setup +// Test containers usage +// Mocking strategies +``` + +### Phase 10: Best Practices & Patterns + +**Topics:** + +- Event naming conventions (`domain.action.version`) +- Topic naming (`domain.entity.action`) +- Partition key selection +- Event ordering guarantees +- Event size limits +- Performance optimization +- Common anti-patterns to avoid + +**Examples:** + +- Complete event flow example +- Service integration examples +- Real-world use cases + +### Phase 11: Documentation Examples + +**Complete Examples:** + +1. UserCreated event flow (Auth Service → Notification Service) +2. OrderPlaced event với multiple consumers +3. SSE endpoint với filtering +4. Outbox pattern implementation +5. Event replay scenario + +**Integration Examples:** + +- Service publishing events +- Service consuming events +- Traefik routing SSE endpoints +- Error recovery flows + +## Skill File Structure (Detailed Outline) + +```markdown +--- +name: event-driven-architecture +description: Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. +--- + +# Event-Driven Architecture Patterns + +## When to Use This Skill +[Use cases và scenarios] + +## Core Concepts +[Event-driven fundamentals, Kafka basics, Traefik integration] + +## Kafka Setup & Configuration +[Infrastructure, client setup, configuration] + +## Event Publishing Patterns +[Publisher patterns, outbox pattern, best practices] + +## Event Consuming Patterns +[Consumer patterns, DLQ, processing strategies] + +## Schema Versioning +[Schema Registry, Avro, evolution strategies] + +## Traefik Integration (SSE/WebSocket) +[Exposing events via HTTP, SSE endpoints, routing] + +## Error Handling & Resilience +[Error patterns, retries, DLQ, replay] + +## Observability +[Logging, metrics, tracing events] + +## Testing Patterns +[Unit, integration, E2E testing] + +## Best Practices +[Naming conventions, patterns, anti-patterns] + +## Complete Examples +[End-to-end examples] + +## Resources +[Links to related skills, docs] +``` + +## Key Design Decisions + +1. **Kafka over RabbitMQ**: User selected Kafka for high-throughput and streaming capabilities +2. **Traefik Integration**: Expose events via SSE/WebSocket through Traefik, keeping Traefik as API Gateway +3. **Schema Registry**: Use Confluent Schema Registry với Avro for schema versioning +4. **Outbox Pattern**: Include outbox pattern for transactional event publishing +5. **Consistency với existing skills**: Reuse patterns từ resilience-patterns, error-handling-patterns, observability-monitoring + +## Dependencies & Integration Points + +**Existing Skills to Reference:** + +- `resilience-patterns` - Circuit breaker, retry patterns +- `error-handling-patterns` - Error classes và handling +- `observability-monitoring` - Logging, metrics, tracing +- `middleware-patterns` - SSE endpoint middleware +- `project-rules` - Coding standards, naming conventions + +**Infrastructure:** + +- Kafka cluster setup (docker-compose, K8s) +- Schema Registry setup +- Traefik configuration updates + +## Acceptance Criteria + +- [ ] Skill file created với comprehensive coverage +- [ ] All code examples are working và follow GoodGo patterns +- [ ] Bilingual comments (EN/VI) trong code examples +- [ ] Integration examples với Traefik SSE/WebSocket +- [ ] Error handling patterns documented +- [ ] Observability patterns documented +- [ ] Testing patterns documented +- [ ] References to related skills +- [ ] Complete end-to-end examples +- [ ] Consistent với existing skill format và style \ No newline at end of file diff --git a/.cursor/plans/fix_template_structure_870b6de9.plan.md b/.cursor/plans/fix_template_structure_870b6de9.plan.md index f0845d98..faa2512d 100644 --- a/.cursor/plans/fix_template_structure_870b6de9.plan.md +++ b/.cursor/plans/fix_template_structure_870b6de9.plan.md @@ -67,9 +67,9 @@ The `services/_template` currently contains configurations that conflict with th services: my-service: labels: - - "traefik.enable=true" - - "traefik.http.routers.my-service.rule=PathPrefix(`/api/v1/my-service`)" - - "traefik.http.services.my-service.loadbalancer.server.port=5000" + - "traefik.enable=true" + - "traefik.http.routers.my-service.rule=PathPrefix(`/api/v1/my-service`)" + - "traefik.http.services.my-service.loadbalancer.server.port=5000" ``` ### 3. Refactor ARCHITECTURE.md @@ -163,24 +163,24 @@ graph TD 1. Remove "Option 2: Docker Development" section (lines 73-91) - - This references the deleted docker-compose.yml - - Replace with reference to platform deployment + - This references the deleted docker-compose.yml + - Replace with reference to platform deployment 2. Update "Getting Started" section: - - Point to `deployments/local/docker-compose.yml` for full platform - - Explain how to add this service to the platform - - Keep local development instructions (without docker-compose) + - Point to `deployments/local/docker-compose.yml` for full platform + - Explain how to add this service to the platform + - Keep local development instructions (without docker-compose) 3. Remove "Docker Compose Files" section (lines 472-476) - - Only docker-compose.test.yml remains + - Only docker-compose.test.yml remains 4. Update "Traefik API Gateway" section (lines 494+) - - Remove references to local traefik/ directory - - Point to `infra/traefik/` for configuration - - Explain Docker labels for service registration + - Remove references to local traefik/ directory + - Point to `infra/traefik/` for configuration + - Explain Docker labels for service registration **New section to add:** @@ -197,20 +197,20 @@ services: dockerfile: services/your-service/Dockerfile container_name: your-service-local environment: - - NODE_ENV=development - - PORT=5002 - - DATABASE_URL=${DATABASE_URL} - - REDIS_HOST=redis - - JWT_SECRET=${JWT_SECRET} + - NODE_ENV=development + - PORT=5002 + - DATABASE_URL=${DATABASE_URL} + - REDIS_HOST=redis + - JWT_SECRET=${JWT_SECRET} depends_on: redis: condition: service_healthy networks: - - microservices-network + - microservices-network labels: - - "traefik.enable=true" - - "traefik.http.routers.your-service.rule=PathPrefix(`/api/v1/your-service`)" - - "traefik.http.services.your-service.loadbalancer.server.port=5002" + - "traefik.enable=true" + - "traefik.http.routers.your-service.rule=PathPrefix(`/api/v1/your-service`)" + - "traefik.http.services.your-service.loadbalancer.server.port=5002" ```` ### 2. Start the Platform @@ -235,16 +235,16 @@ docker-compose up -d 3. Delete `services/_template/traefik/` directory 4. Refactor `services/_template/ARCHITECTURE.md`: - - Add clear section headers for Single Service vs Platform Integration - - Update diagrams to show correct context - - Add deployment context section + - Add clear section headers for Single Service vs Platform Integration + - Update diagrams to show correct context + - Add deployment context section 5. Update `services/_template/README.md`: - - Remove Docker Development option - - Remove Docker Compose Files section - - Update Traefik references - - Add "Adding This Service to the Platform" section + - Remove Docker Development option + - Remove Docker Compose Files section + - Update Traefik references + - Add "Adding This Service to the Platform" section 6. Keep `services/_template/docker-compose.test.yml` for isolated testing diff --git a/.cursor/skills/api-gateway-advanced/SKILL.md b/.cursor/skills/api-gateway-advanced/SKILL.md new file mode 100644 index 00000000..3cfa541b --- /dev/null +++ b/.cursor/skills/api-gateway-advanced/SKILL.md @@ -0,0 +1,428 @@ +--- +name: api-gateway-advanced +description: Advanced API Gateway patterns for GoodGo microservices including API composition, request/response transformation, service mesh integration, advanced routing, and gateway-level resilience. Use when implementing API aggregation, service composition, or advanced gateway features. +--- + +# API Gateway Advanced Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing API composition and aggregation +- Transforming requests/responses at gateway level +- Integrating service mesh (Istio/Linkerd) with Traefik +- Implementing advanced routing strategies +- Adding gateway-level circuit breakers +- Implementing API versioning at gateway +- Building API composition services +- Optimizing gateway performance +- Implementing request/response caching at gateway +- Managing complex routing rules + +## Core Concepts + +### API Gateway Responsibilities + +1. **Request Routing**: Route requests to appropriate services +2. **API Composition**: Aggregate multiple service responses +3. **Protocol Translation**: Convert between protocols (HTTP, gRPC, GraphQL) +4. **Request/Response Transformation**: Modify requests and responses +5. **Security**: Authentication, authorization, rate limiting +6. **Resilience**: Circuit breaker, retry, timeout at gateway level +7. **Observability**: Logging, metrics, tracing + +### API Composition Patterns + +**Aggregation**: Combine multiple service responses into single response +**Chaining**: Call services sequentially, use previous response in next call +**Fan-out/Fan-in**: Call multiple services in parallel, aggregate results + +## API Composition Service + +```typescript +// src/modules/gateway/api-composition.service.ts +// EN: API composition service +// VI: Service API composition +import { ServiceClient } from '../../core/clients/service-client'; +import { logger } from '@goodgo/logger'; + +export class ApiCompositionService { + private userClient: ServiceClient; + private orderClient: ServiceClient; + private paymentClient: ServiceClient; + + constructor() { + this.userClient = new ServiceClient({ + baseURL: process.env.USER_SERVICE_URL || 'http://user-service:5002', + serviceName: 'user-service', + }); + + this.orderClient = new ServiceClient({ + baseURL: process.env.ORDER_SERVICE_URL || 'http://order-service:5003', + serviceName: 'order-service', + }); + + this.paymentClient = new ServiceClient({ + baseURL: process.env.PAYMENT_SERVICE_URL || 'http://payment-service:5004', + serviceName: 'payment-service', + }); + } + + /** + * EN: Aggregate user profile with orders and payment history + * VI: Tổng hợp profile user với orders và payment history + */ + async getUserProfile(userId: string): Promise { + try { + // EN: Fan-out: Call multiple services in parallel + // VI: Fan-out: Gọi nhiều services song song + const [user, orders, payments] = await Promise.all([ + this.userClient.get(`/api/v1/users/${userId}`), + this.orderClient.get(`/api/v1/orders?userId=${userId}`), + this.paymentClient.get(`/api/v1/payments?userId=${userId}`), + ]); + + // EN: Transform and aggregate response + // VI: Transform và tổng hợp response + return { + success: true, + data: { + user: user.data, + orders: orders.data.orders, + paymentHistory: payments.data.payments, + summary: { + totalOrders: orders.data.total, + totalSpent: payments.data.totalAmount, + }, + }, + }; + } catch (error) { + logger.error('API composition failed', { userId, error }); + throw error; + } + } + + /** + * EN: Chain services: Create order then process payment + * VI: Chain services: Tạo order rồi xử lý payment + */ + async createOrderWithPayment(orderData: any): Promise { + // EN: Step 1: Create order + // VI: Bước 1: Tạo order + const order = await this.orderClient.post('/api/v1/orders', orderData); + + try { + // EN: Step 2: Process payment using order data + // VI: Bước 2: Xử lý payment sử dụng order data + const payment = await this.paymentClient.post('/api/v1/payments', { + orderId: order.data.id, + amount: order.data.total, + paymentMethod: orderData.paymentMethod, + }); + + return { + success: true, + data: { + order: order.data, + payment: payment.data, + }, + }; + } catch (error) { + // EN: Compensate: Cancel order if payment fails + // VI: Compensate: Hủy order nếu payment fails + await this.orderClient.delete(`/api/v1/orders/${order.data.id}`); + throw error; + } + } +} +``` + +## Request/Response Transformation + +```typescript +// src/middlewares/gateway-transform.middleware.ts +// EN: Gateway transformation middleware +// VI: Middleware transformation gateway +import { Request, Response, NextFunction } from 'express'; +import { logger } from '@goodgo/logger'; + +export interface TransformRule { + path: string; + requestTransform?: (req: Request) => Request; + responseTransform?: (res: Response, data: any) => any; +} + +export class GatewayTransformMiddleware { + private rules: TransformRule[] = []; + + addRule(rule: TransformRule): void { + this.rules.push(rule); + } + + middleware() { + return (req: Request, res: Response, next: NextFunction) => { + // EN: Find matching transform rule + // VI: Tìm transform rule khớp + const rule = this.rules.find((r) => req.path.startsWith(r.path)); + + if (rule?.requestTransform) { + // EN: Transform request + // VI: Transform request + Object.assign(req, rule.requestTransform(req)); + } + + // EN: Store original json method + // VI: Lưu method json gốc + const originalJson = res.json.bind(res); + + // EN: Override json to transform response + // VI: Override json để transform response + res.json = (data: any) => { + if (rule?.responseTransform) { + data = rule.responseTransform(res, data); + } + return originalJson(data); + }; + + next(); + }; + } +} + +// Usage +const transformer = new GatewayTransformMiddleware(); + +transformer.addRule({ + path: '/api/v1/users', + requestTransform: (req) => { + // EN: Add default pagination + // VI: Thêm pagination mặc định + if (!req.query.page) req.query.page = '1'; + if (!req.query.limit) req.query.limit = '10'; + return req; + }, + responseTransform: (res, data) => { + // EN: Standardize response format + // VI: Chuẩn hóa format response + return { + success: true, + data: data.data || data, + metadata: { + timestamp: new Date().toISOString(), + }, + }; + }, +}); +``` + +## Advanced Traefik Configuration + +```yaml +# infra/traefik/dynamic/routes.yml +http: + routers: + # EN: API composition route + # VI: Route API composition + user-profile: + rule: "Path(`/api/v1/user-profile/{userId}`)" + service: api-composition-service + middlewares: + - cors + - compress + - rate-limit + + # EN: Versioned routes + # VI: Routes có version + user-service-v1: + rule: "PathPrefix(`/api/v1/users`)" + service: user-service-v1 + priority: 10 + + user-service-v2: + rule: "PathPrefix(`/api/v2/users`)" + service: user-service-v2 + priority: 5 # EN: Lower priority / VI: Độ ưu tiên thấp hơn + + services: + api-composition-service: + loadBalancer: + servers: + - url: "http://api-composition-service:5000" + + middlewares: + # EN: Request transformation + # VI: Transform request + add-default-headers: + headers: + customRequestHeaders: + X-Request-ID: "{{.RequestHeader.X-Request-ID}}" + X-Source: "traefik-gateway" + + # EN: Response transformation + # VI: Transform response + add-response-headers: + headers: + customResponseHeaders: + X-Response-Time: "{{.ResponseTime}}" + + # EN: Circuit breaker at gateway + # VI: Circuit breaker ở gateway + circuit-breaker: + circuitBreaker: + expression: "NetworkErrorRatio() > 0.50" +``` + +## Service Mesh Integration + +```yaml +# EN: Traefik with Istio integration +# VI: Traefik với tích hợp Istio +apiVersion: networking.istio.io/v1alpha3 +kind: VirtualService +metadata: + name: traefik-gateway +spec: + hosts: + - api.goodgo.com + gateways: + - traefik-gateway + http: + - match: + - uri: + prefix: "/api/v1" + route: + - destination: + host: user-service + port: + number: 5002 + - destination: + host: order-service + port: + number: 5003 + fault: + delay: + percentage: + value: 0.1 + fixedDelay: 5s + retries: + attempts: 3 + perTryTimeout: 2s +``` + +## Gateway-Level Circuit Breaker + +```typescript +// src/core/gateway/circuit-breaker.middleware.ts +// EN: Gateway-level circuit breaker +// VI: Circuit breaker ở gateway level +import { Request, Response, NextFunction } from 'express'; +import { createCircuitBreaker } from '../resilience/circuit-breaker'; +import { logger } from '@goodgo/logger'; + +const serviceCircuitBreakers = new Map>(); + +export function gatewayCircuitBreaker(serviceName: string) { + return (req: Request, res: Response, next: NextFunction) => { + if (!serviceCircuitBreakers.has(serviceName)) { + serviceCircuitBreakers.set( + serviceName, + createCircuitBreaker( + async () => { + // EN: Continue to next middleware + // VI: Tiếp tục tới middleware tiếp theo + return next(); + }, + `gateway-${serviceName}`, + { + timeout: 5000, + errorThresholdPercentage: 50, + resetTimeout: 30000, + } + ) + ); + } + + const breaker = serviceCircuitBreakers.get(serviceName)!; + + breaker.fire().catch((error) => { + logger.error('Gateway circuit breaker opened', { serviceName, error }); + res.status(503).json({ + success: false, + error: { + code: 'SERVICE_UNAVAILABLE', + message: `Service ${serviceName} is currently unavailable`, + }, + }); + }); + }; +} +``` + +## Caching at Gateway + +```typescript +// src/core/gateway/gateway-cache.middleware.ts +// EN: Gateway-level caching +// VI: Caching ở gateway level +import { Request, Response, NextFunction } from 'express'; +import { cacheService } from '../cache/cache.service'; +import { logger } from '@goodgo/logger'; + +export function gatewayCache(ttl: number = 60) { + return async (req: Request, res: Response, next: NextFunction) => { + // EN: Only cache GET requests + // VI: Chỉ cache GET requests + if (req.method !== 'GET') { + return next(); + } + + const cacheKey = `gateway:${req.path}:${JSON.stringify(req.query)}`; + + try { + // EN: Check cache + // VI: Kiểm tra cache + const cached = await cacheService.get(cacheKey); + if (cached) { + logger.debug('Gateway cache hit', { path: req.path }); + return res.json(cached); + } + + // EN: Store original json method + // VI: Lưu method json gốc + const originalJson = res.json.bind(res); + + // EN: Override to cache response + // VI: Override để cache response + res.json = (data: any) => { + cacheService.set(cacheKey, data, ttl).catch((error) => { + logger.warn('Gateway cache set failed', { error }); + }); + return originalJson(data); + }; + + next(); + } catch (error) { + logger.error('Gateway cache error', { error }); + next(); // EN: Continue on cache error / VI: Tiếp tục khi có lỗi cache + } + }; +} +``` + +## Best Practices + +1. **API Composition**: Use for aggregating related data +2. **Caching**: Cache at gateway for frequently accessed data +3. **Circuit Breaker**: Implement at gateway to protect services +4. **Transformation**: Keep transformations simple and testable +5. **Versioning**: Use path-based or header-based versioning +6. **Monitoring**: Monitor gateway metrics (latency, error rate) + +## Resources + +- [Traefik Documentation](https://doc.traefik.io/traefik/) - Official Traefik docs +- [API Gateway Pattern](https://microservices.io/patterns/apigateway.html) - Gateway patterns +- [Service Mesh](https://istio.io/) - Istio service mesh +- [Middleware Patterns](../middleware-patterns/SKILL.md) - Middleware patterns +- [Resilience Patterns](../resilience-patterns/SKILL.md) - Circuit breaker patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/api-versioning-strategy/SKILL.md b/.cursor/skills/api-versioning-strategy/SKILL.md new file mode 100644 index 00000000..2686b028 --- /dev/null +++ b/.cursor/skills/api-versioning-strategy/SKILL.md @@ -0,0 +1,358 @@ +--- +name: api-versioning-strategy +description: API versioning strategies for GoodGo microservices including semantic versioning, backward compatibility patterns, API deprecation, version negotiation, and breaking changes handling. Use when versioning APIs, handling breaking changes, or implementing API deprecation strategies. +--- + +# API Versioning Strategy + +## When to Use This Skill + +Use this skill when: +- Versioning APIs +- Handling breaking changes +- Implementing API deprecation +- Maintaining backward compatibility +- Implementing version negotiation +- Managing multiple API versions +- Planning API evolution +- Communicating API changes to consumers + +## Core Concepts + +### Versioning Strategies + +1. **URL Path Versioning**: `/api/v1/users`, `/api/v2/users` +2. **Header Versioning**: `Accept: application/vnd.goodgo.v1+json` +3. **Query Parameter**: `/api/users?version=1` +4. **Semantic Versioning**: Major.Minor.Patch (e.g., 1.2.3) + +### Compatibility Types + +- **Backward Compatible**: New version works with old clients +- **Forward Compatible**: Old version works with new clients +- **Breaking Changes**: Incompatible changes requiring new version + +## URL Path Versioning + +### Implementation + +```typescript +// src/routes/index.ts +// EN: Route versioning +// VI: Route versioning +import { Router } from 'express'; +import v1Router from './v1'; +import v2Router from './v2'; + +const router = Router(); + +// EN: Version 1 routes +// VI: Routes version 1 +router.use('/v1', v1Router); + +// EN: Version 2 routes +// VI: Routes version 2 +router.use('/v2', v2Router); + +export default router; +``` + +### Version Router + +```typescript +// src/routes/v1/index.ts +// EN: Version 1 routes +// VI: Routes version 1 +import { Router } from 'express'; +import userRoutes from './users'; + +const router = Router(); + +router.use('/users', userRoutes); + +export default router; + +// src/routes/v2/index.ts +// EN: Version 2 routes with breaking changes +// VI: Routes version 2 với breaking changes +import { Router } from 'express'; +import userRoutes from './users'; // EN: Different implementation / VI: Implementation khác + +const router = Router(); + +router.use('/users', userRoutes); // EN: Different response format / VI: Format response khác + +export default router; +``` + +## Header-Based Versioning + +### Version Negotiation Middleware + +```typescript +// src/middlewares/version-negotiation.middleware.ts +// EN: Version negotiation middleware +// VI: Middleware version negotiation +import { Request, Response, NextFunction } from 'express'; +import { logger } from '@goodgo/logger'; + +export function versionNegotiation( + req: Request, + res: Response, + next: NextFunction +): void { + // EN: Extract version from Accept header + // VI: Trích xuất version từ Accept header + const acceptHeader = req.headers.accept || ''; + const versionMatch = acceptHeader.match(/application\/vnd\.goodgo\.v(\d+)\+json/); + + if (versionMatch) { + const requestedVersion = parseInt(versionMatch[1], 10); + req.apiVersion = requestedVersion; + + // EN: Check if version is supported + // VI: Kiểm tra xem version có được hỗ trợ không + const supportedVersions = [1, 2]; + if (!supportedVersions.includes(requestedVersion)) { + return res.status(400).json({ + success: false, + error: { + code: 'UNSUPPORTED_VERSION', + message: `API version ${requestedVersion} is not supported. Supported versions: ${supportedVersions.join(', ')}`, + }, + }); + } + } else { + // EN: Default to latest version + // VI: Mặc định version mới nhất + req.apiVersion = 2; + } + + next(); +} +``` + +### Version-Aware Controller + +```typescript +// src/modules/user/user.controller.ts +// EN: Version-aware controller +// VI: Controller nhận biết version +export class UserController { + async getUser(req: Request, res: Response): Promise { + const version = req.apiVersion || 2; + + if (version === 1) { + // EN: Version 1 response format + // VI: Format response version 1 + const user = await this.userService.findById(req.params.id); + res.json({ + id: user.id, + email: user.email, + name: user.name, + }); + } else { + // EN: Version 2 response format + // VI: Format response version 2 + const user = await this.userService.findById(req.params.id); + res.json({ + success: true, + data: { + user: { + id: user.id, + email: user.email, + name: user.name, + profile: user.profile, // EN: New field in v2 / VI: Field mới trong v2 + }, + }, + metadata: { + version: '2.0.0', + timestamp: new Date().toISOString(), + }, + }); + } + } +} +``` + +## Semantic Versioning + +### Version Structure + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking changes +MINOR: Backward-compatible additions +PATCH: Backward-compatible bug fixes +``` + +### Version Response + +```typescript +// src/core/api/version.middleware.ts +// EN: Add version to response +// VI: Thêm version vào response +export function versionMiddleware(req: Request, res: Response, next: NextFunction): void { + const originalJson = res.json.bind(res); + + res.json = (data: any) => { + const response = { + ...data, + metadata: { + ...data.metadata, + apiVersion: req.apiVersion || '2.0.0', + serviceVersion: process.env.SERVICE_VERSION || '1.0.0', + }, + }; + + return originalJson(response); + }; + + next(); +} +``` + +## API Deprecation + +### Deprecation Headers + +```typescript +// src/middlewares/deprecation.middleware.ts +// EN: Deprecation warning middleware +// VI: Middleware cảnh báo deprecation +export function deprecationMiddleware(version: string, sunsetDate: string) { + return (req: Request, res: Response, next: NextFunction): void => { + if (req.apiVersion && parseInt(req.apiVersion.toString()) < parseInt(version)) { + res.setHeader('Deprecation', 'true'); + res.setHeader('Sunset', sunsetDate); + res.setHeader('Link', `<${req.url.replace(/\/v\d+/, `/v${version}`)}>; rel="successor-version"`); + res.setHeader('Warning', `299 - "API version ${req.apiVersion} is deprecated. Please migrate to version ${version} by ${sunsetDate}"`); + } + + next(); + }; +} + +// Usage +router.use('/v1', deprecationMiddleware('2', '2024-12-31'), v1Router); +``` + +### Deprecation Documentation + +```typescript +// src/docs/deprecation.md +// EN: Deprecation notices +// VI: Thông báo deprecation + +## API Version 1 Deprecation + +**Deprecated**: 2024-01-01 +**Sunset Date**: 2024-12-31 +**Migration Guide**: [Migration Guide](./migration-v1-to-v2.md) + +### Changes in Version 2 + +- New response format +- Additional user profile fields +- Updated authentication flow +``` + +## Backward Compatibility + +### Compatibility Layer + +```typescript +// src/core/api/compatibility.adapter.ts +// EN: Backward compatibility adapter +// VI: Adapter tương thích ngược +export class CompatibilityAdapter { + /** + * EN: Adapt v1 response to v2 format + * VI: Adapt response v1 sang format v2 + */ + adaptV1ToV2(v1Data: any): any { + return { + success: true, + data: { + user: { + ...v1Data, + profile: null, // EN: Add default for new field / VI: Thêm mặc định cho field mới + }, + }, + metadata: { + version: '2.0.0', + adapted: true, + }, + }; + } + + /** + * EN: Adapt v2 request to v1 format + * VI: Adapt request v2 sang format v1 + */ + adaptV2RequestToV1(v2Request: any): any { + return { + email: v2Request.email, + name: v2Request.name, + // EN: Ignore new fields / VI: Bỏ qua các field mới + }; + } +} +``` + +## Breaking Changes Migration + +### Migration Strategy + +```typescript +// src/core/api/migration.strategy.ts +// EN: Migration strategy for breaking changes +// VI: Chiến lược migration cho breaking changes +export class MigrationStrategy { + /** + * EN: Phase 1: Support both versions + * VI: Giai đoạn 1: Hỗ trợ cả hai versions + */ + phase1SupportBoth(): void { + // EN: Keep v1, add v2 + // VI: Giữ v1, thêm v2 + router.use('/v1', v1Router); + router.use('/v2', v2Router); + } + + /** + * EN: Phase 2: Deprecate v1 + * VI: Giai đoạn 2: Deprecate v1 + */ + phase2DeprecateV1(): void { + router.use('/v1', deprecationMiddleware('2', '2024-12-31'), v1Router); + router.use('/v2', v2Router); + } + + /** + * EN: Phase 3: Remove v1 + * VI: Giai đoạn 3: Xóa v1 + */ + phase3RemoveV1(): void { + // EN: After sunset date, remove v1 routes + // VI: Sau sunset date, xóa v1 routes + router.use('/v2', v2Router); + } +} +``` + +## Best Practices + +1. **Versioning Strategy**: Choose URL path or header, be consistent +2. **Semantic Versioning**: Use MAJOR.MINOR.PATCH +3. **Deprecation**: Always deprecate before removing +4. **Migration Guide**: Provide clear migration documentation +5. **Backward Compatibility**: Maintain compatibility when possible +6. **Communication**: Clearly communicate version changes + +## Resources + +- [API Design](../api-design/SKILL.md) - API design patterns +- [Middleware Patterns](../middleware-patterns/SKILL.md) - Middleware patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo standards diff --git a/.cursor/skills/cicd-advanced-patterns/SKILL.md b/.cursor/skills/cicd-advanced-patterns/SKILL.md new file mode 100644 index 00000000..005ffdf8 --- /dev/null +++ b/.cursor/skills/cicd-advanced-patterns/SKILL.md @@ -0,0 +1,448 @@ +--- +name: cicd-advanced-patterns +description: Advanced CI/CD patterns for GoodGo microservices including blue-green deployments, canary releases, automated rollback, deployment verification, and progressive delivery. Use when implementing advanced deployment strategies, automated rollbacks, or progressive delivery pipelines. +--- + +# CI/CD Advanced Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing blue-green deployments +- Setting up canary releases +- Implementing automated rollback mechanisms +- Creating deployment verification pipelines +- Implementing progressive delivery +- Setting up deployment gates +- Implementing smoke tests +- Managing deployment strategies in Kubernetes + +## Core Concepts + +### Deployment Strategies + +1. **Rolling Update**: Gradual replacement (default K8s) +2. **Blue-Green**: Two identical environments, switch traffic +3. **Canary**: Gradual rollout to subset of users +4. **Recreate**: Stop old, start new (downtime) + +### Deployment Verification + +- Smoke tests +- Health checks +- Performance tests +- Rollback triggers + +## Blue-Green Deployment + +### Kubernetes Implementation + +```yaml +# deployments/production/kubernetes/user-service-blue.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: user-service-blue + labels: + app: user-service + version: blue +spec: + replicas: 3 + selector: + matchLabels: + app: user-service + version: blue + template: + metadata: + labels: + app: user-service + version: blue + spec: + containers: + - name: user-service + image: goodgo/user-service:v1.0.0 + ports: + - containerPort: 5000 + +--- +# deployments/production/kubernetes/user-service-green.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: user-service-green + labels: + app: user-service + version: green +spec: + replicas: 3 + selector: + matchLabels: + app: user-service + version: green + template: + metadata: + labels: + app: user-service + version: green + spec: + containers: + - name: user-service + image: goodgo/user-service:v1.1.0 + ports: + - containerPort: 5000 + +--- +# Service selector switches between blue/green +apiVersion: v1 +kind: Service +metadata: + name: user-service +spec: + selector: + app: user-service + version: blue # EN: Switch to green after verification / VI: Chuyển sang green sau khi xác minh + ports: + - port: 80 + targetPort: 5000 +``` + +### Blue-Green Switch Script + +```bash +#!/bin/bash +# scripts/deployment/blue-green-switch.sh +# EN: Switch between blue and green deployments +# VI: Chuyển đổi giữa blue và green deployments + +SERVICE_NAME=$1 +CURRENT_VERSION=$(kubectl get service $SERVICE_NAME -o jsonpath='{.spec.selector.version}') +NEW_VERSION=$([ "$CURRENT_VERSION" = "blue" ] && echo "green" || echo "blue") + +echo "Switching from $CURRENT_VERSION to $NEW_VERSION" + +# EN: Update service selector +# VI: Cập nhật service selector +kubectl patch service $SERVICE_NAME -p "{\"spec\":{\"selector\":{\"version\":\"$NEW_VERSION\"}}}" + +# EN: Wait for rollout +# VI: Đợi rollout +kubectl rollout status deployment/$SERVICE_NAME-$NEW_VERSION + +# EN: Run smoke tests +# VI: Chạy smoke tests +./scripts/deployment/smoke-tests.sh $SERVICE_NAME + +if [ $? -ne 0 ]; then + echo "Smoke tests failed, rolling back" + kubectl patch service $SERVICE_NAME -p "{\"spec\":{\"selector\":{\"version\":\"$CURRENT_VERSION\"}}}" + exit 1 +fi + +echo "Successfully switched to $NEW_VERSION" +``` + +## Canary Deployment + +### Kubernetes Canary with Service Mesh + +```yaml +# deployments/production/kubernetes/user-service-canary.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: user-service-canary + labels: + app: user-service + version: canary +spec: + replicas: 1 # EN: Start with 1 replica (10% traffic) / VI: Bắt đầu với 1 replica (10% traffic) + selector: + matchLabels: + app: user-service + version: canary + template: + metadata: + labels: + app: user-service + version: canary + spec: + containers: + - name: user-service + image: goodgo/user-service:v1.1.0 + +--- +# VirtualService splits traffic +apiVersion: networking.istio.io/v1alpha3 +kind: VirtualService +metadata: + name: user-service +spec: + hosts: + - user-service + http: + - match: + - headers: + canary: + exact: "true" + route: + - destination: + host: user-service + subset: canary + weight: 100 + - route: + - destination: + host: user-service + subset: stable + weight: 90 + - destination: + host: user-service + subset: canary + weight: 10 # EN: 10% traffic to canary / VI: 10% traffic tới canary +``` + +### Progressive Canary Rollout + +```bash +#!/bin/bash +# scripts/deployment/canary-rollout.sh +# EN: Progressive canary rollout +# VI: Progressive canary rollout + +SERVICE_NAME=$1 +CANARY_PERCENTAGES=(10 25 50 75 100) + +for PERCENTAGE in "${CANARY_PERCENTAGES[@]}"; do + echo "Rolling out to $PERCENTAGE%" + + # EN: Update VirtualService weight + # VI: Cập nhật VirtualService weight + kubectl patch virtualservice $SERVICE_NAME -p "{\"spec\":{\"http\":[{\"route\":[{\"destination\":{\"subset\":\"canary\"},\"weight\":$PERCENTAGE},{\"destination\":{\"subset\":\"stable\"},\"weight\":$((100-PERCENTAGE))}]}]}}" + + # EN: Wait for traffic to stabilize + # VI: Đợi traffic ổn định + sleep 60 + + # EN: Run health checks + # VI: Chạy health checks + ./scripts/deployment/health-checks.sh $SERVICE_NAME + + if [ $? -ne 0 ]; then + echo "Health checks failed at $PERCENTAGE%, rolling back" + kubectl patch virtualservice $SERVICE_NAME -p "{\"spec\":{\"http\":[{\"route\":[{\"destination\":{\"subset\":\"canary\"},\"weight\":0},{\"destination\":{\"subset\":\"stable\"},\"weight\":100}]}]}}" + exit 1 + fi + + echo "Successfully rolled out to $PERCENTAGE%" +done + +echo "Canary rollout complete" +``` + +## Automated Rollback + +### Rollback Script + +```bash +#!/bin/bash +# scripts/deployment/rollback.sh +# EN: Automated rollback to previous version +# VI: Rollback tự động về version trước + +SERVICE_NAME=$1 +NAMESPACE=${2:-production} + +# EN: Get previous deployment revision +# VI: Lấy revision deployment trước +PREVIOUS_REVISION=$(kubectl rollout history deployment/$SERVICE_NAME -n $NAMESPACE --no-headers | tail -1 | awk '{print $1}') + +if [ -z "$PREVIOUS_REVISION" ]; then + echo "No previous revision found" + exit 1 +fi + +echo "Rolling back to revision $PREVIOUS_REVISION" + +# EN: Rollback deployment +# VI: Rollback deployment +kubectl rollout undo deployment/$SERVICE_NAME -n $NAMESPACE --to-revision=$PREVIOUS_REVISION + +# EN: Wait for rollout +# VI: Đợi rollout +kubectl rollout status deployment/$SERVICE_NAME -n $NAMESPACE + +echo "Rollback complete" +``` + +### Automated Rollback on Failure + +```yaml +# .github/workflows/deploy-production.yml +name: Deploy Production + +on: + push: + branches: [main] + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Deploy to Kubernetes + run: | + kubectl apply -f deployments/production/kubernetes/ + kubectl rollout status deployment/user-service + + - name: Run Smoke Tests + run: ./scripts/deployment/smoke-tests.sh user-service + + - name: Rollback on Failure + if: failure() + run: ./scripts/deployment/rollback.sh user-service production +``` + +## Deployment Verification + +### Smoke Tests + +```typescript +// scripts/deployment/smoke-tests.ts +// EN: Smoke tests for deployment verification +// VI: Smoke tests để xác minh deployment +import axios from 'axios'; + +const SERVICE_URL = process.env.SERVICE_URL || 'http://localhost'; + +async function runSmokeTests(): Promise { + try { + // EN: Health check + // VI: Health check + const healthResponse = await axios.get(`${SERVICE_URL}/health`); + if (healthResponse.status !== 200) { + console.error('Health check failed'); + return false; + } + + // EN: Basic functionality test + // VI: Test chức năng cơ bản + const testResponse = await axios.get(`${SERVICE_URL}/api/v1/users`, { + timeout: 5000, + }); + + if (testResponse.status !== 200) { + console.error('Functionality test failed'); + return false; + } + + console.log('Smoke tests passed'); + return true; + } catch (error) { + console.error('Smoke tests failed', error); + return false; + } +} + +runSmokeTests().then((success) => { + process.exit(success ? 0 : 1); +}); +``` + +### Health Check Script + +```bash +#!/bin/bash +# scripts/deployment/health-checks.sh +# EN: Comprehensive health checks +# VI: Health checks toàn diện + +SERVICE_NAME=$1 +NAMESPACE=${2:-production} + +echo "Running health checks for $SERVICE_NAME" + +# EN: Check pods are ready +# VI: Kiểm tra pods đã ready +READY_PODS=$(kubectl get pods -n $NAMESPACE -l app=$SERVICE_NAME --field-selector=status.phase=Running --no-headers | wc -l) + +if [ $READY_PODS -eq 0 ]; then + echo "No ready pods found" + exit 1 +fi + +# EN: Check service endpoints +# VI: Kiểm tra service endpoints +ENDPOINTS=$(kubectl get endpoints $SERVICE_NAME -n $NAMESPACE -o jsonpath='{.subsets[0].addresses[*].ip}' | wc -w) + +if [ $ENDPOINTS -eq 0 ]; then + echo "No service endpoints found" + exit 1 +fi + +# EN: Check health endpoint +# VI: Kiểm tra health endpoint +SERVICE_URL=$(kubectl get service $SERVICE_NAME -n $NAMESPACE -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') + +if [ -z "$SERVICE_URL" ]; then + SERVICE_URL="http://$SERVICE_NAME.$NAMESPACE.svc.cluster.local" +fi + +HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" $SERVICE_URL/health) + +if [ $HTTP_CODE -ne 200 ]; then + echo "Health endpoint returned $HTTP_CODE" + exit 1 +fi + +echo "Health checks passed" +``` + +## Deployment Gates + +```yaml +# .github/workflows/deploy-with-gates.yml +name: Deploy with Gates + +jobs: + deploy: + steps: + - name: Deploy + run: kubectl apply -f deployments/ + + - name: Wait for Rollout + run: kubectl rollout status deployment/service + + - name: Smoke Tests Gate + id: smoke-tests + run: ./scripts/deployment/smoke-tests.sh + + - name: Performance Tests Gate + if: steps.smoke-tests.outcome == 'success' + run: ./scripts/deployment/performance-tests.sh + + - name: Manual Approval Gate + if: steps.smoke-tests.outcome == 'success' + uses: trstringer/manual-approval@v1 + with: + secret: ${{ secrets.GITHUB_TOKEN }} + approvers: team-leads + minimum-approvals: 1 + issue-title: "Approve deployment" +``` + +## Best Practices + +1. **Blue-Green**: Use for zero-downtime deployments +2. **Canary**: Use for gradual rollouts with monitoring +3. **Automated Rollback**: Always have rollback plan +4. **Smoke Tests**: Run immediately after deployment +5. **Health Checks**: Monitor health continuously +6. **Gates**: Use deployment gates for critical deployments + +## Resources + +- [Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) +- [Istio Traffic Management](https://istio.io/latest/docs/tasks/traffic-management/) +- [Deployment Kubernetes](../deployment-kubernetes/SKILL.md) - K8s deployment patterns +- [Testing Patterns](../testing-patterns/SKILL.md) - Testing strategies +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/configuration-management/SKILL.md b/.cursor/skills/configuration-management/SKILL.md new file mode 100644 index 00000000..6f904ea0 --- /dev/null +++ b/.cursor/skills/configuration-management/SKILL.md @@ -0,0 +1,412 @@ +--- +name: configuration-management +description: Configuration management patterns for GoodGo microservices including feature flags, dynamic configuration reloading, environment-specific configurations, and secrets management. Use when implementing feature toggles, managing configuration, or handling environment variables. +--- + +# Configuration Management Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing feature flags and feature toggles +- Managing environment-specific configurations +- Implementing dynamic configuration reloading +- Managing secrets and sensitive configuration +- Implementing configuration validation +- Handling configuration versioning +- Building configuration services +- Implementing A/B testing with feature flags +- Managing configuration across multiple environments + +## Core Concepts + +### Configuration Types + +1. **Static Configuration**: Environment variables, config files +2. **Dynamic Configuration**: Feature flags, runtime configs +3. **Secrets**: Sensitive data (API keys, passwords) +4. **Feature Flags**: Runtime feature toggles + +### Configuration Sources + +- Environment variables +- Configuration files (JSON, YAML) +- Configuration service (external) +- Feature flag service +- Secrets manager (Kubernetes Secrets, Vault) + +## Feature Flags + +### Feature Flag Service + +```typescript +// src/core/config/feature-flag.service.ts +// EN: Feature flag service +// VI: Service feature flag +import { PrismaClient } from '@prisma/client'; +import { logger } from '@goodgo/logger'; + +export interface FeatureFlag { + key: string; + enabled: boolean; + userId?: string; + percentage?: number; // EN: Percentage rollout / VI: Phần trăm rollout + metadata?: Record; +} + +export class FeatureFlagService { + constructor(private prisma: PrismaClient) {} + + /** + * EN: Check if feature is enabled + * VI: Kiểm tra xem feature có được bật không + */ + async isEnabled(flagKey: string, userId?: string): Promise { + const flag = await this.prisma.featureFlag.findUnique({ + where: { key: flagKey }, + }); + + if (!flag) { + return false; // EN: Default to disabled / VI: Mặc định tắt + } + + if (!flag.enabled) { + return false; + } + + // EN: User-specific flag + // VI: Flag cụ thể cho user + if (flag.userId && userId && flag.userId === userId) { + return true; + } + + // EN: Percentage rollout + // VI: Rollout theo phần trăm + if (flag.percentage !== null && userId) { + const hash = this.hashUserId(userId); + return hash < flag.percentage; + } + + return flag.enabled; + } + + private hashUserId(userId: string): number { + // EN: Simple hash for consistent percentage + // VI: Hash đơn giản cho phần trăm nhất quán + let hash = 0; + for (let i = 0; i < userId.length; i++) { + hash = ((hash << 5) - hash + userId.charCodeAt(i)) & 0xffffffff; + } + return Math.abs(hash) % 100; + } + + /** + * EN: Get all enabled features for user + * VI: Lấy tất cả features được bật cho user + */ + async getEnabledFeatures(userId?: string): Promise { + const flags = await this.prisma.featureFlag.findMany({ + where: { enabled: true }, + }); + + return flags + .filter((flag) => { + if (flag.userId && userId) { + return flag.userId === userId; + } + if (flag.percentage !== null && userId) { + return this.hashUserId(userId) < flag.percentage; + } + return true; + }) + .map((flag) => flag.key); + } +} +``` + +### Feature Flag Usage + +```typescript +// src/modules/user/user.service.ts +// EN: Using feature flags in service +// VI: Sử dụng feature flags trong service +import { featureFlagService } from '../../core/config/feature-flag.service'; + +export class UserService { + async createUser(data: CreateUserDto): Promise { + // EN: Check feature flag + // VI: Kiểm tra feature flag + const newAuthFlowEnabled = await featureFlagService.isEnabled( + 'new-auth-flow', + data.userId + ); + + if (newAuthFlowEnabled) { + // EN: Use new authentication flow + // VI: Sử dụng authentication flow mới + return await this.createUserWithNewFlow(data); + } + + // EN: Use existing flow + // VI: Sử dụng flow hiện tại + return await this.createUserWithExistingFlow(data); + } +} +``` + +## Dynamic Configuration + +### Configuration Service + +```typescript +// src/core/config/config.service.ts +// EN: Dynamic configuration service +// VI: Service configuration động +import { EventEmitter } from 'events'; +import { logger } from '@goodgo/logger'; + +export interface ConfigValue { + key: string; + value: any; + environment?: string; + service?: string; + version?: number; +} + +export class ConfigService extends EventEmitter { + private config: Map = new Map(); + private loadInterval?: NodeJS.Timeout; + + /** + * EN: Load configuration from source + * VI: Tải configuration từ nguồn + */ + async load(): Promise { + try { + // EN: Load from database, file, or external service + // VI: Tải từ database, file, hoặc external service + const configs = await this.fetchConfigs(); + + for (const config of configs) { + const oldValue = this.config.get(config.key); + this.config.set(config.key, config.value); + + // EN: Emit change event if value changed + // VI: Emit event thay đổi nếu giá trị thay đổi + if (oldValue !== config.value) { + this.emit('config-changed', { + key: config.key, + oldValue, + newValue: config.value, + }); + } + } + + logger.info('Configuration loaded', { count: configs.length }); + } catch (error) { + logger.error('Failed to load configuration', { error }); + throw error; + } + } + + /** + * EN: Get configuration value + * VI: Lấy giá trị configuration + */ + get(key: string, defaultValue?: T): T { + return (this.config.get(key) as T) ?? defaultValue!; + } + + /** + * EN: Start auto-reload + * VI: Bắt đầu auto-reload + */ + startAutoReload(intervalMs: number = 60000): void { + this.loadInterval = setInterval(() => { + this.load().catch((error) => { + logger.error('Auto-reload failed', { error }); + }); + }, intervalMs); + } + + /** + * EN: Stop auto-reload + * VI: Dừng auto-reload + */ + stopAutoReload(): void { + if (this.loadInterval) { + clearInterval(this.loadInterval); + } + } + + private async fetchConfigs(): Promise { + // EN: Fetch from database or external service + // VI: Fetch từ database hoặc external service + // Implementation depends on storage backend + return []; + } +} + +export const configService = new ConfigService(); + +// EN: Listen for config changes +// VI: Lắng nghe thay đổi config +configService.on('config-changed', ({ key, oldValue, newValue }) => { + logger.info('Configuration changed', { key, oldValue, newValue }); +}); +``` + +## Configuration Validation + +```typescript +// src/core/config/config-validator.ts +// EN: Configuration validation with Zod +// VI: Validation configuration với Zod +import { z } from 'zod'; +import { logger } from '@goodgo/logger'; + +const AppConfigSchema = z.object({ + port: z.number().int().min(1).max(65535), + nodeEnv: z.enum(['development', 'staging', 'production']), + databaseUrl: z.string().url(), + redisUrl: z.string().url().optional(), + logLevel: z.enum(['error', 'warn', 'info', 'debug']).default('info'), + jwtSecret: z.string().min(32), + jwtExpiresIn: z.string().default('15m'), +}); + +export type AppConfig = z.infer; + +export function validateConfig(env: Record): AppConfig { + try { + return AppConfigSchema.parse({ + port: parseInt(env.PORT || '5000', 10), + nodeEnv: env.NODE_ENV || 'development', + databaseUrl: env.DATABASE_URL, + redisUrl: env.REDIS_URL, + logLevel: env.LOG_LEVEL, + jwtSecret: env.JWT_SECRET, + jwtExpiresIn: env.JWT_EXPIRES_IN, + }); + } catch (error) { + if (error instanceof z.ZodError) { + logger.error('Configuration validation failed', { + errors: error.errors, + }); + throw new Error(`Invalid configuration: ${error.errors.map((e) => e.message).join(', ')}`); + } + throw error; + } +} +``` + +## Secrets Management + +```typescript +// src/core/config/secrets.service.ts +// EN: Secrets management service +// VI: Service quản lý secrets +import { logger } from '@goodgo/logger'; + +export class SecretsService { + private secrets: Map = new Map(); + private rotationInterval?: NodeJS.Timeout; + + /** + * EN: Get secret value + * VI: Lấy giá trị secret + */ + async getSecret(key: string): Promise { + if (this.secrets.has(key)) { + return this.secrets.get(key)!; + } + + // EN: Load from Kubernetes Secrets, Vault, or environment + // VI: Tải từ Kubernetes Secrets, Vault, hoặc environment + const value = await this.loadSecret(key); + this.secrets.set(key, value); + return value; + } + + /** + * EN: Rotate secrets periodically + * VI: Rotate secrets định kỳ + */ + startRotation(intervalMs: number = 3600000): void { + this.rotationInterval = setInterval(async () => { + logger.info('Rotating secrets'); + // EN: Reload all secrets + // VI: Tải lại tất cả secrets + for (const key of this.secrets.keys()) { + const newValue = await this.loadSecret(key); + this.secrets.set(key, newValue); + } + }, intervalMs); + } + + private async loadSecret(key: string): Promise { + // EN: Load from Kubernetes Secrets + // VI: Tải từ Kubernetes Secrets + if (process.env[`SECRET_${key.toUpperCase()}`]) { + return process.env[`SECRET_${key.toUpperCase()}`]!; + } + + // EN: Or load from external secrets manager + // VI: Hoặc tải từ external secrets manager + throw new Error(`Secret ${key} not found`); + } +} +``` + +## Environment-Specific Configuration + +```typescript +// src/config/app.config.ts +// EN: Environment-specific configuration +// VI: Configuration theo environment +import { validateConfig } from '../core/config/config-validator'; + +const envConfig = { + development: { + logLevel: 'debug', + enableSwagger: true, + databasePoolSize: 5, + }, + staging: { + logLevel: 'info', + enableSwagger: true, + databasePoolSize: 10, + }, + production: { + logLevel: 'warn', + enableSwagger: false, + databasePoolSize: 20, + }, +}; + +export function getAppConfig() { + const baseConfig = validateConfig(process.env); + const env = baseConfig.nodeEnv; + const envSpecific = envConfig[env] || envConfig.development; + + return { + ...baseConfig, + ...envSpecific, + }; +} +``` + +## Best Practices + +1. **Validation**: Always validate configuration at startup +2. **Defaults**: Provide sensible defaults +3. **Secrets**: Never commit secrets to code +4. **Feature Flags**: Use for gradual rollouts +5. **Reloading**: Support dynamic reloading where possible +6. **Monitoring**: Monitor configuration changes + +## Resources + +- [Feature Flags Pattern](https://martinfowler.com/articles/feature-toggles.html) +- [12-Factor App Config](https://12factor.net/config) +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/data-consistency-patterns/SKILL.md b/.cursor/skills/data-consistency-patterns/SKILL.md new file mode 100644 index 00000000..2d256173 --- /dev/null +++ b/.cursor/skills/data-consistency-patterns/SKILL.md @@ -0,0 +1,702 @@ +--- +name: data-consistency-patterns +description: Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services. +--- + +# Data Consistency Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing distributed transactions across multiple services +- Handling eventual consistency in microservices +- Implementing Saga patterns for distributed workflows +- Designing compensation strategies for failed transactions +- Implementing idempotent operations +- Managing data synchronization across services +- Handling conflict resolution +- Implementing optimistic locking strategies +- Building event sourcing systems +- Ensuring data integrity in distributed systems + +## Core Concepts + +### ACID vs BASE + +**ACID (Traditional Databases):** +- Atomicity: All or nothing +- Consistency: Data always valid +- Isolation: Concurrent transactions isolated +- Durability: Committed changes persist + +**BASE (Distributed Systems):** +- Basic Availability: System available most of the time +- Soft state: State may change over time +- Eventual consistency: Consistency achieved eventually + +### Consistency Models + +1. **Strong Consistency**: All nodes see same data at same time +2. **Weak Consistency**: No guarantees about when consistency occurs +3. **Eventual Consistency**: System becomes consistent over time + +### Distributed Transaction Challenges + +- Network partitions +- Service failures +- Clock synchronization +- Partial failures +- Two-Phase Commit (2PC) limitations + +## Saga Pattern + +### Orchestration vs Choreography + +**Orchestration (Centralized):** +- Central orchestrator coordinates steps +- Easier to understand and debug +- Single point of failure +- Use when: Complex workflows, need central control + +**Choreography (Decentralized):** +- Each service knows what to do next +- More resilient, no single point of failure +- Harder to understand overall flow +- Use when: Simple workflows, loose coupling preferred + +### Saga Orchestrator Pattern + +```typescript +// src/core/saga/saga-orchestrator.ts +// EN: Saga orchestrator for distributed transactions +// VI: Saga orchestrator cho distributed transactions +import { logger } from '@goodgo/logger'; +import { eventPublisher } from '../events/event-publisher'; + +export interface SagaStep { + name: string; + execute: () => Promise; + compensate: (context: any) => Promise; + retry?: number; +} + +export interface SagaContext { + sagaId: string; + steps: SagaStep[]; + currentStep: number; + data: Record; + status: 'pending' | 'running' | 'completed' | 'compensating' | 'failed'; +} + +export class SagaOrchestrator { + /** + * EN: Execute saga with all steps + * VI: Thực thi saga với tất cả các bước + */ + async execute(context: SagaContext): Promise { + context.status = 'running'; + + try { + for (let i = 0; i < context.steps.length; i++) { + context.currentStep = i; + const step = context.steps[i]; + + logger.info('Executing saga step', { + sagaId: context.sagaId, + step: step.name, + stepIndex: i, + }); + + try { + const result = await this.executeWithRetry(step, step.retry || 3); + context.data[step.name] = result; + + // EN: Publish step completed event + // VI: Publish event step đã hoàn thành + await eventPublisher.publish('saga.step.completed', { + eventType: 'saga.step.completed', + eventVersion: '1.0.0', + data: { + sagaId: context.sagaId, + step: step.name, + stepIndex: i, + }, + }); + } catch (error) { + logger.error('Saga step failed', { + sagaId: context.sagaId, + step: step.name, + error: error.message, + }); + + // EN: Compensate all completed steps + // VI: Compensate tất cả các bước đã hoàn thành + await this.compensate(context, i - 1); + throw error; + } + } + + context.status = 'completed'; + logger.info('Saga completed successfully', { sagaId: context.sagaId }); + } catch (error) { + context.status = 'failed'; + logger.error('Saga failed', { + sagaId: context.sagaId, + error: error.message, + }); + throw error; + } + } + + private async executeWithRetry(step: SagaStep, maxRetries: number): Promise { + let lastError: Error; + + for (let attempt = 0; attempt <= maxRetries; attempt++) { + try { + return await step.execute(); + } catch (error) { + lastError = error; + if (attempt < maxRetries) { + const delay = Math.pow(2, attempt) * 1000; // Exponential backoff + await new Promise((resolve) => setTimeout(resolve, delay)); + } + } + } + + throw lastError!; + } + + /** + * EN: Compensate completed steps in reverse order + * VI: Compensate các bước đã hoàn thành theo thứ tự ngược + */ + private async compensate(context: SagaContext, lastCompletedStep: number): Promise { + context.status = 'compensating'; + + for (let i = lastCompletedStep; i >= 0; i--) { + const step = context.steps[i]; + + try { + logger.info('Compensating saga step', { + sagaId: context.sagaId, + step: step.name, + stepIndex: i, + }); + + await step.compensate(context.data); + + await eventPublisher.publish('saga.step.compensated', { + eventType: 'saga.step.compensated', + eventVersion: '1.0.0', + data: { + sagaId: context.sagaId, + step: step.name, + stepIndex: i, + }, + }); + } catch (error) { + logger.error('Compensation failed', { + sagaId: context.sagaId, + step: step.name, + error: error.message, + }); + // EN: Log but continue compensating other steps + // VI: Log nhưng tiếp tục compensate các bước khác + } + } + } +} +``` + +### Saga Example: Order Processing + +```typescript +// src/modules/order/order.saga.ts +// EN: Order processing saga +// VI: Saga xử lý order +import { SagaOrchestrator, SagaContext, SagaStep } from '../../core/saga/saga-orchestrator'; +import { orderService } from './order.service'; +import { paymentService } from '../payment/payment.service'; +import { inventoryService } from '../inventory/inventory.service'; +import { notificationService } from '../notification/notification.service'; + +export class OrderProcessingSaga { + private orchestrator: SagaOrchestrator; + + constructor() { + this.orchestrator = new SagaOrchestrator(); + } + + async processOrder(orderData: any): Promise { + const sagaId = `saga_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`; + + const steps: SagaStep[] = [ + { + name: 'create-order', + execute: async () => { + return await orderService.create(orderData); + }, + compensate: async (context) => { + await orderService.cancel(context.data['create-order'].id); + }, + }, + { + name: 'reserve-inventory', + execute: async () => { + const order = context.data['create-order']; + return await inventoryService.reserve(order.items); + }, + compensate: async (context) => { + const reservation = context.data['reserve-inventory']; + await inventoryService.release(reservation.id); + }, + }, + { + name: 'process-payment', + execute: async () => { + const order = context.data['create-order']; + return await paymentService.charge(order.total, order.paymentMethod); + }, + compensate: async (context) => { + const payment = context.data['process-payment']; + await paymentService.refund(payment.id); + }, + }, + { + name: 'send-confirmation', + execute: async () => { + const order = context.data['create-order']; + await notificationService.sendOrderConfirmation(order.userId, order.id); + // EN: Non-critical step, no compensation needed + // VI: Bước không quan trọng, không cần compensation + }, + compensate: async () => { + // EN: No compensation needed for notification + // VI: Không cần compensation cho notification + }, + }, + ]; + + const context: SagaContext = { + sagaId, + steps, + currentStep: 0, + data: {}, + status: 'pending', + }; + + await this.orchestrator.execute(context); + } +} +``` + +### Saga Choreography Pattern + +```typescript +// EN: Choreography pattern - services react to events +// VI: Pattern choreography - services phản ứng với events +// src/modules/order/order.service.ts +export class OrderService { + async createOrder(data: CreateOrderDto): Promise { + const order = await this.orderRepository.create(data); + + // EN: Publish event for next step + // VI: Publish event cho bước tiếp theo + await eventPublisher.publish('order.created', { + eventType: 'order.created', + eventVersion: '1.0.0', + data: { + orderId: order.id, + items: order.items, + total: order.total, + }, + }); + + return order; + } + + async cancelOrder(orderId: string): Promise { + await this.orderRepository.update(orderId, { status: 'cancelled' }); + } +} + +// src/modules/inventory/inventory.consumer.ts +// EN: Inventory service reacts to order.created event +// VI: Inventory service phản ứng với order.created event +eventConsumer.on('order.created', { + handle: async (event) => { + try { + const reservation = await inventoryService.reserve(event.data.items); + + // EN: Publish next step event + // VI: Publish event bước tiếp theo + await eventPublisher.publish('inventory.reserved', { + eventType: 'inventory.reserved', + eventVersion: '1.0.0', + data: { + orderId: event.data.orderId, + reservationId: reservation.id, + }, + }); + } catch (error) { + // EN: Publish compensation event + // VI: Publish event compensation + await eventPublisher.publish('order.cancelled', { + eventType: 'order.cancelled', + eventVersion: '1.0.0', + data: { + orderId: event.data.orderId, + reason: 'inventory_reservation_failed', + }, + }); + } + }, +}); + +// src/modules/payment/payment.consumer.ts +eventConsumer.on('inventory.reserved', { + handle: async (event) => { + try { + const order = await orderService.findById(event.data.orderId); + const payment = await paymentService.charge(order.total, order.paymentMethod); + + await eventPublisher.publish('payment.processed', { + eventType: 'payment.processed', + eventVersion: '1.0.0', + data: { + orderId: event.data.orderId, + paymentId: payment.id, + }, + }); + } catch (error) { + // EN: Trigger compensation + // VI: Kích hoạt compensation + await eventPublisher.publish('order.cancelled', { + eventType: 'order.cancelled', + eventVersion: '1.0.0', + data: { + orderId: event.data.orderId, + reason: 'payment_failed', + }, + }); + } + }, +}); +``` + +## Idempotency Patterns + +### Idempotent Operations + +```typescript +// src/core/idempotency/idempotency.ts +// EN: Idempotency handler +// VI: Handler idempotency +import { PrismaClient } from '@prisma/client'; +import { logger } from '@goodgo/logger'; +import { v4 as uuidv4 } from 'uuid'; + +export class IdempotencyHandler { + constructor(private prisma: PrismaClient) {} + + /** + * EN: Execute operation with idempotency check + * VI: Thực thi operation với idempotency check + */ + async execute( + idempotencyKey: string, + operation: () => Promise, + ttl: number = 3600 // EN: 1 hour default / VI: Mặc định 1 giờ + ): Promise { + // EN: Check if already processed + // VI: Kiểm tra xem đã được xử lý chưa + const existing = await this.prisma.idempotencyRecord.findUnique({ + where: { idempotencyKey }, + }); + + if (existing) { + logger.info('Idempotent operation skipped', { idempotencyKey }); + return existing.result as T; + } + + // EN: Execute operation + // VI: Thực thi operation + try { + const result = await operation(); + + // EN: Store result + // VI: Lưu kết quả + await this.prisma.idempotencyRecord.create({ + data: { + idempotencyKey, + result: result as any, + expiresAt: new Date(Date.now() + ttl * 1000), + }, + }); + + return result; + } catch (error) { + // EN: Store error for idempotency (optional) + // VI: Lưu lỗi cho idempotency (tùy chọn) + logger.error('Idempotent operation failed', { idempotencyKey, error }); + throw error; + } + } +} + +// Usage +const idempotencyHandler = new IdempotencyHandler(prisma); + +await idempotencyHandler.execute( + `create-user-${userId}`, + async () => { + return await userService.create(userData); + } +); +``` + +### Idempotency Key Generation + +```typescript +// EN: Generate idempotency key from request +// VI: Tạo idempotency key từ request +export function generateIdempotencyKey( + userId: string, + operation: string, + requestData: any +): string { + const hash = createHash('sha256') + .update(JSON.stringify({ userId, operation, requestData })) + .digest('hex'); + + return `${operation}:${userId}:${hash.substr(0, 16)}`; +} +``` + +## Optimistic Locking + +### Version-Based Locking + +```typescript +// src/core/locking/optimistic-lock.ts +// EN: Optimistic locking with version field +// VI: Optimistic locking với trường version +export class OptimisticLockService { + /** + * EN: Update with optimistic lock + * VI: Update với optimistic lock + */ + async updateWithLock( + repository: any, + id: string, + updateFn: (current: T) => Partial, + maxRetries: number = 3 + ): Promise { + for (let attempt = 0; attempt < maxRetries; attempt++) { + // EN: Read current version + // VI: Đọc version hiện tại + const current = await repository.findById(id); + + if (!current) { + throw new Error('Resource not found'); + } + + // EN: Apply update + // VI: Áp dụng update + const updates = updateFn(current); + + try { + // EN: Update with version check + // VI: Update với kiểm tra version + const updated = await repository.update(id, { + ...updates, + version: current.version + 1, + }, { + where: { + id, + version: current.version, // EN: Only update if version matches / VI: Chỉ update nếu version khớp + }, + }); + + return updated; + } catch (error) { + if (attempt === maxRetries - 1) { + throw new Error('Optimistic lock conflict - max retries exceeded'); + } + + // EN: Retry with exponential backoff + // VI: Retry với exponential backoff + await new Promise((resolve) => + setTimeout(resolve, Math.pow(2, attempt) * 100) + ); + } + } + + throw new Error('Optimistic lock failed'); + } +} +``` + +## Eventual Consistency Strategies + +### Read Models and CQRS + +```typescript +// EN: Separate read and write models +// VI: Tách biệt read và write models +// src/modules/user/user.write.service.ts +export class UserWriteService { + async createUser(data: CreateUserDto): Promise { + const user = await this.userRepository.create(data); + + // EN: Publish event for read model update + // VI: Publish event để cập nhật read model + await eventPublisher.publish('user.created', { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { + userId: user.id, + email: user.email, + name: user.name, + }, + }); + + return user; + } +} + +// src/modules/user/user.read.service.ts +// EN: Read model updated via events +// VI: Read model được cập nhật qua events +export class UserReadService { + async getUser(userId: string): Promise { + return await this.userReadRepository.findById(userId); + } +} + +// src/modules/user/user.read.consumer.ts +eventConsumer.on('user.created', { + handle: async (event) => { + // EN: Update read model + // VI: Cập nhật read model + await userReadRepository.create({ + userId: event.data.userId, + email: event.data.email, + name: event.data.name, + // EN: Additional denormalized data for queries + // VI: Dữ liệu denormalized thêm cho queries + }); + }, +}); +``` + +### Conflict Resolution + +```typescript +// src/core/consistency/conflict-resolver.ts +// EN: Conflict resolution strategies +// VI: Các chiến lược giải quyết conflict +export enum ConflictResolutionStrategy { + LAST_WRITE_WINS = 'last_write_wins', + FIRST_WRITE_WINS = 'first_write_wins', + MERGE = 'merge', + MANUAL = 'manual', +} + +export class ConflictResolver { + resolve( + strategy: ConflictResolutionStrategy, + current: any, + incoming: any + ): any { + switch (strategy) { + case ConflictResolutionStrategy.LAST_WRITE_WINS: + return incoming.timestamp > current.timestamp ? incoming : current; + + case ConflictResolutionStrategy.FIRST_WRITE_WINS: + return incoming.timestamp < current.timestamp ? incoming : current; + + case ConflictResolutionStrategy.MERGE: + return { ...current, ...incoming, timestamp: Date.now() }; + + case ConflictResolutionStrategy.MANUAL: + // EN: Store conflict for manual resolution + // VI: Lưu conflict để giải quyết thủ công + this.storeConflict(current, incoming); + return current; + + default: + return current; + } + } +} +``` + +## Best Practices + +### Saga Pattern + +1. **Design Compensations**: Every step needs compensation +2. **Idempotent Steps**: Make steps idempotent for retries +3. **Timeout Handling**: Set timeouts for saga execution +4. **Monitoring**: Track saga execution and compensation +5. **Choreography vs Orchestration**: Choose based on complexity + +### Idempotency + +1. **Idempotency Keys**: Generate from request data +2. **Key Storage**: Store keys with results +3. **TTL**: Set appropriate TTL for idempotency records +4. **Cleanup**: Regularly clean expired records + +### Eventual Consistency + +1. **Accept Delays**: Accept that consistency is eventual +2. **Read Models**: Use separate read models for queries +3. **Conflict Resolution**: Define resolution strategies +4. **Monitoring**: Monitor consistency lag + +### Optimistic Locking + +1. **Version Fields**: Add version fields to entities +2. **Retry Logic**: Implement retry with backoff +3. **Conflict Handling**: Handle conflicts gracefully +4. **Performance**: Better than pessimistic locking for read-heavy workloads + +## Testing Patterns + +### Testing Sagas + +```typescript +// src/__tests__/saga/order-processing.saga.test.ts +describe('OrderProcessingSaga', () => { + it('should complete saga successfully', async () => { + const saga = new OrderProcessingSaga(); + await saga.processOrder(orderData); + + expect(orderService.create).toHaveBeenCalled(); + expect(inventoryService.reserve).toHaveBeenCalled(); + expect(paymentService.charge).toHaveBeenCalled(); + }); + + it('should compensate on payment failure', async () => { + paymentService.charge.mockRejectedValue(new Error('Payment failed')); + + await expect(saga.processOrder(orderData)).rejects.toThrow(); + + expect(orderService.cancel).toHaveBeenCalled(); + expect(inventoryService.release).toHaveBeenCalled(); + expect(paymentService.refund).not.toHaveBeenCalled(); + }); +}); +``` + +## Resources + +- [Saga Pattern](https://microservices.io/patterns/data/saga.html) - Saga pattern overview +- [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html) - Event sourcing pattern +- [CQRS Pattern](https://martinfowler.com/bliki/CQRS.html) - CQRS pattern +- [Event-Driven Architecture](../event-driven-architecture/SKILL.md) - Event patterns +- [Error Handling Patterns](../error-handling-patterns/SKILL.md) - Error handling +- [Database & Prisma](../database-prisma/SKILL.md) - Database patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/event-driven-architecture/SKILL.md b/.cursor/skills/event-driven-architecture/SKILL.md new file mode 100644 index 00000000..58972cc8 --- /dev/null +++ b/.cursor/skills/event-driven-architecture/SKILL.md @@ -0,0 +1,1592 @@ +--- +name: event-driven-architecture +description: Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. +--- + +# Event-Driven Architecture Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing asynchronous communication between services +- Decoupling services for better scalability +- Publishing domain events for downstream consumers +- Consuming events from other services +- Implementing event sourcing patterns +- Implementing CQRS (Command Query Responsibility Segregation) +- Exposing event streams via HTTP (SSE/WebSocket) +- Handling eventual consistency across services +- Building reactive systems that respond to changes +- Integrating with Apache Kafka message broker + +## Core Concepts + +### Event-Driven vs Request-Response + +**Request-Response (Synchronous):** +- Client waits for response +- Tight coupling between services +- Blocking operations +- Immediate consistency +- Use Traefik API Gateway for HTTP/REST + +**Event-Driven (Asynchronous):** +- Fire-and-forget publishing +- Loose coupling between services +- Non-blocking operations +- Eventual consistency +- Use Kafka for message broker + +### Kafka Fundamentals + +**Topics**: Named streams of events (e.g., `user.created`, `order.placed`) +- Organized by domain and action +- Divided into partitions for parallelism + +**Partitions**: Physical division of topics +- Enables horizontal scaling +- Maintains ordering per partition key +- Multiple consumers can process different partitions + +**Consumer Groups**: Group of consumers working together +- Each partition consumed by only one consumer in group +- Enables parallel processing +- Automatically rebalances on consumer join/leave + +**Producers**: Services that publish events to topics + +**Consumers**: Services that subscribe to topics and process events + +### Event Schema Versioning + +Events evolve over time. Use Schema Registry (Avro) for: +- Schema validation +- Backward/forward compatibility +- Version management +- Type safety + +### Traefik Integration for Event Streaming + +Traefik serves dual purpose: +- **API Gateway**: Routes synchronous HTTP/REST requests +- **Event Streaming Gateway**: Routes SSE/WebSocket connections to event streaming endpoints + +Services publish events to Kafka, then expose SSE/WebSocket endpoints that consume from Kafka for HTTP clients. + +## Kafka Setup & Configuration + +### Infrastructure Setup + +**Docker Compose (Local Development):** + +```yaml +# deployments/local/docker-compose.yml +services: + zookeeper: + image: confluentinc/cp-zookeeper:7.4.0 + environment: + ZOOKEEPER_CLIENT_PORT: 2181 + ZOOKEEPER_TICK_TIME: 2000 + networks: + - microservices-network + + kafka: + image: confluentinc/cp-kafka:7.4.0 + depends_on: + - zookeeper + ports: + - "9092:9092" + environment: + KAFKA_BROKER_ID: 1 + KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true' + networks: + - microservices-network + + schema-registry: + image: confluentinc/cp-schema-registry:7.4.0 + depends_on: + - kafka + ports: + - "8081:8081" + environment: + SCHEMA_REGISTRY_HOST_NAME: schema-registry + SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:9092 + networks: + - microservices-network +``` + +**Kubernetes (Production):** + +Use Confluent Operator or Strimzi operator for Kafka cluster management. + +### KafkaJS Client Configuration + +```typescript +// src/config/kafka.config.ts +// EN: Kafka client configuration +// VI: Cấu hình Kafka client +import { Kafka, logLevel } from 'kafkajs'; +import { logger } from '@goodgo/logger'; + +const kafkaConfig = { + brokers: (process.env.KAFKA_BROKERS || 'localhost:9092').split(','), + clientId: process.env.KAFKA_CLIENT_ID || process.env.SERVICE_NAME || 'unknown-service', + retry: { + initialRetryTime: 100, + retries: 8, + multiplier: 2, + maxRetryTime: 30000, + }, + logLevel: process.env.KAFKA_LOG_LEVEL === 'debug' ? logLevel.DEBUG : logLevel.INFO, + logCreator: () => ({ level, log }) => { + const { message, ...extra } = log; + if (level >= logLevel.ERROR) { + logger.error('Kafka Error', { message, ...extra }); + } else if (level >= logLevel.WARN) { + logger.warn('Kafka Warning', { message, ...extra }); + } else { + logger.debug('Kafka Log', { message, ...extra }); + } + }, +}; + +export const kafka = new Kafka(kafkaConfig); + +// EN: Producer instance (reuse across service) +// VI: Instance producer (tái sử dụng trong service) +export const producer = kafka.producer({ + idempotent: true, // EN: Ensure exactly-once semantics / VI: Đảm bảo exactly-once semantics + maxInFlightRequests: 1, // EN: Required for idempotency / VI: Cần thiết cho idempotency + transactionTimeout: 30000, +}); + +// EN: Admin client for topic management +// VI: Admin client để quản lý topics +export const admin = kafka.admin(); +``` + +### Environment Variables + +```bash +# Kafka Configuration +KAFKA_BROKERS=localhost:9092 +KAFKA_CLIENT_ID=my-service +KAFKA_LOG_LEVEL=info + +# Schema Registry +SCHEMA_REGISTRY_URL=http://localhost:8081 + +# Consumer Configuration +KAFKA_CONSUMER_GROUP_ID=my-service-consumers +KAFKA_CONSUMER_SESSION_TIMEOUT=30000 +KAFKA_CONSUMER_HEARTBEAT_INTERVAL=3000 +``` + +### Connection Management + +```typescript +// src/core/kafka/client.ts +// EN: Initialize Kafka connections +// VI: Khởi tạo kết nối Kafka +import { producer, admin } from '../config/kafka.config'; +import { logger } from '@goodgo/logger'; + +let isConnected = false; + +export async function connectKafka(): Promise { + if (isConnected) { + logger.warn('Kafka already connected'); + return; + } + + try { + await producer.connect(); + await admin.connect(); + isConnected = true; + logger.info('Kafka connected successfully'); + } catch (error) { + logger.error('Failed to connect to Kafka', { error }); + throw error; + } +} + +export async function disconnectKafka(): Promise { + if (!isConnected) { + return; + } + + try { + await producer.disconnect(); + await admin.disconnect(); + isConnected = false; + logger.info('Kafka disconnected'); + } catch (error) { + logger.error('Error disconnecting from Kafka', { error }); + throw error; + } +} + +// EN: Graceful shutdown handler +// VI: Xử lý graceful shutdown +process.on('SIGTERM', async () => { + await disconnectKafka(); + process.exit(0); +}); +``` + +## Event Publishing Patterns + +### Event Structure + +```typescript +// src/types/event.types.ts +// EN: Base event interface +// VI: Interface event cơ bản +export interface BaseEvent { + eventId: string; // EN: Unique event identifier / VI: Định danh event duy nhất + eventType: string; // EN: Event type (e.g., "user.created") / VI: Loại event + eventVersion: string; // EN: Schema version (e.g., "1.0.0") / VI: Phiên bản schema + timestamp: string; // EN: ISO 8601 timestamp / VI: Timestamp ISO 8601 + source: string; // EN: Service that published the event / VI: Service phát hành event + correlationId?: string; // EN: Request correlation ID / VI: Correlation ID của request + traceId?: string; // EN: Distributed tracing ID / VI: ID phân tán tracing + data: unknown; // EN: Event payload / VI: Payload của event +} + +// EN: Example domain event +// VI: Ví dụ domain event +export interface UserCreatedEvent extends BaseEvent { + eventType: 'user.created'; + eventVersion: '1.0.0'; + data: { + userId: string; + email: string; + name: string; + createdAt: string; + }; +} +``` + +### Event Publisher Service + +```typescript +// src/core/events/event-publisher.ts +// EN: Event publisher service +// VI: Service phát hành events +import { producer } from '../config/kafka.config'; +import { logger } from '@goodgo/logger'; +import { v4 as uuidv4 } from 'uuid'; +import type { BaseEvent } from '../../types/event.types'; + +export class EventPublisher { + private serviceName: string; + + constructor(serviceName: string = process.env.SERVICE_NAME || 'unknown') { + this.serviceName = serviceName; + } + + /** + * EN: Publish single event to Kafka topic + * VI: Phát hành một event tới Kafka topic + */ + async publish( + topic: string, + event: Omit, + options?: { + partitionKey?: string; // EN: Partition key for ordering / VI: Partition key để đảm bảo thứ tự + headers?: Record; // EN: Additional headers / VI: Headers bổ sung + } + ): Promise { + const fullEvent: T = { + ...event, + eventId: uuidv4(), + timestamp: new Date().toISOString(), + source: this.serviceName, + } as T; + + try { + await producer.send({ + topic, + messages: [ + { + key: options?.partitionKey || fullEvent.eventId, + value: JSON.stringify(fullEvent), + headers: { + 'event-type': event.eventType, + 'event-version': event.eventVersion, + 'correlation-id': event.correlationId || '', + ...options?.headers, + }, + }, + ], + }); + + logger.info('Event published', { + topic, + eventType: event.eventType, + eventId: fullEvent.eventId, + correlationId: event.correlationId, + }); + } catch (error) { + logger.error('Failed to publish event', { + topic, + eventType: event.eventType, + error, + }); + throw error; + } + } + + /** + * EN: Publish multiple events in batch + * VI: Phát hành nhiều events cùng lúc + */ + async publishBatch( + events: Array<{ + topic: string; + event: Omit; + partitionKey?: string; + }> + ): Promise { + const messages = events.map(({ topic, event, partitionKey }) => { + const fullEvent: T = { + ...event, + eventId: uuidv4(), + timestamp: new Date().toISOString(), + source: this.serviceName, + } as T; + + return { + topic, + messages: [ + { + key: partitionKey || fullEvent.eventId, + value: JSON.stringify(fullEvent), + headers: { + 'event-type': event.eventType, + 'event-version': event.eventVersion, + 'correlation-id': event.correlationId || '', + }, + }, + ], + }; + }); + + try { + await producer.sendBatch({ + topicMessages: messages, + }); + + logger.info('Events published in batch', { + count: events.length, + }); + } catch (error) { + logger.error('Failed to publish events batch', { error }); + throw error; + } + } +} + +export const eventPublisher = new EventPublisher(); +``` + +### Publishing Events from Services + +```typescript +// src/modules/user/user.service.ts +// EN: Example: Publishing event after user creation +// VI: Ví dụ: Phát hành event sau khi tạo user +import { eventPublisher } from '../../core/events/event-publisher'; +import { logger } from '@goodgo/logger'; +import type { UserCreatedEvent } from '../../types/event.types'; + +export class UserService { + async createUser(data: CreateUserDto, correlationId?: string): Promise { + // EN: Create user in database + // VI: Tạo user trong database + const user = await this.userRepository.create(data); + + // EN: Publish event after successful creation + // VI: Phát hành event sau khi tạo thành công + try { + await eventPublisher.publish( + 'user.created', + { + eventType: 'user.created', + eventVersion: '1.0.0', + correlationId, + data: { + userId: user.id, + email: user.email, + name: user.name, + createdAt: user.createdAt.toISOString(), + }, + }, + { + partitionKey: user.id, // EN: Ensure events for same user are ordered / VI: Đảm bảo events của cùng user có thứ tự + } + ); + } catch (error) { + // EN: Log error but don't fail user creation + // VI: Ghi log lỗi nhưng không fail việc tạo user + logger.error('Failed to publish user.created event', { error, userId: user.id }); + } + + return user; + } +} +``` + +### Transactional Publishing (Outbox Pattern) + +```typescript +// src/core/events/outbox-pattern.ts +// EN: Outbox pattern for transactional event publishing +// VI: Outbox pattern để phát hành events trong transaction +import { PrismaClient } from '@prisma/client'; +import { EventPublisher } from './event-publisher'; +import { logger } from '@goodgo/logger'; + +export class OutboxService { + constructor(private prisma: PrismaClient) {} + + /** + * EN: Store event in outbox table within transaction + * VI: Lưu event vào outbox table trong transaction + */ + async addToOutbox( + eventType: string, + eventData: unknown, + topic: string, + correlationId?: string + ): Promise { + await this.prisma.outboxEvent.create({ + data: { + eventType, + eventData: eventData as any, + topic, + correlationId, + status: 'PENDING', + }, + }); + } + + /** + * EN: Process outbox events and publish to Kafka + * VI: Xử lý outbox events và phát hành tới Kafka + */ + async processOutbox(eventPublisher: EventPublisher): Promise { + const events = await this.prisma.outboxEvent.findMany({ + where: { status: 'PENDING' }, + take: 100, + orderBy: { createdAt: 'asc' }, + }); + + for (const event of events) { + try { + await eventPublisher.publish( + event.topic, + { + eventType: event.eventType, + eventVersion: '1.0.0', + correlationId: event.correlationId || undefined, + data: event.eventData, + }, + { + partitionKey: event.id, + } + ); + + await this.prisma.outboxEvent.update({ + where: { id: event.id }, + data: { status: 'PUBLISHED', publishedAt: new Date() }, + }); + } catch (error) { + logger.error('Failed to process outbox event', { eventId: event.id, error }); + // EN: Mark as failed, will retry later + // VI: Đánh dấu failed, sẽ retry sau + await this.prisma.outboxEvent.update({ + where: { id: event.id }, + data: { status: 'FAILED', retryCount: { increment: 1 } }, + }); + } + } + } +} + +// EN: Example: Using outbox in transaction +// VI: Ví dụ: Sử dụng outbox trong transaction +export class UserService { + constructor( + private prisma: PrismaClient, + private outboxService: OutboxService + ) {} + + async createUserWithOutbox(data: CreateUserDto): Promise { + return await this.prisma.$transaction(async (tx) => { + // EN: Create user + // VI: Tạo user + const user = await tx.user.create({ data }); + + // EN: Add event to outbox (within same transaction) + // VI: Thêm event vào outbox (trong cùng transaction) + await this.outboxService.addToOutbox( + 'user.created', + { + userId: user.id, + email: user.email, + name: user.name, + }, + 'user.created' + ); + + return user; + }); + } +} +``` + +### Event Naming Conventions + +**Event Type Format**: `{domain}.{action}.{version}` + +Examples: +- `user.created.v1` +- `user.updated.v1` +- `order.placed.v1` +- `payment.processed.v2` + +**Topic Naming**: `{domain}.{entity}.{action}` + +Examples: +- `user.created` +- `user.updated` +- `order.placed` +- `payment.processed` + +## Event Consuming Patterns + +### Event Consumer Service + +```typescript +// src/core/events/event-consumer.ts +// EN: Event consumer service +// VI: Service tiêu thụ events +import { kafka } from '../config/kafka.config'; +import { logger } from '@goodgo/logger'; +import type { BaseEvent } from '../../types/event.types'; + +export interface EventHandler { + handle(event: T): Promise; +} + +export class EventConsumer { + private consumer: ReturnType; + private handlers: Map = new Map(); + + constructor( + groupId: string = process.env.KAFKA_CONSUMER_GROUP_ID || 'default-group' + ) { + this.consumer = kafka.consumer({ + groupId, + sessionTimeout: 30000, + heartbeatInterval: 3000, + }); + } + + /** + * EN: Register event handler for specific event type + * VI: Đăng ký event handler cho loại event cụ thể + */ + on(eventType: string, handler: EventHandler): void { + if (!this.handlers.has(eventType)) { + this.handlers.set(eventType, []); + } + this.handlers.get(eventType)!.push(handler as EventHandler); + } + + /** + * EN: Start consuming from topics + * VI: Bắt đầu tiêu thụ từ topics + */ + async start(topics: string[]): Promise { + await this.consumer.connect(); + await this.consumer.subscribe({ topics, fromBeginning: false }); + + await this.consumer.run({ + eachMessage: async ({ topic, partition, message }) => { + try { + const event: BaseEvent = JSON.parse(message.value?.toString() || '{}'); + + // EN: Get handlers for this event type + // VI: Lấy handlers cho loại event này + const handlers = this.handlers.get(event.eventType) || []; + + if (handlers.length === 0) { + logger.warn('No handler registered for event type', { + eventType: event.eventType, + topic, + }); + return; + } + + // EN: Execute all handlers + // VI: Thực thi tất cả handlers + await Promise.all(handlers.map((handler) => handler.handle(event))); + + logger.debug('Event processed successfully', { + eventType: event.eventType, + eventId: event.eventId, + topic, + partition, + }); + } catch (error) { + logger.error('Error processing event', { + topic, + partition, + offset: message.offset, + error, + }); + // EN: Event will be retried by consumer + // VI: Event sẽ được retry bởi consumer + throw error; + } + }, + }); + + logger.info('Event consumer started', { topics, groupId: this.consumer.groupId }); + } + + /** + * EN: Stop consuming + * VI: Dừng tiêu thụ + */ + async stop(): Promise { + await this.consumer.disconnect(); + logger.info('Event consumer stopped'); + } +} +``` + +### Consumer Implementation + +```typescript +// src/modules/notification/notification.consumer.ts +// EN: Example: Consuming user.created events +// VI: Ví dụ: Tiêu thụ user.created events +import { EventConsumer, EventHandler } from '../../core/events/event-consumer'; +import { logger } from '@goodgo/logger'; +import type { UserCreatedEvent } from '../../types/event.types'; + +class UserCreatedHandler implements EventHandler { + async handle(event: UserCreatedEvent): Promise { + logger.info('Processing user.created event', { eventId: event.eventId, userId: event.data.userId }); + + // EN: Send welcome email + // VI: Gửi email chào mừng + await this.sendWelcomeEmail(event.data.email, event.data.name); + + // EN: Create notification preference + // VI: Tạo notification preference + await this.createNotificationPreferences(event.data.userId); + } + + private async sendWelcomeEmail(email: string, name: string): Promise { + // EN: Email sending logic + // VI: Logic gửi email + } + + private async createNotificationPreferences(userId: string): Promise { + // EN: Create default preferences + // VI: Tạo preferences mặc định + } +} + +// EN: Initialize consumer +// VI: Khởi tạo consumer +export const notificationConsumer = new EventConsumer('notification-service'); + +// EN: Register handlers +// VI: Đăng ký handlers +notificationConsumer.on('user.created', new UserCreatedHandler()); + +// EN: Start consuming +// VI: Bắt đầu tiêu thụ +await notificationConsumer.start(['user.created']); +``` + +### Dead Letter Queue (DLQ) Pattern + +```typescript +// src/core/events/dlq-handler.ts +// EN: Dead Letter Queue handler +// VI: Handler cho Dead Letter Queue +import { eventPublisher } from './event-publisher'; +import { logger } from '@goodgo/logger'; + +export class DLQHandler { + /** + * EN: Send failed event to DLQ topic + * VI: Gửi event failed tới DLQ topic + */ + async sendToDLQ( + originalEvent: BaseEvent, + originalTopic: string, + error: Error, + retryCount: number + ): Promise { + const dlqEvent = { + ...originalEvent, + eventType: `${originalEvent.eventType}.dlq`, + data: { + originalEvent, + originalTopic, + error: { + message: error.message, + stack: error.stack, + }, + retryCount, + dlqTimestamp: new Date().toISOString(), + }, + }; + + await eventPublisher.publish(`${originalTopic}.dlq`, dlqEvent); + + logger.error('Event sent to DLQ', { + originalEventType: originalEvent.eventType, + originalTopic, + retryCount, + error: error.message, + }); + } +} + +// EN: Consumer with DLQ support +// VI: Consumer với hỗ trợ DLQ +export class EventConsumerWithDLQ extends EventConsumer { + private dlqHandler = new DLQHandler(); + private maxRetries = 3; + + async start(topics: string[]): Promise { + await this.consumer.connect(); + await this.consumer.subscribe({ topics, fromBeginning: false }); + + await this.consumer.run({ + eachMessage: async ({ topic, partition, message }) => { + let retryCount = 0; + const maxAttempts = this.maxRetries + 1; + + while (retryCount < maxAttempts) { + try { + const event: BaseEvent = JSON.parse(message.value?.toString() || '{}'); + const handlers = this.handlers.get(event.eventType) || []; + + if (handlers.length === 0) { + logger.warn('No handler registered for event type', { + eventType: event.eventType, + topic, + }); + return; + } + + await Promise.all(handlers.map((handler) => handler.handle(event))); + return; // EN: Success, exit retry loop / VI: Thành công, thoát khỏi vòng lặp retry + } catch (error) { + retryCount++; + if (retryCount >= maxAttempts) { + // EN: Max retries exceeded, send to DLQ + // VI: Vượt quá số lần retry, gửi tới DLQ + const event: BaseEvent = JSON.parse(message.value?.toString() || '{}'); + await this.dlqHandler.sendToDLQ(event, topic, error as Error, retryCount); + return; + } + + // EN: Wait before retry with exponential backoff + // VI: Đợi trước khi retry với exponential backoff + const delay = Math.min(1000 * Math.pow(2, retryCount), 30000); + await new Promise((resolve) => setTimeout(resolve, delay)); + + logger.warn('Retrying event processing', { + topic, + partition, + offset: message.offset, + retryCount, + delay, + }); + } + } + }, + }); + } +} +``` + +### Processing Strategies + +**At-Least-Once Delivery:** +- Default Kafka behavior +- Messages may be delivered multiple times +- Consumers must be idempotent +- Use idempotency keys in event data + +**Exactly-Once Semantics:** +- Use transactional producer +- Consumer processes with idempotency checks +- More complex, higher latency + +## Schema Versioning + +### Schema Registry Setup + +```typescript +// src/core/schema/schema-registry.ts +// EN: Schema Registry client +// VI: Client Schema Registry +import { SchemaRegistry } from '@kafkajs/confluent-schema-registry'; +import { logger } from '@goodgo/logger'; + +const registry = new SchemaRegistry({ + host: process.env.SCHEMA_REGISTRY_URL || 'http://localhost:8081', +}); + +export async function registerSchema( + subject: string, + schema: object +): Promise { + try { + const { id } = await registry.register({ type: 'AVRO', schema: JSON.stringify(schema) }); + logger.info('Schema registered', { subject, id }); + return id; + } catch (error) { + logger.error('Failed to register schema', { subject, error }); + throw error; + } +} + +export async function getSchema(subject: string, version?: number) { + try { + const schema = await registry.getSchema(subject, version); + return schema; + } catch (error) { + logger.error('Failed to get schema', { subject, version, error }); + throw error; + } +} +``` + +### Avro Schema Definition + +```json +// schemas/user.created.v1.avsc +{ + "type": "record", + "name": "UserCreated", + "namespace": "com.goodgo.events.user", + "fields": [ + { + "name": "userId", + "type": "string" + }, + { + "name": "email", + "type": "string" + }, + { + "name": "name", + "type": "string" + }, + { + "name": "createdAt", + "type": "string" + } + ] +} +``` + +### Schema Evolution + +**Backward Compatible Changes (Safe):** +- Adding optional fields +- Removing fields (old consumers ignore) + +**Forward Compatible Changes:** +- Removing optional fields +- Adding required fields (with defaults) + +**Breaking Changes:** +- Change field type +- Remove required field +- Rename field + +**Versioning Strategy:** +- Increment version for breaking changes: `user.created.v2` +- Use same topic with versioned event type +- Consumers handle multiple versions + +## Traefik Integration (SSE/WebSocket) + +### SSE Endpoint Implementation + +```typescript +// src/modules/events/events.controller.ts +// EN: SSE endpoint for event streaming +// VI: SSE endpoint để stream events +import { Request, Response } from 'express'; +import { kafka } from '../../config/kafka.config'; +import { logger } from '@goodgo/logger'; + +export class EventsController { + /** + * EN: Server-Sent Events endpoint + * VI: Endpoint Server-Sent Events + */ + async streamEvents(req: Request, res: Response): Promise { + // EN: Set SSE headers + // VI: Thiết lập SSE headers + res.setHeader('Content-Type', 'text/event-stream'); + res.setHeader('Cache-Control', 'no-cache'); + res.setHeader('Connection', 'keep-alive'); + res.setHeader('X-Accel-Buffering', 'no'); // EN: Disable nginx buffering / VI: Tắt nginx buffering + + const topic = req.query.topic as string; + const eventType = req.query.eventType as string | undefined; + + if (!topic) { + res.status(400).json({ success: false, error: { code: 'TOPIC_REQUIRED', message: 'Topic parameter required' } }); + return; + } + + // EN: Create consumer for this connection + // VI: Tạo consumer cho connection này + const consumer = kafka.consumer({ + groupId: `sse-${Date.now()}`, // EN: Unique group per connection / VI: Group duy nhất cho mỗi connection + }); + + try { + await consumer.connect(); + await consumer.subscribe({ topic, fromBeginning: false }); + + // EN: Send initial connection message + // VI: Gửi message kết nối ban đầu + res.write(`data: ${JSON.stringify({ type: 'connected', topic })}\n\n`); + + await consumer.run({ + eachMessage: async ({ message }) => { + try { + const event = JSON.parse(message.value?.toString() || '{}'); + + // EN: Filter by event type if specified + // VI: Lọc theo event type nếu được chỉ định + if (eventType && event.eventType !== eventType) { + return; + } + + // EN: Send event to client + // VI: Gửi event tới client + res.write(`data: ${JSON.stringify(event)}\n\n`); + } catch (error) { + logger.error('Error processing SSE message', { error }); + } + }, + }); + + // EN: Handle client disconnect + // VI: Xử lý khi client ngắt kết nối + req.on('close', async () => { + await consumer.disconnect(); + logger.info('SSE connection closed', { topic }); + }); + } catch (error) { + logger.error('SSE connection error', { error }); + res.write(`data: ${JSON.stringify({ type: 'error', message: 'Connection failed' })}\n\n`); + await consumer.disconnect(); + } + } +} +``` + +### Traefik Routing Configuration + +```yaml +# infra/traefik/dynamic/routes.yml +http: + routers: + events-sse: + rule: "PathPrefix(`/api/v1/events/stream`)" + entrypoints: + - web + service: events-service + middlewares: + - cors + - compress + - events-ratelimit + + services: + events-service: + loadBalancer: + servers: + - url: "http://events-service:5000" + + middlewares: + events-ratelimit: + rateLimit: + average: 100 + period: 1m + burst: 50 +``` + +### Route Registration + +```typescript +// src/routes/events.routes.ts +// EN: Event streaming routes +// VI: Routes cho event streaming +import { Router } from 'express'; +import { EventsController } from '../modules/events/events.controller'; +import { authenticate } from '../middlewares/auth.middleware'; + +const router = Router(); +const eventsController = new EventsController(); + +// EN: SSE endpoint (requires authentication) +// VI: SSE endpoint (yêu cầu authentication) +router.get('/stream', authenticate(), (req, res) => { + eventsController.streamEvents(req, res); +}); + +export default router; +``` + +### Client Subscription Example + +```typescript +// EN: Client-side SSE subscription +// VI: Client-side subscription SSE +const eventSource = new EventSource('/api/v1/events/stream?topic=user.created&eventType=user.created'); + +eventSource.onmessage = (event) => { + const data = JSON.parse(event.data); + console.log('Received event:', data); +}; + +eventSource.onerror = (error) => { + console.error('SSE error:', error); + eventSource.close(); +}; +``` + +## Error Handling & Resilience + +### Event Publishing Errors + +```typescript +// EN: Retry logic for publishing failures +// VI: Logic retry cho lỗi phát hành +import { retryWithBackoff } from '../resilience/retry'; + +async function publishWithRetry( + topic: string, + event: T, + maxRetries: number = 3 +): Promise { + try { + await retryWithBackoff( + async () => { + await eventPublisher.publish(topic, event); + }, + maxRetries, + 1000 // EN: Base delay 1s / VI: Delay cơ bản 1s + ); + } catch (error) { + // EN: After max retries, store in DLQ or persistent storage + // VI: Sau khi retry tối đa, lưu vào DLQ hoặc persistent storage + logger.error('Failed to publish event after retries', { + topic, + eventType: event.eventType, + eventId: event.eventId, + error, + }); + throw error; + } +} +``` + +### Circuit Breaker for Kafka Operations + +```typescript +// EN: Circuit breaker for Kafka producer +// VI: Circuit breaker cho Kafka producer +import { createCircuitBreaker } from '../resilience/circuit-breaker'; + +const kafkaCircuitBreaker = createCircuitBreaker( + async (topic: string, event: BaseEvent) => { + await eventPublisher.publish(topic, event); + }, + 'kafka-producer', + { + timeout: 5000, + errorThresholdPercentage: 50, + resetTimeout: 30000, + } +); + +// EN: Usage with circuit breaker +// VI: Sử dụng với circuit breaker +try { + await kafkaCircuitBreaker.fire('user.created', event); +} catch (error) { + if (kafkaCircuitBreaker.opened) { + // EN: Circuit is open, use fallback (e.g., store in DB for later) + // VI: Circuit đang mở, sử dụng fallback (ví dụ: lưu vào DB để xử lý sau) + await storeEventForLaterProcessing(event); + } + throw error; +} +``` + +### Event Replay + +```typescript +// src/core/events/event-replay.ts +// EN: Event replay service +// VI: Service replay events +export class EventReplayService { + /** + * EN: Replay events from specific offset + * VI: Replay events từ offset cụ thể + */ + async replayEvents( + topic: string, + fromOffset: string, + toOffset?: string, + eventType?: string + ): Promise { + const consumer = kafka.consumer({ + groupId: `replay-${Date.now()}`, + }); + + await consumer.connect(); + await consumer.subscribe({ topic, fromBeginning: false }); + + // EN: Seek to specific offset + // VI: Tìm tới offset cụ thể + // Implementation depends on partition selection + + logger.info('Event replay started', { topic, fromOffset, toOffset }); + } +} +``` + +## Observability + +### Event Logging + +```typescript +// EN: Structured logging for events +// VI: Structured logging cho events +import { logger } from '@goodgo/logger'; + +// EN: Log when publishing +// VI: Log khi phát hành +logger.info('Event published', { + eventType: event.eventType, + eventId: event.eventId, + topic, + correlationId: event.correlationId, + traceId: event.traceId, + source: event.source, +}); + +// EN: Log when consuming +// VI: Log khi tiêu thụ +logger.info('Event consumed', { + eventType: event.eventType, + eventId: event.eventId, + topic, + partition, + offset, + processingTime: Date.now() - eventTimestamp, +}); +``` + +### Metrics Collection + +```typescript +// src/core/events/event-metrics.ts +// EN: Event metrics +// VI: Metrics cho events +import { Counter, Histogram, Gauge } from 'prom-client'; + +const eventsPublished = new Counter({ + name: 'events_published_total', + help: 'Total number of events published', + labelNames: ['topic', 'event_type'], +}); + +const eventsConsumed = new Counter({ + name: 'events_consumed_total', + help: 'Total number of events consumed', + labelNames: ['topic', 'event_type'], +}); + +const eventProcessingDuration = new Histogram({ + name: 'event_processing_duration_seconds', + help: 'Event processing duration in seconds', + labelNames: ['topic', 'event_type'], +}); + +const consumerLag = new Gauge({ + name: 'kafka_consumer_lag', + help: 'Consumer lag per topic and partition', + labelNames: ['topic', 'partition'], +}); + +// EN: Usage in publisher +// VI: Sử dụng trong publisher +eventsPublished.inc({ topic, event_type: event.eventType }); + +// EN: Usage in consumer +// VI: Sử dụng trong consumer +const timer = eventProcessingDuration.startTimer({ topic, event_type: event.eventType }); +try { + await handler.handle(event); + eventsConsumed.inc({ topic, event_type: event.eventType }); +} finally { + timer(); +} +``` + +### Distributed Tracing + +```typescript +// EN: Add tracing to events +// VI: Thêm tracing vào events +import { trace, context, SpanStatusCode } from '@opentelemetry/api'; + +const tracer = trace.getTracer('event-service'); + +// EN: Create span for event publishing +// VI: Tạo span cho việc phát hành event +const span = tracer.startSpan('event.publish', { + attributes: { + 'event.type': event.eventType, + 'event.topic': topic, + 'event.id': event.eventId, + }, +}); + +try { + await eventPublisher.publish(topic, event); + span.setStatus({ code: SpanStatusCode.OK }); +} catch (error) { + span.setStatus({ code: SpanStatusCode.ERROR, message: error.message }); + throw error; +} finally { + span.end(); +} +``` + +### Health Checks + +```typescript +// src/routes/health.routes.ts +// EN: Health check for Kafka connectivity +// VI: Health check cho kết nối Kafka +import { admin } from '../config/kafka.config'; + +router.get('/health/kafka', async (req, res) => { + try { + await admin.listTopics(); + res.json({ status: 'healthy', kafka: 'connected' }); + } catch (error) { + res.status(503).json({ status: 'unhealthy', kafka: 'disconnected', error: error.message }); + } +}); +``` + +## Testing Patterns + +### Unit Testing Publishers + +```typescript +// src/core/events/__tests__/event-publisher.test.ts +// EN: Unit tests for event publisher +// VI: Unit tests cho event publisher +import { EventPublisher } from '../event-publisher'; +import { producer } from '../../config/kafka.config'; + +jest.mock('../../config/kafka.config'); + +describe('EventPublisher', () => { + let publisher: EventPublisher; + const mockSend = jest.fn(); + + beforeEach(() => { + publisher = new EventPublisher('test-service'); + (producer.send as jest.Mock) = mockSend; + mockSend.mockResolvedValue({}); + }); + + it('should publish event successfully', async () => { + const event = { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { userId: '123' }, + }; + + await publisher.publish('user.created', event); + + expect(mockSend).toHaveBeenCalledWith({ + topic: 'user.created', + messages: expect.arrayContaining([ + expect.objectContaining({ + key: expect.any(String), + value: expect.stringContaining('"eventType":"user.created"'), + }), + ]), + }); + }); +}); +``` + +### Integration Testing with Test Containers + +```typescript +// src/__tests__/integration/event-flow.e2e.ts +// EN: E2E test with Kafka test containers +// VI: E2E test với Kafka test containers +import { KafkaContainer, StartedKafkaContainer } from '@testcontainers/kafka'; +import { EventPublisher } from '../../core/events/event-publisher'; +import { EventConsumer } from '../../core/events/event-consumer'; +import type { BaseEvent } from '../../types/event.types'; + +describe('Event Flow E2E', () => { + let kafkaContainer: StartedKafkaContainer; + let publisher: EventPublisher; + let consumer: EventConsumer; + + beforeAll(async () => { + kafkaContainer = await new KafkaContainer().start(); + process.env.KAFKA_BROKERS = kafkaContainer.getBootstrapServer(); + + publisher = new EventPublisher('test-service'); + consumer = new EventConsumer('test-group'); + }); + + afterAll(async () => { + await kafkaContainer.stop(); + }); + + it('should publish and consume event', async () => { + const receivedEvents: BaseEvent[] = []; + + consumer.on('user.created', { + handle: async (event) => { + receivedEvents.push(event); + }, + }); + + await consumer.start(['user.created']); + + const event = { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { userId: '123', email: 'test@example.com' }, + }; + + await publisher.publish('user.created', event); + + // EN: Wait for event processing + // VI: Đợi xử lý event + await new Promise((resolve) => setTimeout(resolve, 1000)); + + expect(receivedEvents).toHaveLength(1); + expect(receivedEvents[0].data.userId).toBe('123'); + }); +}); +``` + +### Mocking Kafka + +```typescript +// src/__tests__/mocks/kafka.mock.ts +// EN: Kafka mocks for testing +// VI: Kafka mocks cho testing +export const createMockKafka = () => { + const messages: any[] = []; + + return { + producer: () => ({ + connect: jest.fn(), + send: jest.fn((params) => { + messages.push(...params.messages); + return Promise.resolve({}); + }), + disconnect: jest.fn(), + }), + consumer: () => ({ + connect: jest.fn(), + subscribe: jest.fn(), + run: jest.fn(), + disconnect: jest.fn(), + }), + admin: () => ({ + connect: jest.fn(), + listTopics: jest.fn().mockResolvedValue([]), + disconnect: jest.fn(), + }), + messages, // EN: Access sent messages / VI: Truy cập messages đã gửi + }; +}; +``` + +## Best Practices + +### Event Naming Conventions + +- **Event Type**: `{domain}.{action}.v{version}` (e.g., `user.created.v1`) +- **Topic**: `{domain}.{entity}.{action}` (e.g., `user.created`) +- Use lowercase with dots as separators +- Keep names descriptive and consistent + +### Partition Key Selection + +- Use entity ID for ordering guarantees (same entity → same partition) +- Use correlation ID for request tracing +- Use user ID for user-scoped events +- Avoid high-cardinality keys (distributes evenly) + +### Event Ordering Guarantees + +- Kafka guarantees ordering **per partition** +- Use partition key to ensure related events go to same partition +- Events in different partitions have no ordering guarantee +- Don't rely on global ordering across all events + +### Event Size Limits + +- Recommended: < 1MB per event +- Kafka default: 1MB (configurable) +- For large payloads: Store data elsewhere, send reference in event +- Consider compression for large events + +### Performance Optimization + +- **Batch Publishing**: Group multiple events for better throughput +- **Async Publishing**: Don't block request handlers +- **Consumer Parallelism**: Use multiple partitions and consumers +- **Connection Pooling**: Reuse Kafka client instances +- **Compression**: Enable compression for better network usage + +### Common Anti-Patterns to Avoid + +1. **Publishing in Request Handler**: Use async/outbox pattern +2. **Blocking on Event Publishing**: Fire-and-forget with error handling +3. **No Idempotency**: Consumers must handle duplicate events +4. **Ignoring Consumer Lag**: Monitor and alert on lag +5. **Breaking Schema Changes**: Use versioning strategy +6. **Global Ordering Expectations**: Understand partition ordering only + +## Complete Examples + +### Example 1: User Created Event Flow + +```typescript +// EN: Auth Service publishes user.created event +// VI: Auth Service phát hành user.created event +// services/auth-service/src/modules/user/user.service.ts +export class UserService { + async createUser(data: CreateUserDto): Promise { + const user = await this.userRepository.create(data); + + await eventPublisher.publish( + 'user.created', + { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { + userId: user.id, + email: user.email, + name: user.name, + }, + }, + { partitionKey: user.id } + ); + + return user; + } +} + +// EN: Notification Service consumes and sends welcome email +// VI: Notification Service tiêu thụ và gửi email chào mừng +// services/notification-service/src/modules/notification/notification.consumer.ts +notificationConsumer.on('user.created', { + handle: async (event: UserCreatedEvent) => { + await emailService.sendWelcomeEmail(event.data.email, event.data.name); + await notificationPreferenceService.createDefaults(event.data.userId); + }, +}); +``` + +### Example 2: Order Placed with Multiple Consumers + +```typescript +// EN: Order Service publishes order.placed +// VI: Order Service phát hành order.placed +await eventPublisher.publish('order.placed', { + eventType: 'order.placed', + eventVersion: '1.0.0', + data: { + orderId: order.id, + userId: order.userId, + total: order.total, + items: order.items, + }, +}); + +// EN: Multiple consumers handle the event +// VI: Nhiều consumers xử lý event + +// Payment Service: Process payment +paymentConsumer.on('order.placed', { handle: processPayment }); + +// Inventory Service: Reserve items +inventoryConsumer.on('order.placed', { handle: reserveItems }); + +// Notification Service: Send confirmation +notificationConsumer.on('order.placed', { handle: sendConfirmation }); +``` + +### Example 3: SSE Endpoint with Filtering + +```typescript +// EN: Client subscribes to user events +// VI: Client subscribe tới user events +const eventSource = new EventSource( + '/api/v1/events/stream?topic=user.created&eventType=user.created', + { + headers: { + Authorization: `Bearer ${token}`, + }, + } +); + +eventSource.addEventListener('user.created', (event) => { + const data = JSON.parse(event.data); + updateUI(data); +}); +``` + +## Resources + +- [KafkaJS Documentation](https://kafka.js.org/) - Node.js Kafka client +- [Confluent Schema Registry](https://docs.confluent.io/platform/current/schema-registry/index.html) - Schema versioning +- [Kafka Best Practices](https://kafka.apache.org/documentation/#best_practices) - Official Kafka documentation +- [Resilience Patterns](../resilience-patterns/SKILL.md) - Circuit breaker, retry patterns +- [Error Handling Patterns](../error-handling-patterns/SKILL.md) - Error handling best practices +- [Observability & Monitoring](../observability-monitoring/SKILL.md) - Logging, metrics, tracing +- [Middleware Patterns](../middleware-patterns/SKILL.md) - Express middleware patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/infrastructure-as-code/SKILL.md b/.cursor/skills/infrastructure-as-code/SKILL.md new file mode 100644 index 00000000..68f18bd0 --- /dev/null +++ b/.cursor/skills/infrastructure-as-code/SKILL.md @@ -0,0 +1,318 @@ +--- +name: infrastructure-as-code +description: Infrastructure as Code patterns for GoodGo platform including Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, and multi-environment management. Use when managing infrastructure, implementing GitOps, or creating reusable infrastructure modules. +--- + +# Infrastructure as Code Patterns + +## When to Use This Skill + +Use this skill when: +- Managing infrastructure with code +- Implementing Terraform modules +- Setting up GitOps workflows +- Creating Kubernetes operators +- Testing infrastructure changes +- Managing multi-environment infrastructure +- Versioning infrastructure +- Automating infrastructure provisioning + +## Core Concepts + +### Infrastructure as Code Benefits + +1. **Version Control**: Track infrastructure changes +2. **Reproducibility**: Consistent environments +3. **Automation**: Reduce manual errors +4. **Testing**: Test infrastructure changes +5. **Collaboration**: Team can review changes + +### IaC Tools + +- **Terraform**: Infrastructure provisioning +- **Kubernetes Manifests**: K8s resource definitions +- **Helm**: K8s package manager +- **ArgoCD/Flux**: GitOps tools + +## Terraform Patterns + +### Module Structure + +``` +infra/terraform/ +├── modules/ +│ ├── kubernetes-cluster/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ └── outputs.tf +│ ├── postgresql/ +│ │ ├── main.tf +│ │ ├── variables.tf +│ │ └── outputs.tf +│ └── redis/ +│ ├── main.tf +│ ├── variables.tf +│ └── outputs.tf +├── environments/ +│ ├── staging/ +│ │ ├── main.tf +│ │ └── terraform.tfvars +│ └── production/ +│ ├── main.tf +│ └── terraform.tfvars +└── shared/ + └── backend.tf +``` + +### Terraform Module Example + +```hcl +# infra/terraform/modules/postgresql/main.tf +# EN: PostgreSQL module +# VI: Module PostgreSQL +variable "database_name" { + description = "Database name" + type = string +} + +variable "environment" { + description = "Environment name" + type = string +} + +resource "google_sql_database_instance" "postgres" { + name = "${var.database_name}-${var.environment}" + database_version = "POSTGRES_15" + region = "us-central1" + + settings { + tier = "db-f1-micro" + + backup_configuration { + enabled = true + start_time = "03:00" + } + } +} + +resource "google_sql_database" "database" { + name = var.database_name + instance = google_sql_database_instance.postgres.name +} + +output "connection_name" { + value = google_sql_database_instance.postgres.connection_name +} + +output "database_url" { + value = "postgresql://user:pass@${google_sql_database_instance.postgres.ip_address}/${var.database_name}" + sensitive = true +} +``` + +### Using Modules + +```hcl +# infra/terraform/environments/staging/main.tf +# EN: Use PostgreSQL module +# VI: Sử dụng module PostgreSQL +module "postgresql" { + source = "../../modules/postgresql" + + database_name = "goodgo_staging" + environment = "staging" +} + +output "database_url" { + value = module.postgresql.database_url + sensitive = true +} +``` + +## GitOps Patterns + +### ArgoCD Setup + +```yaml +# infra/gitops/argocd/applications/user-service.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: user-service + namespace: argocd +spec: + project: default + source: + repoURL: https://github.com/goodgo/platform + targetRevision: main + path: deployments/production/kubernetes/user-service + destination: + server: https://kubernetes.default.svc + namespace: production + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=true +``` + +### Flux Setup + +```yaml +# infra/gitops/flux/kustomizations/user-service.yaml +apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 +kind: Kustomization +metadata: + name: user-service + namespace: flux-system +spec: + interval: 5m + path: ./deployments/production/kubernetes/user-service + prune: true + sourceRef: + kind: GitRepository + name: platform-repo + validation: client +``` + +## Infrastructure Testing + +### Terraform Testing + +```bash +#!/bin/bash +# scripts/infra/test-terraform.sh +# EN: Test Terraform changes +# VI: Test các thay đổi Terraform + +cd infra/terraform/environments/staging + +# EN: Validate Terraform +# VI: Validate Terraform +terraform init +terraform validate + +# EN: Plan changes +# VI: Plan changes +terraform plan -out=tfplan + +# EN: Review plan +# VI: Review plan +terraform show tfplan +``` + +### Infrastructure Validation + +```typescript +// scripts/infra/validate-k8s.ts +// EN: Validate Kubernetes manifests +// VI: Validate Kubernetes manifests +import { execSync } from 'child_process'; + +function validateKubernetesManifests(path: string): boolean { + try { + execSync(`kubectl apply --dry-run=client -f ${path}`, { + stdio: 'inherit', + }); + console.log('Kubernetes manifests are valid'); + return true; + } catch (error) { + console.error('Kubernetes validation failed', error); + return false; + } +} +``` + +## Environment Management + +### Environment Configuration + +```hcl +# infra/terraform/environments/staging/terraform.tfvars +environment = "staging" +cluster_name = "goodgo-staging" +node_count = 3 +instance_type = "t3.medium" + +# infra/terraform/environments/production/terraform.tfvars +environment = "production" +cluster_name = "goodgo-production" +node_count = 5 +instance_type = "t3.large" +``` + +### Multi-Environment Module + +```hcl +# infra/terraform/modules/service/main.tf +variable "environment" { + type = string +} + +variable "replicas" { + type = map(number) + default = { + staging = 2 + production = 5 + } +} + +resource "kubernetes_deployment" "service" { + metadata { + name = "${var.service_name}-${var.environment}" + } + + spec { + replicas = var.replicas[var.environment] + # ... + } +} +``` + +## Kubernetes Operators + +### Custom Resource Definition + +```yaml +# infra/operators/database-operator/crd.yaml +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: databases.example.com +spec: + group: example.com + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + databaseName: + type: string + environment: + type: string +``` + +## Best Practices + +1. **Version Control**: Keep all infrastructure in version control +2. **Modules**: Create reusable Terraform modules +3. **Testing**: Test infrastructure changes before applying +4. **GitOps**: Use GitOps for Kubernetes deployments +5. **Environment Isolation**: Separate environments completely +6. **State Management**: Use remote state backends +7. **Secrets**: Never commit secrets, use secrets managers + +## Resources + +- [Terraform Documentation](https://www.terraform.io/docs) +- [ArgoCD Documentation](https://argo-cd.readthedocs.io/) +- [Flux Documentation](https://fluxcd.io/docs/) +- [Deployment Kubernetes](../deployment-kubernetes/SKILL.md) - K8s patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo standards diff --git a/.cursor/skills/inter-service-communication/SKILL.md b/.cursor/skills/inter-service-communication/SKILL.md new file mode 100644 index 00000000..25879a22 --- /dev/null +++ b/.cursor/skills/inter-service-communication/SKILL.md @@ -0,0 +1,1090 @@ +--- +name: inter-service-communication +description: Inter-service communication patterns for GoodGo microservices including gRPC, GraphQL, service-to-service authentication, protocol selection, and client patterns. Use when implementing service-to-service calls, choosing communication protocols, or building service clients. +--- + +# Inter-Service Communication Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing service-to-service communication +- Choosing between REST, gRPC, or GraphQL protocols +- Setting up gRPC services and clients +- Implementing GraphQL services and resolvers +- Implementing service-to-service authentication +- Building resilient service clients with circuit breakers +- Managing connection pooling for service clients +- Implementing request/response interceptors +- Handling service discovery for internal calls +- Optimizing inter-service communication performance + +## Core Concepts + +### Communication Protocol Options + +**HTTP/REST:** +- Human-readable, easy to debug +- Browser-compatible +- Standard HTTP semantics +- JSON payloads +- Good for external APIs +- Use for: Client-facing APIs, simple CRUD operations + +**gRPC:** +- Binary protocol (Protocol Buffers) +- High performance, low latency +- Streaming support (unary, server streaming, client streaming, bidirectional) +- Strong typing with .proto files +- HTTP/2 based +- Use for: Internal service-to-service calls, high-throughput scenarios, streaming data + +**GraphQL:** +- Flexible query language +- Single endpoint +- Client-controlled data fetching +- Strong typing with schema +- Use for: Complex data requirements, mobile apps, reducing over-fetching + +### Protocol Selection Guidelines + +Choose protocol based on: + +1. **Use Case**: + - External APIs → REST + - Internal microservices → gRPC (performance) or REST (simplicity) + - Complex queries → GraphQL + +2. **Performance Requirements**: + - High throughput/low latency → gRPC + - Standard web APIs → REST + - Flexible data needs → GraphQL + +3. **Team Expertise**: + - REST: Widely known, easy onboarding + - gRPC: Requires Protocol Buffers knowledge + - GraphQL: Requires schema design skills + +4. **Ecosystem**: + - Browser clients → REST or GraphQL + - Mobile apps → REST or GraphQL + - Service-to-service → gRPC or REST + +## HTTP/REST Service Client + +### Service Client Setup + +```typescript +// src/core/clients/service-client.ts +// EN: Base service client for HTTP/REST communication +// VI: Service client cơ bản cho giao tiếp HTTP/REST +import axios, { AxiosInstance, AxiosRequestConfig } from 'axios'; +import { logger } from '@goodgo/logger'; +import { createCircuitBreaker } from '../resilience/circuit-breaker'; + +export interface ServiceClientConfig { + baseURL: string; + serviceName: string; + timeout?: number; + retries?: number; + enableCircuitBreaker?: boolean; +} + +export class ServiceClient { + private client: AxiosInstance; + private serviceName: string; + private circuitBreaker?: ReturnType; + + constructor(config: ServiceClientConfig) { + this.serviceName = config.serviceName; + + this.client = axios.create({ + baseURL: config.baseURL, + timeout: config.timeout || 5000, + headers: { + 'Content-Type': 'application/json', + 'User-Agent': `${process.env.SERVICE_NAME || 'unknown'}/1.0`, + }, + }); + + this.setupInterceptors(); + + // EN: Setup circuit breaker if enabled + // VI: Thiết lập circuit breaker nếu được bật + if (config.enableCircuitBreaker !== false) { + this.circuitBreaker = createCircuitBreaker( + async (requestConfig: AxiosRequestConfig) => { + return await this.client.request(requestConfig); + }, + `service-client-${config.serviceName}`, + { + timeout: config.timeout || 5000, + errorThresholdPercentage: 50, + resetTimeout: 30000, + } + ); + } + } + + private setupInterceptors(): void { + // EN: Request interceptor - add authentication and correlation ID + // VI: Request interceptor - thêm authentication và correlation ID + this.client.interceptors.request.use( + (config) => { + // EN: Add service-to-service authentication token + // VI: Thêm token xác thực service-to-service + if (process.env.INTERNAL_API_KEY) { + config.headers['X-Service-Auth'] = process.env.INTERNAL_API_KEY; + } + + // EN: Add correlation ID for tracing + // VI: Thêm correlation ID để tracing + const correlationId = config.headers['x-correlation-id'] || this.generateCorrelationId(); + config.headers['x-correlation-id'] = correlationId; + + logger.debug('Service request', { + service: this.serviceName, + method: config.method, + url: config.url, + correlationId, + }); + + return config; + }, + (error) => { + logger.error('Service request error', { service: this.serviceName, error }); + return Promise.reject(error); + } + ); + + // EN: Response interceptor - handle errors + // VI: Response interceptor - xử lý lỗi + this.client.interceptors.response.use( + (response) => { + logger.debug('Service response', { + service: this.serviceName, + status: response.status, + url: response.config.url, + }); + return response; + }, + async (error) => { + if (error.response) { + logger.warn('Service error response', { + service: this.serviceName, + status: error.response.status, + url: error.config?.url, + data: error.response.data, + }); + } else if (error.request) { + logger.error('Service request timeout', { + service: this.serviceName, + url: error.config?.url, + }); + } + + return Promise.reject(error); + } + ); + } + + private generateCorrelationId(): string { + return `corr_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`; + } + + /** + * EN: Make request with circuit breaker and retry + * VI: Thực hiện request với circuit breaker và retry + */ + async request(config: AxiosRequestConfig): Promise { + try { + const response = this.circuitBreaker + ? await this.circuitBreaker.fire(config) + : await this.client.request(config); + + return response.data; + } catch (error) { + logger.error('Service request failed', { + service: this.serviceName, + url: config.url, + error: error.message, + }); + throw error; + } + } + + async get(url: string, config?: AxiosRequestConfig): Promise { + return this.request({ ...config, method: 'GET', url }); + } + + async post(url: string, data?: any, config?: AxiosRequestConfig): Promise { + return this.request({ ...config, method: 'POST', url, data }); + } + + async put(url: string, data?: any, config?: AxiosRequestConfig): Promise { + return this.request({ ...config, method: 'PUT', url, data }); + } + + async delete(url: string, config?: AxiosRequestConfig): Promise { + return this.request({ ...config, method: 'DELETE', url }); + } +} +``` + +### Usage Example + +```typescript +// src/modules/notification/notification-client.service.ts +// EN: Example: Notification service client +// VI: Ví dụ: Notification service client +import { ServiceClient } from '../../core/clients/service-client'; + +const notificationClient = new ServiceClient({ + baseURL: process.env.NOTIFICATION_SERVICE_URL || 'http://notification-service:5003', + serviceName: 'notification-service', + timeout: 5000, + enableCircuitBreaker: true, +}); + +export class NotificationClientService { + async sendNotification(userId: string, message: string): Promise { + try { + await notificationClient.post('/api/v1/notifications', { + userId, + message, + }); + } catch (error) { + logger.error('Failed to send notification', { userId, error }); + throw error; + } + } +} +``` + +## gRPC Communication + +### Protocol Buffer Definitions + +```protobuf +// proto/user_service.proto +syntax = "proto3"; + +package goodgo.user.v1; + +option go_package = "github.com/goodgo/user-service/proto"; + +// EN: User service definition +// VI: Định nghĩa user service +service UserService { + // EN: Get user by ID + // VI: Lấy user theo ID + rpc GetUser(GetUserRequest) returns (GetUserResponse); + + // EN: Create user + // VI: Tạo user + rpc CreateUser(CreateUserRequest) returns (CreateUserResponse); + + // EN: Stream user updates + // VI: Stream cập nhật user + rpc StreamUserUpdates(StreamUserUpdatesRequest) returns (stream UserUpdate); +} + +message GetUserRequest { + string user_id = 1; +} + +message GetUserResponse { + User user = 1; +} + +message CreateUserRequest { + string email = 1; + string name = 2; +} + +message CreateUserResponse { + User user = 1; +} + +message User { + string id = 1; + string email = 2; + string name = 3; + int64 created_at = 4; +} + +message StreamUserUpdatesRequest { + string user_id = 1; +} + +message UserUpdate { + string user_id = 1; + string action = 2; + User user = 3; + int64 timestamp = 4; +} +``` + +### gRPC Server Implementation + +```typescript +// src/modules/user/user.grpc.service.ts +// EN: gRPC server implementation +// VI: Implementation gRPC server +import * as grpc from '@grpc/grpc-js'; +import * as protoLoader from '@grpc/proto-loader'; +import { UserService } from './user.service'; +import { logger } from '@goodgo/logger'; + +const PROTO_PATH = './proto/user_service.proto'; + +const packageDefinition = protoLoader.loadSync(PROTO_PATH, { + keepCase: true, + longs: String, + enums: String, + defaults: true, + oneofs: true, +}); + +const userProto = grpc.loadPackageDefinition(packageDefinition).goodgo?.user?.v1; + +export class UserGrpcServer { + private server: grpc.Server; + private userService: UserService; + + constructor(userService: UserService) { + this.userService = userService; + this.server = new grpc.Server(); + this.setupServices(); + } + + private setupServices(): void { + if (!userProto?.UserService) { + throw new Error('UserService proto not loaded'); + } + + this.server.addService(userProto.UserService.service, { + getUser: this.getUser.bind(this), + createUser: this.createUser.bind(this), + streamUserUpdates: this.streamUserUpdates.bind(this), + }); + } + + private async getUser( + call: grpc.ServerUnaryCall, + callback: grpc.sendUnaryData + ): Promise { + try { + const { user_id } = call.request; + const user = await this.userService.findById(user_id); + + if (!user) { + return callback({ + code: grpc.status.NOT_FOUND, + message: 'User not found', + }); + } + + callback(null, { + user: { + id: user.id, + email: user.email, + name: user.name, + created_at: user.createdAt.getTime(), + }, + }); + } catch (error) { + logger.error('gRPC getUser error', { error }); + callback({ + code: grpc.status.INTERNAL, + message: error.message, + }); + } + } + + private async createUser( + call: grpc.ServerUnaryCall, + callback: grpc.sendUnaryData + ): Promise { + try { + const { email, name } = call.request; + const user = await this.userService.create({ email, name }); + + callback(null, { + user: { + id: user.id, + email: user.email, + name: user.name, + created_at: user.createdAt.getTime(), + }, + }); + } catch (error) { + logger.error('gRPC createUser error', { error }); + callback({ + code: grpc.status.INTERNAL, + message: error.message, + }); + } + } + + private streamUserUpdates(call: grpc.ServerWritableStream): void { + const { user_id } = call.request; + + // EN: Stream user updates + // VI: Stream cập nhật user + const subscription = this.userService.subscribeToUpdates(user_id, (update) => { + call.write({ + user_id: update.userId, + action: update.action, + user: update.user, + timestamp: Date.now(), + }); + }); + + call.on('cancelled', () => { + subscription.unsubscribe(); + }); + } + + start(port: number): void { + this.server.bindAsync( + `0.0.0.0:${port}`, + grpc.ServerCredentials.createInsecure(), + (error, port) => { + if (error) { + logger.error('Failed to start gRPC server', { error }); + return; + } + + this.server.start(); + logger.info('gRPC server started', { port }); + } + ); + } + + stop(): Promise { + return new Promise((resolve) => { + this.server.tryShutdown(() => { + logger.info('gRPC server stopped'); + resolve(); + }); + }); + } +} +``` + +### gRPC Client Implementation + +```typescript +// src/core/clients/grpc-client.ts +// EN: gRPC client wrapper +// VI: Wrapper gRPC client +import * as grpc from '@grpc/grpc-js'; +import * as protoLoader from '@grpc/proto-loader'; +import { logger } from '@goodgo/logger'; + +export interface GrpcClientConfig { + protoPath: string; + packageName: string; + serviceName: string; + serverUrl: string; + options?: protoLoader.Options; +} + +export class GrpcClient { + private client: any; + private serviceName: string; + + constructor(config: GrpcClientConfig) { + this.serviceName = config.serviceName; + + const packageDefinition = protoLoader.loadSync(config.protoPath, { + keepCase: true, + longs: String, + enums: String, + defaults: true, + oneofs: true, + ...config.options, + }); + + const proto = grpc.loadPackageDefinition(packageDefinition); + const service = this.getNestedProperty(proto, config.packageName); + + if (!service?.[config.serviceName]) { + throw new Error(`Service ${config.serviceName} not found in proto`); + } + + this.client = new service[config.serviceName]( + config.serverUrl, + grpc.credentials.createInsecure() + ); + } + + private getNestedProperty(obj: any, path: string): any { + return path.split('.').reduce((current, prop) => current?.[prop], obj); + } + + /** + * EN: Make unary RPC call + * VI: Thực hiện unary RPC call + */ + async call( + methodName: string, + request: TRequest, + metadata?: grpc.Metadata + ): Promise { + return new Promise((resolve, reject) => { + // EN: Add service authentication metadata + // VI: Thêm metadata xác thực service + const callMetadata = metadata || new grpc.Metadata(); + if (process.env.INTERNAL_API_KEY) { + callMetadata.add('x-service-auth', process.env.INTERNAL_API_KEY); + } + + this.client[methodName](request, callMetadata, (error: any, response: TResponse) => { + if (error) { + logger.error('gRPC call error', { + service: this.serviceName, + method: methodName, + error: error.message, + }); + reject(error); + return; + } + + resolve(response); + }); + }); + } + + /** + * EN: Create streaming client + * VI: Tạo streaming client + */ + createStream( + methodName: string, + request: TRequest, + metadata?: grpc.Metadata + ): grpc.ClientReadableStream { + const callMetadata = metadata || new grpc.Metadata(); + if (process.env.INTERNAL_API_KEY) { + callMetadata.add('x-service-auth', process.env.INTERNAL_API_KEY); + } + + return this.client[methodName](request, callMetadata); + } +} +``` + +### gRPC Client Usage + +```typescript +// src/modules/user/user.grpc.client.ts +// EN: User service gRPC client +// VI: User service gRPC client +import { GrpcClient } from '../../core/clients/grpc-client'; + +const userGrpcClient = new GrpcClient({ + protoPath: './proto/user_service.proto', + packageName: 'goodgo.user.v1', + serviceName: 'UserService', + serverUrl: process.env.USER_SERVICE_GRPC_URL || 'localhost:50051', +}); + +export class UserGrpcClientService { + async getUser(userId: string) { + return userGrpcClient.call('getUser', { user_id: userId }); + } + + async createUser(email: string, name: string) { + return userGrpcClient.call('createUser', { email, name }); + } + + streamUserUpdates(userId: string, callback: (update: any) => void) { + const stream = userGrpcClient.createStream('streamUserUpdates', { + user_id: userId, + }); + + stream.on('data', callback); + stream.on('error', (error) => { + logger.error('gRPC stream error', { error }); + }); + } +} +``` + +## GraphQL Communication + +### GraphQL Schema Definition + +```graphql +# schema/user.graphql +# EN: User service GraphQL schema +# VI: GraphQL schema cho user service +type Query { + user(id: ID!): User + users(filter: UserFilter, pagination: Pagination): UserConnection +} + +type Mutation { + createUser(input: CreateUserInput!): User! + updateUser(id: ID!, input: UpdateUserInput!): User! + deleteUser(id: ID!): Boolean! +} + +type User { + id: ID! + email: String! + name: String + createdAt: DateTime! + updatedAt: DateTime! +} + +input CreateUserInput { + email: String! + name: String +} + +input UpdateUserInput { + name: String +} + +input UserFilter { + email: String + name: String +} + +input Pagination { + skip: Int + take: Int +} + +type UserConnection { + nodes: [User!]! + total: Int! + hasNextPage: Boolean! +} + +scalar DateTime +``` + +### GraphQL Server (Apollo) + +```typescript +// src/modules/user/user.graphql.service.ts +// EN: GraphQL server implementation +// VI: Implementation GraphQL server +import { ApolloServer } from '@apollo/server'; +import { expressMiddleware } from '@apollo/server/express4'; +import { UserService } from './user.service'; +import { logger } from '@goodgo/logger'; +import { readFileSync } from 'fs'; +import { resolvers } from './user.resolvers'; + +const typeDefs = readFileSync('./schema/user.graphql', 'utf-8'); + +export function createGraphQLServer(userService: UserService): ApolloServer { + return new ApolloServer({ + typeDefs, + resolvers: resolvers(userService), + plugins: [ + { + requestDidStart() { + return { + didResolveOperation(requestContext) { + logger.debug('GraphQL operation', { + operation: requestContext.operationName, + query: requestContext.request.query, + }); + }, + didEncounterErrors(requestContext) { + logger.error('GraphQL errors', { + errors: requestContext.errors, + }); + }, + }; + }, + }, + ], + }); +} + +// src/modules/user/user.resolvers.ts +export function resolvers(userService: UserService) { + return { + Query: { + user: async (_: any, { id }: { id: string }) => { + return await userService.findById(id); + }, + users: async ( + _: any, + { filter, pagination }: { filter?: any; pagination?: any } + ) => { + const result = await userService.findAll({ filter, pagination }); + return { + nodes: result.users, + total: result.total, + hasNextPage: result.users.length === (pagination?.take || 10), + }; + }, + }, + Mutation: { + createUser: async (_: any, { input }: { input: any }) => { + return await userService.create(input); + }, + updateUser: async (_: any, { id, input }: { id: string; input: any }) => { + return await userService.update(id, input); + }, + deleteUser: async (_: any, { id }: { id: string }) => { + await userService.delete(id); + return true; + }, + }, + }; +} +``` + +### GraphQL Client + +```typescript +// src/core/clients/graphql-client.ts +// EN: GraphQL client wrapper +// VI: Wrapper GraphQL client +import { GraphQLClient } from 'graphql-request'; +import { logger } from '@goodgo/logger'; + +export interface GraphQLClientConfig { + endpoint: string; + headers?: Record; + timeout?: number; +} + +export class GraphQLServiceClient { + private client: GraphQLClient; + private endpoint: string; + + constructor(config: GraphQLClientConfig) { + this.endpoint = config.endpoint; + this.client = new GraphQLClient(config.endpoint, { + headers: { + 'Content-Type': 'application/json', + ...(process.env.INTERNAL_API_KEY && { + 'x-service-auth': process.env.INTERNAL_API_KEY, + }), + ...config.headers, + }, + timeout: config.timeout || 5000, + }); + } + + /** + * EN: Execute GraphQL query + * VI: Thực hiện GraphQL query + */ + async query(query: string, variables?: any): Promise { + try { + const data = await this.client.request(query, variables); + return data; + } catch (error) { + logger.error('GraphQL query error', { + endpoint: this.endpoint, + error: error.message, + }); + throw error; + } + } + + /** + * EN: Execute GraphQL mutation + * VI: Thực hiện GraphQL mutation + */ + async mutate(mutation: string, variables?: any): Promise { + try { + const data = await this.client.request(mutation, variables); + return data; + } catch (error) { + logger.error('GraphQL mutation error', { + endpoint: this.endpoint, + error: error.message, + }); + throw error; + } + } +} + +// Usage example +const userGraphQLClient = new GraphQLServiceClient({ + endpoint: process.env.USER_SERVICE_GRAPHQL_URL || 'http://user-service:5002/graphql', +}); + +const GET_USER_QUERY = ` + query GetUser($id: ID!) { + user(id: $id) { + id + email + name + createdAt + } + } +`; + +export async function getUser(userId: string) { + return userGraphQLClient.query(GET_USER_QUERY, { id: userId }); +} +``` + +## Service-to-Service Authentication + +### Internal Authentication Middleware + +```typescript +// src/middlewares/internal-auth.middleware.ts +// EN: Middleware for authenticating internal service requests +// VI: Middleware để xác thực requests từ services nội bộ +import { Request, Response, NextFunction } from 'express'; +import { logger } from '@goodgo/logger'; + +export const internalAuthMiddleware = ( + req: Request, + res: Response, + next: NextFunction +): void => { + const serviceAuthToken = req.headers['x-service-auth']; + + if (!serviceAuthToken) { + logger.warn('Missing service auth token', { + ip: req.ip, + path: req.path, + }); + res.status(401).json({ + success: false, + error: { + code: 'INTERNAL_AUTH_REQUIRED', + message: 'Service authentication required', + }, + }); + return; + } + + // EN: Validate service token + // VI: Xác thực service token + if (serviceAuthToken !== process.env.INTERNAL_API_KEY) { + logger.warn('Invalid service auth token', { + ip: req.ip, + path: req.path, + }); + res.status(403).json({ + success: false, + error: { + code: 'INVALID_SERVICE_AUTH', + message: 'Invalid service authentication', + }, + }); + return; + } + + // EN: Add service context to request + // VI: Thêm context service vào request + req.serviceContext = { + authenticated: true, + timestamp: new Date(), + }; + + next(); +}; +``` + +### Mutual TLS (mTLS) for gRPC + +```typescript +// src/config/grpc-tls.config.ts +// EN: mTLS configuration for gRPC +// VI: Cấu hình mTLS cho gRPC +import * as grpc from '@grpc/grpc-js'; +import { readFileSync } from 'fs'; + +export function createServerCredentials(): grpc.ServerCredentials { + return grpc.ServerCredentials.createSsl( + null, + [ + { + cert_chain: readFileSync(process.env.GRPC_SERVER_CERT_PATH!), + private_key: readFileSync(process.env.GRPC_SERVER_KEY_PATH!), + }, + ], + true // EN: Request client certificate / VI: Yêu cầu client certificate + ); +} + +export function createClientCredentials(): grpc.ChannelCredentials { + return grpc.credentials.createSsl( + readFileSync(process.env.GRPC_CA_CERT_PATH!), + readFileSync(process.env.GRPC_CLIENT_KEY_PATH!), + readFileSync(process.env.GRPC_CLIENT_CERT_PATH!) + ); +} +``` + +## Connection Pooling + +### HTTP Connection Pool + +```typescript +// src/core/clients/connection-pool.ts +// EN: Connection pool for service clients +// VI: Connection pool cho service clients +import { ServiceClient } from './service-client'; + +export class ServiceClientPool { + private pools: Map = new Map(); + + getClient(serviceName: string, config: ServiceClientConfig): ServiceClient { + if (!this.pools.has(serviceName)) { + this.pools.set(serviceName, new ServiceClient(config)); + } + return this.pools.get(serviceName)!; + } + + clearPool(serviceName: string): void { + this.pools.delete(serviceName); + } + + clearAll(): void { + this.pools.clear(); + } +} + +export const serviceClientPool = new ServiceClientPool(); +``` + +## Error Handling + +### Service Error Classes + +```typescript +// src/core/errors/service-errors.ts +// EN: Custom errors for service communication +// VI: Custom errors cho giao tiếp service +export class ServiceUnavailableError extends Error { + constructor(public serviceName: string, message?: string) { + super(message || `Service ${serviceName} is unavailable`); + this.name = 'ServiceUnavailableError'; + } +} + +export class ServiceTimeoutError extends Error { + constructor(public serviceName: string, public timeout: number) { + super(`Service ${serviceName} request timed out after ${timeout}ms`); + this.name = 'ServiceTimeoutError'; + } +} + +export class ServiceError extends Error { + constructor( + public serviceName: string, + public statusCode: number, + message: string, + public details?: any + ) { + super(message); + this.name = 'ServiceError'; + } +} +``` + +## Best Practices + +### Protocol Selection + +1. **Use REST for**: + - External/public APIs + - Browser clients + - Simple CRUD operations + - When team is familiar with REST + +2. **Use gRPC for**: + - Internal service-to-service calls + - High-performance requirements + - Streaming data + - Strong typing needs + +3. **Use GraphQL for**: + - Complex data requirements + - Mobile applications + - Reducing over-fetching + - Flexible querying + +### Performance Optimization + +- **Connection Pooling**: Reuse connections for HTTP clients +- **Keep-Alive**: Enable HTTP keep-alive for persistent connections +- **Compression**: Use gzip compression for HTTP responses +- **Timeouts**: Set appropriate timeouts for all service calls +- **Circuit Breaker**: Implement circuit breakers to prevent cascade failures + +### Security + +- **Service Authentication**: Always authenticate internal service calls +- **TLS/mTLS**: Use TLS for all service communication, mTLS for gRPC +- **Secrets Management**: Store API keys in environment variables or secrets manager +- **Rate Limiting**: Implement rate limiting for service clients + +### Observability + +- **Logging**: Log all service calls with correlation IDs +- **Metrics**: Track service call duration, success rate, error rate +- **Tracing**: Add distributed tracing to service calls +- **Health Checks**: Monitor service health before making calls + +## Testing Patterns + +### Mocking Service Clients + +```typescript +// src/__tests__/mocks/service-client.mock.ts +// EN: Mock service client for testing +// VI: Mock service client cho testing +export const createMockServiceClient = () => { + return { + get: jest.fn(), + post: jest.fn(), + put: jest.fn(), + delete: jest.fn(), + request: jest.fn(), + }; +}; + +// Usage in tests +const mockUserClient = createMockServiceClient(); +mockUserClient.get.mockResolvedValue({ id: '123', email: 'test@example.com' }); +``` + +### Integration Testing + +```typescript +// src/__tests__/integration/service-communication.e2e.ts +// EN: Integration tests for service communication +// VI: Integration tests cho giao tiếp service +describe('Service Communication E2E', () => { + it('should communicate with user service via HTTP', async () => { + const client = new ServiceClient({ + baseURL: process.env.USER_SERVICE_URL || 'http://localhost:5002', + serviceName: 'user-service', + }); + + const user = await client.get('/api/v1/users/123'); + expect(user).toHaveProperty('id'); + }); +}); +``` + +## Resources + +- [gRPC Documentation](https://grpc.io/docs/) - Official gRPC documentation +- [GraphQL Documentation](https://graphql.org/learn/) - GraphQL learning resources +- [Protocol Buffers](https://developers.google.com/protocol-buffers) - Protocol Buffer guide +- [Apollo Server](https://www.apollographql.com/docs/apollo-server/) - GraphQL server framework +- [Resilience Patterns](../resilience-patterns/SKILL.md) - Circuit breaker, retry patterns +- [Security](../security/SKILL.md) - Authentication, authorization patterns +- [API Design](../api-design/SKILL.md) - RESTful API patterns +- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards diff --git a/.cursor/skills/performance-optimization/SKILL.md b/.cursor/skills/performance-optimization/SKILL.md new file mode 100644 index 00000000..3cf1e3f7 --- /dev/null +++ b/.cursor/skills/performance-optimization/SKILL.md @@ -0,0 +1,368 @@ +--- +name: performance-optimization +description: Performance optimization patterns for GoodGo microservices including database query optimization, memory leak detection, profiling, connection pooling, and caching strategies. Use when optimizing performance, profiling applications, or detecting performance bottlenecks. +--- + +# Performance Optimization Patterns + +## When to Use This Skill + +Use this skill when: +- Optimizing database queries +- Detecting and fixing memory leaks +- Profiling application performance +- Optimizing connection pooling +- Improving caching strategies +- Identifying N+1 query problems +- Optimizing batch operations +- Improving application startup time +- Reducing memory usage +- Optimizing API response times + +## Core Concepts + +### Performance Metrics + +1. **Response Time**: Time to process request +2. **Throughput**: Requests processed per second +3. **Memory Usage**: Memory consumption +4. **CPU Usage**: CPU utilization +5. **Database Query Time**: Query execution time + +### Optimization Areas + +- Database queries +- Memory management +- Connection pooling +- Caching +- Batch processing +- Lazy loading + +## Database Query Optimization + +### Query Analysis + +```typescript +// src/core/db/query-analyzer.ts +// EN: Query performance analyzer +// VI: Phân tích hiệu suất query +import { PrismaClient } from '@prisma/client'; +import { logger } from '@goodgo/logger'; + +export class QueryAnalyzer { + constructor(private prisma: PrismaClient) {} + + /** + * EN: Analyze query performance + * VI: Phân tích hiệu suất query + */ + async analyzeQuery(query: string): Promise { + const result = await this.prisma.$queryRawUnsafe(`EXPLAIN ANALYZE ${query}`); + logger.info('Query analysis', { query, result }); + return result; + } + + /** + * EN: Check for missing indexes + * VI: Kiểm tra indexes bị thiếu + */ + async checkIndexes(tableName: string): Promise { + const indexes = await this.prisma.$queryRawUnsafe(` + SELECT * FROM pg_indexes WHERE tablename = '${tableName}' + `); + return indexes; + } +} +``` + +### N+1 Query Prevention + +```typescript +// EN: Bad: N+1 queries +// VI: Xấu: N+1 queries +async function getUsersWithOrdersBad(): Promise { + const users = await userRepository.findAll(); + for (const user of users) { + user.orders = await orderRepository.findByUserId(user.id); // N+1! + } + return users; +} + +// EN: Good: Use include/join +// VI: Tốt: Sử dụng include/join +async function getUsersWithOrdersGood(): Promise { + return await userRepository.findAll({ + include: { + orders: true, // EN: Single query with join / VI: Single query với join + }, + }); +} +``` + +### Batch Operations + +```typescript +// src/core/db/batch-operations.ts +// EN: Batch database operations +// VI: Batch database operations +export class BatchOperations { + /** + * EN: Batch create operations + * VI: Batch create operations + */ + async batchCreate(items: T[], batchSize: number = 100): Promise { + const results: T[] = []; + + for (let i = 0; i < items.length; i += batchSize) { + const batch = items.slice(i, i + batchSize); + const batchResults = await prisma.$transaction( + batch.map((item) => prisma.item.create({ data: item })) + ); + results.push(...batchResults); + } + + return results; + } + + /** + * EN: Batch update operations + * VI: Batch update operations + */ + async batchUpdate( + updates: Array<{ id: string; data: any }>, + batchSize: number = 100 + ): Promise { + for (let i = 0; i < updates.length; i += batchSize) { + const batch = updates.slice(i, i + batchSize); + await prisma.$transaction( + batch.map(({ id, data }) => prisma.item.update({ where: { id }, data })) + ); + } + } +} +``` + +## Memory Leak Detection + +### Memory Profiling + +```typescript +// src/core/performance/memory-profiler.ts +// EN: Memory leak detection +// VI: Phát hiện memory leak +import { performance } from 'perf_hooks'; +import { logger } from '@goodgo/logger'; + +export class MemoryProfiler { + private baseline?: NodeJS.MemoryUsage; + private interval?: NodeJS.Timeout; + + /** + * EN: Start memory profiling + * VI: Bắt đầu memory profiling + */ + start(): void { + this.baseline = process.memoryUsage(); + + this.interval = setInterval(() => { + const current = process.memoryUsage(); + const delta = { + heapUsed: current.heapUsed - (this.baseline?.heapUsed || 0), + heapTotal: current.heapTotal - (this.baseline?.heapTotal || 0), + external: current.external - (this.baseline?.external || 0), + }; + + logger.info('Memory usage', { + current: { + heapUsed: `${(current.heapUsed / 1024 / 1024).toFixed(2)} MB`, + heapTotal: `${(current.heapTotal / 1024 / 1024).toFixed(2)} MB`, + }, + delta: { + heapUsed: `${(delta.heapUsed / 1024 / 1024).toFixed(2)} MB`, + }, + }); + + // EN: Alert if memory growth is excessive + // VI: Cảnh báo nếu memory tăng quá mức + if (delta.heapUsed > 100 * 1024 * 1024) { + // 100MB growth + logger.warn('Potential memory leak detected', { delta }); + } + }, 60000); // EN: Check every minute / VI: Kiểm tra mỗi phút + } + + stop(): void { + if (this.interval) { + clearInterval(this.interval); + } + } + + /** + * EN: Force garbage collection (if --expose-gc flag is set) + * VI: Ép garbage collection (nếu --expose-gc flag được set) + */ + forceGC(): void { + if (global.gc) { + global.gc(); + } + } +} +``` + +## Connection Pooling + +### Database Connection Pool + +```typescript +// src/config/database.config.ts +// EN: Optimize Prisma connection pool +// VI: Tối ưu Prisma connection pool +export const prisma = new PrismaClient({ + datasources: { + db: { + url: process.env.DATABASE_URL, + }, + }, + log: process.env.NODE_ENV === 'development' ? ['query', 'error'] : ['error'], +}); + +// EN: Connection pool configuration in DATABASE_URL +// VI: Cấu hình connection pool trong DATABASE_URL +// DATABASE_URL="postgresql://user:pass@host:5432/db?connection_limit=20&pool_timeout=10" +``` + +### HTTP Connection Pool + +```typescript +// src/core/clients/http-pool.config.ts +// EN: HTTP connection pooling with keep-alive +// VI: HTTP connection pooling với keep-alive +import axios from 'axios'; +import https from 'https'; + +const httpAgent = new https.Agent({ + keepAlive: true, + keepAliveMsecs: 1000, + maxSockets: 50, + maxFreeSockets: 10, + timeout: 60000, +}); + +export const httpClient = axios.create({ + httpAgent, + httpsAgent: httpAgent, + timeout: 30000, +}); +``` + +## Performance Monitoring + +```typescript +// src/core/performance/performance-monitor.ts +// EN: Performance monitoring middleware +// VI: Middleware giám sát hiệu suất +import { Request, Response, NextFunction } from 'express'; +import { performance } from 'perf_hooks'; +import { logger } from '@goodgo/logger'; + +export function performanceMonitor(req: Request, res: Response, next: NextFunction) { + const start = performance.now(); + const startMemory = process.memoryUsage(); + + res.on('finish', () => { + const duration = performance.now() - start; + const endMemory = process.memoryUsage(); + + logger.info('Request performance', { + method: req.method, + path: req.path, + status: res.statusCode, + duration: `${duration.toFixed(2)}ms`, + memoryDelta: `${((endMemory.heapUsed - startMemory.heapUsed) / 1024 / 1024).toFixed(2)} MB`, + }); + + // EN: Alert on slow requests + // VI: Cảnh báo trên requests chậm + if (duration > 1000) { + logger.warn('Slow request detected', { + path: req.path, + duration: `${duration.toFixed(2)}ms`, + }); + } + }); + + next(); +} +``` + +## Caching Optimization + +```typescript +// src/core/cache/cache-optimizer.ts +// EN: Cache optimization strategies +// VI: Chiến lược tối ưu cache +export class CacheOptimizer { + /** + * EN: Preload frequently accessed data + * VI: Tải trước dữ liệu thường truy cập + */ + async preloadHotData(keys: string[]): Promise { + for (const key of keys) { + // EN: Fetch and cache data + // VI: Fetch và cache dữ liệu + await cacheService.getOrSet(key, async () => { + return await this.fetchData(key); + }, 3600); + } + } + + /** + * EN: Batch cache operations + * VI: Batch cache operations + */ + async batchGet(keys: string[]): Promise> { + const results = new Map(); + const missing: string[] = []; + + // EN: Check cache for all keys + // VI: Kiểm tra cache cho tất cả keys + for (const key of keys) { + const cached = await cacheService.get(key); + if (cached) { + results.set(key, cached); + } else { + missing.push(key); + } + } + + // EN: Fetch missing data + // VI: Fetch dữ liệu thiếu + if (missing.length > 0) { + const data = await this.fetchBatch(missing); + for (const [key, value] of Object.entries(data)) { + results.set(key, value); + await cacheService.set(key, value, 3600); + } + } + + return results; + } +} +``` + +## Best Practices + +1. **Database**: Use indexes, avoid N+1 queries, use batch operations +2. **Memory**: Monitor memory usage, detect leaks, use streaming for large data +3. **Caching**: Cache frequently accessed data, use appropriate TTLs +4. **Connection Pooling**: Configure pool sizes appropriately +5. **Profiling**: Profile regularly to identify bottlenecks +6. **Monitoring**: Monitor performance metrics continuously + +## Resources + +- [Node.js Performance](https://nodejs.org/en/docs/guides/simple-profiling/) +- [Prisma Performance](https://www.prisma.io/docs/guides/performance-and-optimization) +- [Caching Patterns](../caching-patterns/SKILL.md) - Caching strategies +- [Observability & Monitoring](../observability-monitoring/SKILL.md) - Monitoring patterns +- [Database & Prisma](../database-prisma/SKILL.md) - Database patterns diff --git a/.cursor/skills/service-discovery-registry/SKILL.md b/.cursor/skills/service-discovery-registry/SKILL.md new file mode 100644 index 00000000..c9286431 --- /dev/null +++ b/.cursor/skills/service-discovery-registry/SKILL.md @@ -0,0 +1,428 @@ +--- +name: service-discovery-registry +description: Service discovery and registry patterns for GoodGo microservices including service registry, health check orchestration, load balancing strategies, and service mesh integration. Use when implementing service discovery, managing service health, or integrating with service mesh. +--- + +# Service Discovery & Registry Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing service discovery mechanisms +- Managing service registry +- Orchestrating health checks +- Implementing load balancing strategies +- Integrating with service mesh +- Managing dynamic service registration +- Implementing DNS-based discovery +- Building service catalog + +## Core Concepts + +### Service Discovery Types + +1. **Client-Side Discovery**: Client queries service registry +2. **Server-Side Discovery**: Load balancer queries registry (Kubernetes DNS) +3. **Service Mesh**: Automatic service discovery (Istio, Linkerd) + +### Service Registry + +- **Static Configuration**: Hardcoded service addresses +- **Dynamic Registry**: Services register/unregister dynamically +- **DNS-Based**: Use DNS for service discovery (Kubernetes) + +## Kubernetes Service Discovery + +### DNS-Based Discovery + +```yaml +# Kubernetes automatically creates DNS records +# Service: user-service.namespace.svc.cluster.local +apiVersion: v1 +kind: Service +metadata: + name: user-service + namespace: production +spec: + selector: + app: user-service + ports: + - port: 80 + targetPort: 5000 +``` + +### Service Discovery in Code + +```typescript +// src/core/discovery/kubernetes-discovery.ts +// EN: Kubernetes DNS-based service discovery +// VI: Service discovery dựa trên DNS của Kubernetes +export class KubernetesServiceDiscovery { + /** + * EN: Get service URL using Kubernetes DNS + * VI: Lấy service URL sử dụng Kubernetes DNS + */ + getServiceUrl(serviceName: string, namespace: string = 'default'): string { + // EN: Kubernetes DNS format: service.namespace.svc.cluster.local + // VI: Format DNS Kubernetes: service.namespace.svc.cluster.local + return `http://${serviceName}.${namespace}.svc.cluster.local`; + } + + /** + * EN: Get service URL with port + * VI: Lấy service URL với port + */ + getServiceUrlWithPort(serviceName: string, port: number, namespace: string = 'default'): string { + return `http://${serviceName}.${namespace}.svc.cluster.local:${port}`; + } +} + +// Usage +const discovery = new KubernetesServiceDiscovery(); +const userServiceUrl = discovery.getServiceUrl('user-service', 'production'); +``` + +## Service Registry Pattern + +### Service Registry Implementation + +```typescript +// src/core/registry/service-registry.ts +// EN: Service registry for dynamic service registration +// VI: Service registry cho đăng ký service động +import { PrismaClient } from '@prisma/client'; +import { logger } from '@goodgo/logger'; + +export interface ServiceInfo { + name: string; + version: string; + url: string; + healthCheckUrl: string; + status: 'healthy' | 'unhealthy' | 'unknown'; + lastHeartbeat: Date; + metadata?: Record; +} + +export class ServiceRegistry { + constructor(private prisma: PrismaClient) {} + + /** + * EN: Register service + * VI: Đăng ký service + */ + async register(serviceInfo: ServiceInfo): Promise { + await this.prisma.serviceRegistry.upsert({ + where: { name: serviceInfo.name }, + update: { + version: serviceInfo.version, + url: serviceInfo.url, + healthCheckUrl: serviceInfo.healthCheckUrl, + status: serviceInfo.status, + lastHeartbeat: new Date(), + metadata: serviceInfo.metadata || {}, + }, + create: { + name: serviceInfo.name, + version: serviceInfo.version, + url: serviceInfo.url, + healthCheckUrl: serviceInfo.healthCheckUrl, + status: serviceInfo.status, + lastHeartbeat: new Date(), + metadata: serviceInfo.metadata || {}, + }, + }); + + logger.info('Service registered', { serviceName: serviceInfo.name }); + } + + /** + * EN: Discover service + * VI: Tìm service + */ + async discover(serviceName: string): Promise { + const service = await this.prisma.serviceRegistry.findUnique({ + where: { name: serviceName }, + }); + + if (!service || service.status !== 'healthy') { + return null; + } + + return { + name: service.name, + version: service.version, + url: service.url, + healthCheckUrl: service.healthCheckUrl, + status: service.status as 'healthy' | 'unhealthy' | 'unknown', + lastHeartbeat: service.lastHeartbeat, + metadata: service.metadata as Record || {}, + }; + } + + /** + * EN: List all healthy services + * VI: Liệt kê tất cả services healthy + */ + async listHealthyServices(): Promise { + const services = await this.prisma.serviceRegistry.findMany({ + where: { + status: 'healthy', + lastHeartbeat: { + gte: new Date(Date.now() - 60000), // EN: Last heartbeat within 1 minute / VI: Heartbeat trong 1 phút qua + }, + }, + }); + + return services.map((s) => ({ + name: s.name, + version: s.version, + url: s.url, + healthCheckUrl: s.healthCheckUrl, + status: s.status as 'healthy', + lastHeartbeat: s.lastHeartbeat, + metadata: s.metadata as Record || {}, + })); + } + + /** + * EN: Unregister service + * VI: Hủy đăng ký service + */ + async unregister(serviceName: string): Promise { + await this.prisma.serviceRegistry.delete({ + where: { name: serviceName }, + }); + + logger.info('Service unregistered', { serviceName }); + } +} +``` + +### Service Registration on Startup + +```typescript +// src/core/registry/service-registration.ts +// EN: Auto-register service on startup +// VI: Tự động đăng ký service khi khởi động +import { ServiceRegistry, ServiceInfo } from './service-registry'; +import { logger } from '@goodgo/logger'; + +export class ServiceRegistration { + private registry: ServiceRegistry; + private heartbeatInterval?: NodeJS.Timeout; + private serviceInfo: ServiceInfo; + + constructor(registry: ServiceRegistry, serviceInfo: ServiceInfo) { + this.registry = registry; + this.serviceInfo = serviceInfo; + } + + /** + * EN: Register and start heartbeat + * VI: Đăng ký và bắt đầu heartbeat + */ + async start(): Promise { + // EN: Initial registration + // VI: Đăng ký ban đầu + await this.registry.register(this.serviceInfo); + + // EN: Start heartbeat + // VI: Bắt đầu heartbeat + this.heartbeatInterval = setInterval(async () => { + try { + // EN: Check health and update registry + // VI: Kiểm tra health và cập nhật registry + const isHealthy = await this.checkHealth(); + await this.registry.register({ + ...this.serviceInfo, + status: isHealthy ? 'healthy' : 'unhealthy', + }); + } catch (error) { + logger.error('Heartbeat failed', { error }); + } + }, 30000); // EN: Every 30 seconds / VI: Mỗi 30 giây + + logger.info('Service registration started', { serviceName: this.serviceInfo.name }); + } + + /** + * EN: Stop registration and cleanup + * VI: Dừng registration và cleanup + */ + async stop(): Promise { + if (this.heartbeatInterval) { + clearInterval(this.heartbeatInterval); + } + + await this.registry.unregister(this.serviceInfo.name); + logger.info('Service registration stopped', { serviceName: this.serviceInfo.name }); + } + + private async checkHealth(): Promise { + try { + const response = await fetch(this.serviceInfo.healthCheckUrl, { + timeout: 5000, + }); + return response.ok; + } catch (error) { + return false; + } + } +} + +// Usage in main.ts +const serviceRegistry = new ServiceRegistry(prisma); +const serviceRegistration = new ServiceRegistration(serviceRegistry, { + name: process.env.SERVICE_NAME || 'unknown-service', + version: process.env.SERVICE_VERSION || '1.0.0', + url: `http://${process.env.SERVICE_NAME}:${process.env.PORT}`, + healthCheckUrl: `http://localhost:${process.env.PORT}/health`, + status: 'healthy', + lastHeartbeat: new Date(), +}); + +await serviceRegistration.start(); + +process.on('SIGTERM', async () => { + await serviceRegistration.stop(); + process.exit(0); +}); +``` + +## Health Check Orchestration + +### Aggregated Health Check + +```typescript +// src/core/health/health-aggregator.ts +// EN: Aggregate health checks from multiple services +// VI: Tổng hợp health checks từ nhiều services +import { ServiceRegistry } from '../registry/service-registry'; +import { logger } from '@goodgo/logger'; + +export class HealthAggregator { + constructor(private serviceRegistry: ServiceRegistry) {} + + /** + * EN: Get aggregated health status + * VI: Lấy trạng thái health tổng hợp + */ + async getAggregatedHealth(): Promise<{ + status: 'healthy' | 'degraded' | 'unhealthy'; + services: Array<{ name: string; status: string; lastHeartbeat: Date }>; + }> { + const services = await this.serviceRegistry.listHealthyServices(); + + const serviceStatuses = services.map((s) => ({ + name: s.name, + status: s.status, + lastHeartbeat: s.lastHeartbeat, + })); + + const unhealthyCount = serviceStatuses.filter((s) => s.status !== 'healthy').length; + const totalCount = serviceStatuses.length; + + let overallStatus: 'healthy' | 'degraded' | 'unhealthy'; + if (unhealthyCount === 0) { + overallStatus = 'healthy'; + } else if (unhealthyCount < totalCount / 2) { + overallStatus = 'degraded'; + } else { + overallStatus = 'unhealthy'; + } + + return { + status: overallStatus, + services: serviceStatuses, + }; + } +} +``` + +## Load Balancing Strategies + +### Client-Side Load Balancing + +```typescript +// src/core/discovery/load-balancer.ts +// EN: Client-side load balancing +// VI: Client-side load balancing +export class LoadBalancer { + /** + * EN: Round-robin load balancing + * VI: Round-robin load balancing + */ + roundRobin(items: T[]): T { + // EN: Simple round-robin implementation + // VI: Implementation round-robin đơn giản + const index = Math.floor(Math.random() * items.length); + return items[index]; + } + + /** + * EN: Least connections load balancing + * VI: Least connections load balancing + */ + leastConnections(items: T[]): T { + return items.reduce((min, item) => + item.connections < min.connections ? item : min + ); + } + + /** + * EN: Weighted round-robin + * VI: Weighted round-robin + */ + weightedRoundRobin(items: T[]): T { + const totalWeight = items.reduce((sum, item) => sum + item.weight, 0); + let random = Math.random() * totalWeight; + + for (const item of items) { + random -= item.weight; + if (random <= 0) { + return item; + } + } + + return items[items.length - 1]; + } +} +``` + +## Service Mesh Integration + +### Istio Service Discovery + +```yaml +# EN: Istio automatically handles service discovery +# VI: Istio tự động xử lý service discovery +apiVersion: networking.istio.io/v1alpha3 +kind: ServiceEntry +metadata: + name: external-service +spec: + hosts: + - external-api.example.com + ports: + - number: 443 + name: https + protocol: HTTPS + location: MESH_EXTERNAL + resolution: DNS +``` + +## Best Practices + +1. **Kubernetes DNS**: Use Kubernetes DNS for service discovery +2. **Health Checks**: Implement comprehensive health checks +3. **Service Registry**: Use registry for dynamic services +4. **Load Balancing**: Use appropriate load balancing strategy +5. **Monitoring**: Monitor service discovery and health + +## Resources + +- [Kubernetes Service Discovery](https://kubernetes.io/docs/concepts/services-networking/service/) +- [Istio Service Mesh](https://istio.io/latest/docs/ops/deployment/) +- [Deployment Kubernetes](../deployment-kubernetes/SKILL.md) - K8s patterns +- [Observability & Monitoring](../observability-monitoring/SKILL.md) - Health checks +- [Project Rules](../project-rules/SKILL.md) - GoodGo standards diff --git a/docs/en/skills/README.md b/docs/en/skills/README.md index 410edf4c..e76723d6 100644 --- a/docs/en/skills/README.md +++ b/docs/en/skills/README.md @@ -8,7 +8,7 @@ Cursor Skills are specialized knowledge modules that guide AI assistants in foll ## Available Skills -The GoodGo platform includes **15 Cursor Skills** organized by category: +The GoodGo platform includes **25 Cursor Skills** organized by category: ### API & Data Layer @@ -46,6 +46,24 @@ Service layer organization and patterns for GoodGo microservices. Use when imple #### [Kubernetes Deployment](./deployment-kubernetes.md) Kubernetes deployment patterns for GoodGo microservices. Use when deploying to staging/production, creating K8s manifests, configuring HPA, setting up ingress, or troubleshooting K8s deployments. +#### [Event-Driven Architecture](./event-driven-architecture.md) +Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. + +#### [Inter-Service Communication](./inter-service-communication.md) +Inter-service communication patterns for GoodGo microservices including gRPC, GraphQL, service-to-service authentication, protocol selection, and client patterns. Use when implementing service-to-service calls, choosing communication protocols, or building service clients. + +#### [Data Consistency Patterns](./data-consistency-patterns.md) +Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services. + +#### [API Gateway Advanced](./api-gateway-advanced.md) +Advanced API Gateway patterns for GoodGo microservices including API composition, request/response transformation, service mesh integration, advanced routing, and gateway-level resilience. Use when implementing API aggregation, service composition, or advanced gateway features. + +#### [Configuration Management](./configuration-management.md) +Configuration management patterns for GoodGo microservices including feature flags, dynamic configuration reloading, environment-specific configurations, and secrets management. Use when implementing feature toggles, managing configuration, or handling environment variables. + +#### [Performance Optimization](./performance-optimization.md) +Performance optimization patterns for GoodGo microservices including database query optimization, memory leak detection, profiling, connection pooling, and caching strategies. Use when optimizing performance, profiling applications, or detecting performance bottlenecks. + #### [Observability & Monitoring](./observability-monitoring.md) Observability and monitoring patterns for GoodGo microservices. Use when adding metrics, implementing logging, setting up tracing, creating health checks, or debugging production issues. @@ -55,6 +73,18 @@ Resilience patterns for GoodGo microservices including circuit breaker, retry st #### [Microservices Development Process](./microservices-development-process.md) Standard development process for creating and maintaining microservices in GoodGo platform. Use when creating new services, migrating services, refactoring services, or planning service implementations. +#### [CI/CD Advanced Patterns](./cicd-advanced-patterns.md) +Advanced CI/CD patterns for GoodGo microservices including blue-green deployments, canary releases, automated rollback, deployment verification, and progressive delivery. Use when implementing advanced deployment strategies, automated rollbacks, or progressive delivery pipelines. + +#### [Infrastructure as Code](./infrastructure-as-code.md) +Infrastructure as Code patterns for GoodGo platform including Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, and multi-environment management. Use when managing infrastructure, implementing GitOps, or creating reusable infrastructure modules. + +#### [API Versioning Strategy](./api-versioning-strategy.md) +API versioning strategies for GoodGo microservices including semantic versioning, backward compatibility patterns, API deprecation, version negotiation, and breaking changes handling. Use when versioning APIs, handling breaking changes, or implementing API deprecation strategies. + +#### [Service Discovery & Registry](./service-discovery-registry.md) +Service discovery and registry patterns for GoodGo microservices including service registry, health check orchestration, load balancing strategies, and service mesh integration. Use when implementing service discovery, managing service health, or integrating with service mesh. + ### Standards & Security #### [Project Rules](./project-rules.md) @@ -80,6 +110,12 @@ Guidelines for writing technical documentation in the GoodGo project. Use when c | Write documentation | Documentation, Comment Code | | Implement authentication | Security, API Design, Database & Prisma | | Optimize database queries | Database & Prisma, Observability | +| Implement event-driven communication | Event-Driven Architecture, Resilience Patterns | +| Implement service-to-service calls | Inter-Service Communication, Security, Resilience Patterns | +| Handle distributed transactions | Data Consistency Patterns, Event-Driven Architecture | +| Optimize performance | Performance Optimization, Observability & Monitoring | +| Manage feature flags | Configuration Management | +| Deploy with zero downtime | CI/CD Advanced Patterns, Deployment Kubernetes | ### Skill Dependencies @@ -108,6 +144,39 @@ Application Layer └── Error Handling Patterns Infrastructure + ├── Event-Driven Architecture + │ ├── Resilience Patterns + │ ├── Error Handling Patterns + │ └── Observability & Monitoring + ├── Inter-Service Communication + │ ├── API Design + │ ├── Security + │ └── Resilience Patterns + ├── Data Consistency Patterns + │ ├── Event-Driven Architecture + │ ├── Database & Prisma + │ └── Error Handling Patterns + ├── API Gateway Advanced + │ ├── Middleware Patterns + │ ├── Security + │ └── API Design + ├── Configuration Management + │ └── Observability & Monitoring + ├── Performance Optimization + │ ├── Database & Prisma + │ ├── Caching Patterns + │ └── Observability & Monitoring + ├── CI/CD Advanced Patterns + │ ├── Deployment Kubernetes + │ └── Testing Patterns + ├── Infrastructure as Code + │ └── Deployment Kubernetes + ├── API Versioning Strategy + │ ├── API Design + │ └── Middleware Patterns + ├── Service Discovery & Registry + │ ├── Deployment Kubernetes + │ └── Observability & Monitoring └── Resilience Patterns ├── Error Handling Patterns └── Service Layer Patterns diff --git a/docs/en/skills/api-gateway-advanced.md b/docs/en/skills/api-gateway-advanced.md new file mode 100644 index 00000000..bdd13e1a --- /dev/null +++ b/docs/en/skills/api-gateway-advanced.md @@ -0,0 +1,58 @@ +--- +name: api-gateway-advanced +description: Advanced API Gateway patterns for GoodGo microservices including API composition, request/response transformation, service mesh integration, advanced routing, and gateway-level resilience. +--- + +# API Gateway Advanced Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing API composition and aggregation +- Transforming requests/responses at gateway level +- Integrating service mesh with Traefik +- Implementing advanced routing strategies +- Adding gateway-level circuit breakers +- Implementing API versioning at gateway + +## Key Patterns + +### API Composition + +```typescript +// Aggregate multiple service responses +const [user, orders, payments] = await Promise.all([ + userClient.get(`/users/${userId}`), + orderClient.get(`/orders?userId=${userId}`), + paymentClient.get(`/payments?userId=${userId}`), +]); +``` + +### Request/Response Transformation + +```typescript +// Transform at gateway level +transformer.addRule({ + path: '/api/v1/users', + requestTransform: (req) => { + if (!req.query.page) req.query.page = '1'; + return req; + }, + responseTransform: (res, data) => { + return { success: true, data }; + }, +}); +``` + +## Best Practices + +1. Use API composition for aggregating related data +2. Cache at gateway for frequently accessed data +3. Implement circuit breaker at gateway level +4. Keep transformations simple and testable + +## Resources + +- [Traefik Documentation](https://doc.traefik.io/traefik/) +- [Middleware Patterns](./middleware-patterns.md) +- Skill Source: `.cursor/skills/api-gateway-advanced/SKILL.md` diff --git a/docs/en/skills/api-versioning-strategy.md b/docs/en/skills/api-versioning-strategy.md new file mode 100644 index 00000000..5fe797f3 --- /dev/null +++ b/docs/en/skills/api-versioning-strategy.md @@ -0,0 +1,55 @@ +--- +name: api-versioning-strategy +description: API versioning strategies for GoodGo microservices including semantic versioning, backward compatibility patterns, API deprecation, version negotiation, and breaking changes handling. +--- + +# API Versioning Strategy + +## When to Use This Skill + +Use this skill when: +- Versioning APIs +- Handling breaking changes +- Implementing API deprecation +- Maintaining backward compatibility +- Implementing version negotiation + +## Key Patterns + +### URL Path Versioning + +```typescript +// Version routes +router.use('/v1', v1Router); +router.use('/v2', v2Router); +``` + +### Header-Based Versioning + +```typescript +// Version negotiation +const acceptHeader = req.headers.accept; +const version = acceptHeader.match(/application\/vnd\.goodgo\.v(\d+)\+json/)[1]; +``` + +### Deprecation + +```typescript +// Deprecation headers +res.setHeader('Deprecation', 'true'); +res.setHeader('Sunset', '2024-12-31'); +res.setHeader('Warning', 'API version 1 is deprecated'); +``` + +## Best Practices + +1. Choose versioning strategy and be consistent +2. Use semantic versioning (MAJOR.MINOR.PATCH) +3. Always deprecate before removing +4. Provide migration guides +5. Maintain backward compatibility when possible + +## Resources + +- [API Design](./api-design.md) +- Skill Source: `.cursor/skills/api-versioning-strategy/SKILL.md` diff --git a/docs/en/skills/cicd-advanced-patterns.md b/docs/en/skills/cicd-advanced-patterns.md new file mode 100644 index 00000000..b9c0bd8f --- /dev/null +++ b/docs/en/skills/cicd-advanced-patterns.md @@ -0,0 +1,61 @@ +--- +name: cicd-advanced-patterns +description: Advanced CI/CD patterns for GoodGo microservices including blue-green deployments, canary releases, automated rollback, deployment verification, and progressive delivery. +--- + +# CI/CD Advanced Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing blue-green deployments +- Setting up canary releases +- Implementing automated rollback mechanisms +- Creating deployment verification pipelines +- Implementing progressive delivery + +## Key Patterns + +### Blue-Green Deployment + +```yaml +# Switch service selector between blue/green +apiVersion: v1 +kind: Service +spec: + selector: + version: blue # Switch to green after verification +``` + +### Canary Deployment + +```yaml +# Split traffic between stable and canary +http: + - route: + - destination: + subset: stable + weight: 90 + - destination: + subset: canary + weight: 10 +``` + +### Automated Rollback + +```bash +# Rollback to previous revision +kubectl rollout undo deployment/service --to-revision=1 +``` + +## Best Practices + +1. Use blue-green for zero-downtime deployments +2. Use canary for gradual rollouts with monitoring +3. Always have automated rollback plan +4. Run smoke tests immediately after deployment + +## Resources + +- [Deployment Kubernetes](./deployment-kubernetes.md) +- Skill Source: `.cursor/skills/cicd-advanced-patterns/SKILL.md` diff --git a/docs/en/skills/configuration-management.md b/docs/en/skills/configuration-management.md new file mode 100644 index 00000000..610d2d9d --- /dev/null +++ b/docs/en/skills/configuration-management.md @@ -0,0 +1,57 @@ +--- +name: configuration-management +description: Configuration management patterns for GoodGo microservices including feature flags, dynamic configuration reloading, environment-specific configurations, and secrets management. +--- + +# Configuration Management Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing feature flags and feature toggles +- Managing environment-specific configurations +- Implementing dynamic configuration reloading +- Managing secrets and sensitive configuration +- Implementing configuration validation + +## Key Patterns + +### Feature Flags + +```typescript +// Check if feature is enabled +const enabled = await featureFlagService.isEnabled('new-feature', userId); + +if (enabled) { + // Use new feature +} +``` + +### Dynamic Configuration + +```typescript +// Load and auto-reload configuration +await configService.load(); +configService.startAutoReload(60000); // Reload every minute + +const value = configService.get('config-key', 'default-value'); +``` + +### Configuration Validation + +```typescript +// Validate with Zod +const config = validateConfig(process.env); +``` + +## Best Practices + +1. Always validate configuration at startup +2. Provide sensible defaults +3. Never commit secrets to code +4. Use feature flags for gradual rollouts + +## Resources + +- [Feature Flags Pattern](https://martinfowler.com/articles/feature-toggles.html) +- Skill Source: `.cursor/skills/configuration-management/SKILL.md` diff --git a/docs/en/skills/data-consistency-patterns.md b/docs/en/skills/data-consistency-patterns.md new file mode 100644 index 00000000..16755725 --- /dev/null +++ b/docs/en/skills/data-consistency-patterns.md @@ -0,0 +1,96 @@ +--- +name: data-consistency-patterns +description: Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services. +--- + +# Data Consistency Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing distributed transactions across multiple services +- Handling eventual consistency in microservices +- Implementing Saga patterns for distributed workflows +- Designing compensation strategies for failed transactions +- Implementing idempotent operations +- Managing data synchronization across services +- Handling conflict resolution + +## Core Concepts + +### ACID vs BASE + +**ACID (Traditional):** Atomicity, Consistency, Isolation, Durability + +**BASE (Distributed):** Basic Availability, Soft state, Eventual consistency + +### Consistency Models + +- **Strong Consistency**: All nodes see same data at same time +- **Eventual Consistency**: System becomes consistent over time +- **Weak Consistency**: No guarantees about when consistency occurs + +## Key Patterns + +### Saga Orchestrator Pattern + +```typescript +// Centralized orchestrator coordinates steps +const saga = new SagaOrchestrator(); +await saga.execute({ + sagaId: 'saga_123', + steps: [ + { name: 'create-order', execute: createOrder, compensate: cancelOrder }, + { name: 'reserve-inventory', execute: reserveInventory, compensate: releaseInventory }, + { name: 'process-payment', execute: chargePayment, compensate: refundPayment }, + ], + data: {}, + status: 'pending', +}); +``` + +### Saga Choreography Pattern + +```typescript +// Services react to events +eventConsumer.on('order.created', async (event) => { + await inventoryService.reserve(event.data.items); + await eventPublisher.publish('inventory.reserved', {...}); +}); +``` + +### Idempotency + +```typescript +// Execute operation with idempotency check +await idempotencyHandler.execute( + idempotencyKey, + async () => await userService.create(data) +); +``` + +### Optimistic Locking + +```typescript +// Update with version check +await optimisticLockService.updateWithLock( + repository, + id, + (current) => ({ ...current, name: newName }) +); +``` + +## Best Practices + +1. **Design Compensations**: Every step needs compensation +2. **Idempotent Steps**: Make steps idempotent for retries +3. **Conflict Resolution**: Define resolution strategies +4. **Monitoring**: Track saga execution and consistency lag +5. **Read Models**: Use separate read models for queries (CQRS) + +## Resources + +- [Saga Pattern](https://microservices.io/patterns/data/saga.html) +- [Event-Driven Architecture](./event-driven-architecture.md) +- [Error Handling Patterns](./error-handling-patterns.md) +- Skill Source: `.cursor/skills/data-consistency-patterns/SKILL.md` diff --git a/docs/en/skills/event-driven-architecture.md b/docs/en/skills/event-driven-architecture.md new file mode 100644 index 00000000..ea12f7ca --- /dev/null +++ b/docs/en/skills/event-driven-architecture.md @@ -0,0 +1,332 @@ +--- +name: event-driven-architecture +description: Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. +--- + +# Event-Driven Architecture Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing asynchronous communication between services +- Decoupling services for better scalability +- Publishing domain events for downstream consumers +- Consuming events from other services +- Implementing event sourcing patterns +- Implementing CQRS (Command Query Responsibility Segregation) +- Exposing event streams via HTTP (SSE/WebSocket) +- Handling eventual consistency across services +- Building reactive systems that respond to changes +- Integrating with Apache Kafka message broker + +## Core Concepts + +### Event-Driven vs Request-Response + +**Request-Response (Synchronous):** +- Client waits for response +- Tight coupling between services +- Blocking operations +- Immediate consistency +- Use Traefik API Gateway for HTTP/REST + +**Event-Driven (Asynchronous):** +- Fire-and-forget publishing +- Loose coupling between services +- Non-blocking operations +- Eventual consistency +- Use Kafka for message broker + +### Kafka Fundamentals + +- **Topics**: Named streams of events (e.g., `user.created`, `order.placed`) +- **Partitions**: Physical division of topics for parallelism and scaling +- **Consumer Groups**: Groups of consumers that work together to process events +- **Producers**: Services that publish events to topics +- **Consumers**: Services that subscribe to topics and process events + +### Traefik Integration + +Traefik serves dual purpose: +- **API Gateway**: Routes synchronous HTTP/REST requests +- **Event Streaming Gateway**: Routes SSE/WebSocket connections to event streaming endpoints + +Services publish events to Kafka, then expose SSE/WebSocket endpoints that consume from Kafka for HTTP clients. + +## Key Patterns + +### Event Publishing + +```typescript +// src/core/events/event-publisher.ts +import { producer } from '../config/kafka.config'; +import { logger } from '@goodgo/logger'; +import { v4 as uuidv4 } from 'uuid'; + +export class EventPublisher { + async publish( + topic: string, + event: Omit, + options?: { partitionKey?: string } + ): Promise { + const fullEvent: T = { + ...event, + eventId: uuidv4(), + timestamp: new Date().toISOString(), + source: this.serviceName, + } as T; + + await producer.send({ + topic, + messages: [{ + key: options?.partitionKey || fullEvent.eventId, + value: JSON.stringify(fullEvent), + headers: { + 'event-type': event.eventType, + 'event-version': event.eventVersion, + }, + }], + }); + } +} +``` + +### Event Consuming + +```typescript +// src/core/events/event-consumer.ts +import { kafka } from '../config/kafka.config'; + +export class EventConsumer { + private handlers: Map = new Map(); + + on(eventType: string, handler: EventHandler): void { + if (!this.handlers.has(eventType)) { + this.handlers.set(eventType, []); + } + this.handlers.get(eventType)!.push(handler); + } + + async start(topics: string[]): Promise { + await this.consumer.connect(); + await this.consumer.subscribe({ topics, fromBeginning: false }); + + await this.consumer.run({ + eachMessage: async ({ topic, partition, message }) => { + const event: BaseEvent = JSON.parse(message.value?.toString() || '{}'); + const handlers = this.handlers.get(event.eventType) || []; + await Promise.all(handlers.map(h => h.handle(event))); + }, + }); + } +} +``` + +### Outbox Pattern for Transactional Publishing + +```typescript +// Store event in database within transaction +await prisma.outboxEvent.create({ + data: { + eventType: 'user.created', + eventData: userData, + topic: 'user.created', + status: 'PENDING', + }, +}); + +// Separate process publishes from outbox to Kafka +async function processOutbox() { + const events = await prisma.outboxEvent.findMany({ + where: { status: 'PENDING' }, + }); + + for (const event of events) { + await eventPublisher.publish(event.topic, event.eventData); + await prisma.outboxEvent.update({ + where: { id: event.id }, + data: { status: 'PUBLISHED' }, + }); + } +} +``` + +### SSE Endpoint for Event Streaming + +```typescript +// src/modules/events/events.controller.ts +async streamEvents(req: Request, res: Response): Promise { + res.setHeader('Content-Type', 'text/event-stream'); + res.setHeader('Cache-Control', 'no-cache'); + res.setHeader('Connection', 'keep-alive'); + + const topic = req.query.topic as string; + const consumer = kafka.consumer({ groupId: `sse-${Date.now()}` }); + + await consumer.connect(); + await consumer.subscribe({ topic, fromBeginning: false }); + + await consumer.run({ + eachMessage: async ({ message }) => { + const event = JSON.parse(message.value?.toString() || '{}'); + res.write(`data: ${JSON.stringify(event)}\n\n`); + }, + }); + + req.on('close', async () => { + await consumer.disconnect(); + }); +} +``` + +## Event Structure + +```typescript +interface BaseEvent { + eventId: string; + eventType: string; + eventVersion: string; + timestamp: string; + source: string; + correlationId?: string; + traceId?: string; + data: unknown; +} +``` + +## Best Practices + +### Event Naming Conventions + +- **Event Type**: `{domain}.{action}.v{version}` (e.g., `user.created.v1`) +- **Topic**: `{domain}.{entity}.{action}` (e.g., `user.created`) +- Use lowercase with dots as separators +- Keep names descriptive and consistent + +### Partition Key Selection + +- Use entity ID for ordering guarantees (same entity → same partition) +- Use correlation ID for request tracing +- Use user ID for user-scoped events +- Avoid high-cardinality keys (distributes evenly) + +### Event Ordering Guarantees + +- Kafka guarantees ordering **per partition** +- Use partition key to ensure related events go to same partition +- Events in different partitions have no ordering guarantee +- Don't rely on global ordering across all events + +### Error Handling + +- Implement Dead Letter Queue (DLQ) for failed events +- Use retry with exponential backoff +- Log all event processing failures +- Monitor consumer lag and DLQ size + +### Observability + +- Log all published and consumed events +- Track metrics: events published/consumed, processing duration, consumer lag +- Add distributed tracing to event flows +- Include correlation IDs for request tracking + +## Infrastructure Setup + +### Docker Compose (Local) + +```yaml +services: + kafka: + image: confluentinc/cp-kafka:7.4.0 + ports: + - "9092:9092" + environment: + KAFKA_BROKER_ID: 1 + KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 + + schema-registry: + image: confluentinc/cp-schema-registry:7.4.0 + ports: + - "8081:8081" + environment: + SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:9092 +``` + +## Testing + +### Unit Testing + +```typescript +import { EventPublisher } from '../event-publisher'; +import { producer } from '../../config/kafka.config'; + +jest.mock('../../config/kafka.config'); + +describe('EventPublisher', () => { + it('should publish event successfully', async () => { + const publisher = new EventPublisher(); + const mockSend = jest.fn().mockResolvedValue({}); + (producer.send as jest.Mock) = mockSend; + + await publisher.publish('user.created', { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { userId: '123' }, + }); + + expect(mockSend).toHaveBeenCalled(); + }); +}); +``` + +### Integration Testing with Test Containers + +```typescript +import { KafkaContainer } from '@testcontainers/kafka'; + +describe('Event Flow E2E', () => { + let kafkaContainer: StartedKafkaContainer; + + beforeAll(async () => { + kafkaContainer = await new KafkaContainer().start(); + process.env.KAFKA_BROKERS = kafkaContainer.getBootstrapServer(); + }); + + it('should publish and consume event', async () => { + // Test implementation + }); +}); +``` + +## Common Use Cases + +### User Created Event Flow + +1. Auth Service creates user in database +2. Publishes `user.created` event to Kafka +3. Notification Service consumes event and sends welcome email +4. Analytics Service consumes event and updates metrics + +### Order Processing with Multiple Consumers + +1. Order Service publishes `order.placed` event +2. Payment Service processes payment +3. Inventory Service reserves items +4. Notification Service sends confirmation + +## Related Skills + +- [Resilience Patterns](./resilience-patterns.md) - Circuit breaker, retry patterns +- [Error Handling Patterns](./error-handling-patterns.md) - Error handling best practices +- [Observability & Monitoring](./observability-monitoring.md) - Logging, metrics, tracing +- [Middleware Patterns](./middleware-patterns.md) - SSE endpoint middleware +- [Project Rules](./project-rules.md) - GoodGo coding standards + +## Resources + +- [KafkaJS Documentation](https://kafka.js.org/) - Node.js Kafka client +- [Confluent Schema Registry](https://docs.confluent.io/platform/current/schema-registry/index.html) - Schema versioning +- [Kafka Best Practices](https://kafka.apache.org/documentation/#best_practices) - Official Kafka documentation +- Skill Source: `.cursor/skills/event-driven-architecture/SKILL.md` diff --git a/docs/en/skills/infrastructure-as-code.md b/docs/en/skills/infrastructure-as-code.md new file mode 100644 index 00000000..d88cd266 --- /dev/null +++ b/docs/en/skills/infrastructure-as-code.md @@ -0,0 +1,56 @@ +--- +name: infrastructure-as-code +description: Infrastructure as Code patterns for GoodGo platform including Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, and multi-environment management. +--- + +# Infrastructure as Code Patterns + +## When to Use This Skill + +Use this skill when: +- Managing infrastructure with code +- Implementing Terraform modules +- Setting up GitOps workflows +- Creating Kubernetes operators +- Testing infrastructure changes + +## Key Patterns + +### Terraform Modules + +```hcl +# Reusable module +module "postgresql" { + source = "../../modules/postgresql" + database_name = "goodgo" + environment = "staging" +} +``` + +### GitOps with ArgoCD + +```yaml +# Automated sync from Git +spec: + source: + repoURL: https://github.com/goodgo/platform + path: deployments/production/kubernetes + syncPolicy: + automated: + prune: true + selfHeal: true +``` + +## Best Practices + +1. Keep all infrastructure in version control +2. Create reusable Terraform modules +3. Test infrastructure changes before applying +4. Use GitOps for Kubernetes deployments +5. Separate environments completely + +## Resources + +- [Terraform Documentation](https://www.terraform.io/docs) +- [Deployment Kubernetes](./deployment-kubernetes.md) +- Skill Source: `.cursor/skills/infrastructure-as-code/SKILL.md` diff --git a/docs/en/skills/inter-service-communication.md b/docs/en/skills/inter-service-communication.md new file mode 100644 index 00000000..c33a0494 --- /dev/null +++ b/docs/en/skills/inter-service-communication.md @@ -0,0 +1,177 @@ +--- +name: inter-service-communication +description: Inter-service communication patterns for GoodGo microservices including gRPC, GraphQL, service-to-service authentication, protocol selection, and client patterns. Use when implementing service-to-service calls, choosing communication protocols, or building service clients. +--- + +# Inter-Service Communication Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing service-to-service communication +- Choosing between REST, gRPC, or GraphQL protocols +- Setting up gRPC services and clients +- Implementing GraphQL services and resolvers +- Implementing service-to-service authentication +- Building resilient service clients with circuit breakers +- Managing connection pooling for service clients +- Implementing request/response interceptors +- Handling service discovery for internal calls +- Optimizing inter-service communication performance + +## Core Concepts + +### Communication Protocol Options + +**HTTP/REST:** +- Human-readable, easy to debug +- Browser-compatible +- Standard HTTP semantics +- JSON payloads +- Good for external APIs + +**gRPC:** +- Binary protocol (Protocol Buffers) +- High performance, low latency +- Streaming support +- Strong typing with .proto files +- HTTP/2 based + +**GraphQL:** +- Flexible query language +- Single endpoint +- Client-controlled data fetching +- Strong typing with schema + +### Protocol Selection Guidelines + +Choose protocol based on use case, performance requirements, team expertise, and ecosystem needs. + +## Key Patterns + +### HTTP/REST Service Client + +```typescript +// Base service client with circuit breaker and interceptors +import { ServiceClient } from '../../core/clients/service-client'; + +const notificationClient = new ServiceClient({ + baseURL: process.env.NOTIFICATION_SERVICE_URL || 'http://notification-service:5003', + serviceName: 'notification-service', + timeout: 5000, + enableCircuitBreaker: true, +}); + +// Usage +await notificationClient.post('/api/v1/notifications', { + userId, + message, +}); +``` + +### gRPC Service + +```typescript +// gRPC server implementation +import { UserGrpcServer } from './user.grpc.service'; + +const grpcServer = new UserGrpcServer(userService); +grpcServer.start(50051); + +// gRPC client +import { GrpcClient } from '../../core/clients/grpc-client'; + +const userGrpcClient = new GrpcClient({ + protoPath: './proto/user_service.proto', + packageName: 'goodgo.user.v1', + serviceName: 'UserService', + serverUrl: 'localhost:50051', +}); + +const user = await userGrpcClient.call('getUser', { user_id: '123' }); +``` + +### GraphQL Service + +```typescript +// GraphQL client +import { GraphQLServiceClient } from '../../core/clients/graphql-client'; + +const userGraphQLClient = new GraphQLServiceClient({ + endpoint: 'http://user-service:5002/graphql', +}); + +const GET_USER_QUERY = ` + query GetUser($id: ID!) { + user(id: $id) { + id + email + name + } + } +`; + +const user = await userGraphQLClient.query(GET_USER_QUERY, { id: '123' }); +``` + +### Service-to-Service Authentication + +```typescript +// Internal auth middleware +import { internalAuthMiddleware } from '../../middlewares/internal-auth.middleware'; + +router.use('/internal', internalAuthMiddleware); + +// Client automatically adds auth header +const client = new ServiceClient({ + baseURL: 'http://service:5000', + serviceName: 'service', +}); +// X-Service-Auth header is added automatically +``` + +## Best Practices + +### Protocol Selection + +- **REST**: External APIs, browser clients, simple CRUD +- **gRPC**: Internal services, high performance, streaming +- **GraphQL**: Complex queries, mobile apps, flexible data + +### Performance + +- Use connection pooling +- Enable HTTP keep-alive +- Set appropriate timeouts +- Implement circuit breakers + +### Security + +- Always authenticate internal calls +- Use TLS/mTLS +- Store secrets securely +- Implement rate limiting + +### Observability + +- Log with correlation IDs +- Track metrics (duration, success rate) +- Add distributed tracing +- Monitor service health + +## Testing + +```typescript +// Mock service client +const mockClient = createMockServiceClient(); +mockClient.get.mockResolvedValue({ id: '123' }); +``` + +## Resources + +- [gRPC Documentation](https://grpc.io/docs/) +- [GraphQL Documentation](https://graphql.org/learn/) +- [Protocol Buffers](https://developers.google.com/protocol-buffers) +- [Resilience Patterns](./resilience-patterns.md) +- [Security](./security.md) +- Skill Source: `.cursor/skills/inter-service-communication/SKILL.md` diff --git a/docs/en/skills/performance-optimization.md b/docs/en/skills/performance-optimization.md new file mode 100644 index 00000000..06b7ee3c --- /dev/null +++ b/docs/en/skills/performance-optimization.md @@ -0,0 +1,62 @@ +--- +name: performance-optimization +description: Performance optimization patterns for GoodGo microservices including database query optimization, memory leak detection, profiling, connection pooling, and caching strategies. +--- + +# Performance Optimization Patterns + +## When to Use This Skill + +Use this skill when: +- Optimizing database queries +- Detecting and fixing memory leaks +- Profiling application performance +- Optimizing connection pooling +- Improving caching strategies +- Identifying N+1 query problems + +## Key Patterns + +### Database Query Optimization + +```typescript +// Avoid N+1 queries +// Bad: Multiple queries +for (const user of users) { + user.orders = await orderRepository.findByUserId(user.id); +} + +// Good: Single query with join +const users = await userRepository.findAll({ + include: { orders: true }, +}); +``` + +### Memory Profiling + +```typescript +// Monitor memory usage +const profiler = new MemoryProfiler(); +profiler.start(); // Monitor every minute +``` + +### Batch Operations + +```typescript +// Batch database operations +await batchOperations.batchCreate(items, 100); // Process 100 at a time +``` + +## Best Practices + +1. Use indexes, avoid N+1 queries +2. Monitor memory usage, detect leaks +3. Cache frequently accessed data +4. Configure connection pools appropriately +5. Profile regularly to identify bottlenecks + +## Resources + +- [Caching Patterns](./caching-patterns.md) +- [Observability & Monitoring](./observability-monitoring.md) +- Skill Source: `.cursor/skills/performance-optimization/SKILL.md` diff --git a/docs/en/skills/service-discovery-registry.md b/docs/en/skills/service-discovery-registry.md new file mode 100644 index 00000000..319eba05 --- /dev/null +++ b/docs/en/skills/service-discovery-registry.md @@ -0,0 +1,57 @@ +--- +name: service-discovery-registry +description: Service discovery and registry patterns for GoodGo microservices including service registry, health check orchestration, load balancing strategies, and service mesh integration. +--- + +# Service Discovery & Registry Patterns + +## When to Use This Skill + +Use this skill when: +- Implementing service discovery mechanisms +- Managing service registry +- Orchestrating health checks +- Implementing load balancing strategies +- Integrating with service mesh + +## Key Patterns + +### Kubernetes DNS Discovery + +```typescript +// Use Kubernetes DNS for service discovery +const serviceUrl = `http://user-service.production.svc.cluster.local`; +``` + +### Service Registry + +```typescript +// Register service +await serviceRegistry.register({ + name: 'user-service', + url: 'http://user-service:5000', + status: 'healthy', +}); + +// Discover service +const service = await serviceRegistry.discover('user-service'); +``` + +### Health Check Aggregation + +```typescript +// Aggregate health from multiple services +const health = await healthAggregator.getAggregatedHealth(); +``` + +## Best Practices + +1. Use Kubernetes DNS for service discovery +2. Implement comprehensive health checks +3. Use service registry for dynamic services +4. Choose appropriate load balancing strategy + +## Resources + +- [Deployment Kubernetes](./deployment-kubernetes.md) +- Skill Source: `.cursor/skills/service-discovery-registry/SKILL.md` diff --git a/docs/vi/skills/README.md b/docs/vi/skills/README.md index 46c2d133..e927be21 100644 --- a/docs/vi/skills/README.md +++ b/docs/vi/skills/README.md @@ -11,7 +11,7 @@ Cursor Skills là các module kiến thức chuyên biệt hướng dẫn AI ass ## Các Skills Có Sẵn -GoodGo platform bao gồm **15 Cursor Skills** được tổ chức theo danh mục: +GoodGo platform bao gồm **25 Cursor Skills** được tổ chức theo danh mục: ### API & Data Layer @@ -69,6 +69,36 @@ Kubernetes deployment patterns for GoodGo microservices. Use when deploying to s Kubernetes deployment patterns cho GoodGo microservices. Sử dụng khi deploy lên staging/production, tạo K8s manifests, config HPA, setup ingress, hoặc troubleshoot K8s deployments. +#### [Event-Driven Architecture](./event-driven-architecture.md) +Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. + +Các patterns kiến trúc hướng sự kiện với Apache Kafka cho GoodGo microservices. Sử dụng khi implement giao tiếp bất đồng bộ, publish/consume events, event sourcing, CQRS, hoặc tích hợp event streams với HTTP endpoints. + +#### [Inter-Service Communication](./inter-service-communication.md) +Inter-service communication patterns for GoodGo microservices including gRPC, GraphQL, service-to-service authentication, protocol selection, and client patterns. Use when implementing service-to-service calls, choosing communication protocols, or building service clients. + +Các patterns giao tiếp giữa services cho GoodGo microservices bao gồm gRPC, GraphQL, service-to-service authentication, lựa chọn protocol, và client patterns. Sử dụng khi implement service-to-service calls, chọn communication protocols, hoặc xây dựng service clients. + +#### [Data Consistency Patterns](./data-consistency-patterns.md) +Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services. + +Các patterns nhất quán dữ liệu cho distributed microservices bao gồm Saga patterns, distributed transactions, eventual consistency, compensation, và idempotency. Sử dụng khi xử lý distributed transactions, implement eventual consistency, hoặc quản lý đồng bộ dữ liệu giữa các services. + +#### [API Gateway Advanced](./api-gateway-advanced.md) +Advanced API Gateway patterns for GoodGo microservices including API composition, request/response transformation, service mesh integration, advanced routing, and gateway-level resilience. Use when implementing API aggregation, service composition, or advanced gateway features. + +Các patterns API Gateway nâng cao cho GoodGo microservices bao gồm API composition, request/response transformation, tích hợp service mesh, routing nâng cao, và resilience ở gateway level. Sử dụng khi implement API aggregation, service composition, hoặc các tính năng gateway nâng cao. + +#### [Configuration Management](./configuration-management.md) +Configuration management patterns for GoodGo microservices including feature flags, dynamic configuration reloading, environment-specific configurations, and secrets management. Use when implementing feature toggles, managing configuration, or handling environment variables. + +Các patterns quản lý cấu hình cho GoodGo microservices bao gồm feature flags, dynamic configuration reloading, environment-specific configurations, và secrets management. Sử dụng khi implement feature toggles, quản lý configuration, hoặc xử lý environment variables. + +#### [Performance Optimization](./performance-optimization.md) +Performance optimization patterns for GoodGo microservices including database query optimization, memory leak detection, profiling, connection pooling, and caching strategies. Use when optimizing performance, profiling applications, or detecting performance bottlenecks. + +Các patterns tối ưu hiệu suất cho GoodGo microservices bao gồm tối ưu database queries, phát hiện memory leaks, profiling, connection pooling, và caching strategies. Sử dụng khi tối ưu hiệu suất, profiling ứng dụng, hoặc phát hiện performance bottlenecks. + #### [Observability & Monitoring](./observability-monitoring.md) Observability and monitoring patterns for GoodGo microservices. Use when adding metrics, implementing logging, setting up tracing, creating health checks, or debugging production issues. @@ -84,6 +114,26 @@ Standard development process for creating and maintaining microservices in GoodG Quy trình phát triển chuẩn để tạo và duy trì microservices trong nền tảng GoodGo. Sử dụng khi tạo services mới, migrate services, refactor services, hoặc lập kế hoạch implement services. +#### [CI/CD Advanced Patterns](./cicd-advanced-patterns.md) +Advanced CI/CD patterns for GoodGo microservices including blue-green deployments, canary releases, automated rollback, deployment verification, and progressive delivery. Use when implementing advanced deployment strategies, automated rollbacks, or progressive delivery pipelines. + +Các patterns CI/CD nâng cao cho GoodGo microservices bao gồm blue-green deployments, canary releases, automated rollback, deployment verification, và progressive delivery. Sử dụng khi implement các chiến lược deployment nâng cao, automated rollbacks, hoặc progressive delivery pipelines. + +#### [Infrastructure as Code](./infrastructure-as-code.md) +Infrastructure as Code patterns for GoodGo platform including Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, and multi-environment management. Use when managing infrastructure, implementing GitOps, or creating reusable infrastructure modules. + +Các patterns Infrastructure as Code cho nền tảng GoodGo bao gồm Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, và multi-environment management. Sử dụng khi quản lý infrastructure, implement GitOps, hoặc tạo các infrastructure modules tái sử dụng. + +#### [API Versioning Strategy](./api-versioning-strategy.md) +API versioning strategies for GoodGo microservices including semantic versioning, backward compatibility patterns, API deprecation, version negotiation, and breaking changes handling. Use when versioning APIs, handling breaking changes, or implementing API deprecation strategies. + +Các chiến lược versioning API cho GoodGo microservices bao gồm semantic versioning, backward compatibility patterns, API deprecation, version negotiation, và xử lý breaking changes. Sử dụng khi versioning APIs, xử lý breaking changes, hoặc implement API deprecation strategies. + +#### [Service Discovery & Registry](./service-discovery-registry.md) +Service discovery and registry patterns for GoodGo microservices including service registry, health check orchestration, load balancing strategies, and service mesh integration. Use when implementing service discovery, managing service health, or integrating with service mesh. + +Các patterns service discovery và registry cho GoodGo microservices bao gồm service registry, health check orchestration, load balancing strategies, và tích hợp service mesh. Sử dụng khi implement service discovery, quản lý service health, hoặc tích hợp với service mesh. + ### Standards & Security #### [Project Rules](./project-rules.md) @@ -115,6 +165,12 @@ Hướng dẫn viết technical documentation trong dự án GoodGo. Sử dụng | Viết documentation | Documentation, Comment Code | | Implement authentication | Security, API Design, Database & Prisma | | Optimize database queries | Database & Prisma, Observability | +| Implement event-driven communication | Event-Driven Architecture, Resilience Patterns | +| Implement service-to-service calls | Inter-Service Communication, Security, Resilience Patterns | +| Handle distributed transactions | Data Consistency Patterns, Event-Driven Architecture | +| Optimize performance | Performance Optimization, Observability & Monitoring | +| Manage feature flags | Configuration Management | +| Deploy with zero downtime | CI/CD Advanced Patterns, Deployment Kubernetes | ### Phụ Thuộc Giữa Các Skills @@ -143,6 +199,39 @@ Application Layer └── Error Handling Patterns Infrastructure + ├── Event-Driven Architecture + │ ├── Resilience Patterns + │ ├── Error Handling Patterns + │ └── Observability & Monitoring + ├── Inter-Service Communication + │ ├── API Design + │ ├── Security + │ └── Resilience Patterns + ├── Data Consistency Patterns + │ ├── Event-Driven Architecture + │ ├── Database & Prisma + │ └── Error Handling Patterns + ├── API Gateway Advanced + │ ├── Middleware Patterns + │ ├── Security + │ └── API Design + ├── Configuration Management + │ └── Observability & Monitoring + ├── Performance Optimization + │ ├── Database & Prisma + │ ├── Caching Patterns + │ └── Observability & Monitoring + ├── CI/CD Advanced Patterns + │ ├── Deployment Kubernetes + │ └── Testing Patterns + ├── Infrastructure as Code + │ └── Deployment Kubernetes + ├── API Versioning Strategy + │ ├── API Design + │ └── Middleware Patterns + ├── Service Discovery & Registry + │ ├── Deployment Kubernetes + │ └── Observability & Monitoring └── Resilience Patterns ├── Error Handling Patterns └── Service Layer Patterns diff --git a/docs/vi/skills/api-gateway-advanced.md b/docs/vi/skills/api-gateway-advanced.md new file mode 100644 index 00000000..834f454a --- /dev/null +++ b/docs/vi/skills/api-gateway-advanced.md @@ -0,0 +1,45 @@ +# API Gateway Nâng Cao (API Gateway Advanced) + +Advanced API Gateway patterns for GoodGo microservices including API composition, request/response transformation, service mesh integration, advanced routing, and gateway-level resilience. +> Các patterns API Gateway nâng cao cho GoodGo microservices bao gồm API composition, request/response transformation, tích hợp service mesh, routing nâng cao, và resilience ở gateway level. + +## Tổng Quan + +Advanced API Gateway patterns extend basic gateway functionality with composition, transformation, service mesh integration, and gateway-level resilience patterns. + +Các patterns API Gateway nâng cao mở rộng chức năng gateway cơ bản với composition, transformation, tích hợp service mesh, và các resilience patterns ở gateway level. + +## Khi Nào Sử Dụng + +Use this skill when implementing API composition, request/response transformation, or service mesh integration. + +Sử dụng skill này khi implement API composition, request/response transformation, hoặc tích hợp service mesh. + +## Các Patterns Chính + +### API Composition / API Composition + +```typescript +// EN: Aggregate multiple service responses +// VI: Tổng hợp responses từ nhiều services +const [user, orders, payments] = await Promise.all([ + userClient.get(`/users/${userId}`), + orderClient.get(`/orders?userId=${userId}`), + paymentClient.get(`/payments?userId=${userId}`), +]); +``` + +## Best Practices / Thực Hành Tốt + +1. Use API composition for aggregating related data / Sử dụng API composition để tổng hợp dữ liệu liên quan +2. Cache at gateway / Cache ở gateway +3. Implement circuit breaker at gateway / Implement circuit breaker ở gateway + +## Skills Liên Quan + +- [Middleware Patterns](./middleware-patterns.md) - Middleware patterns / Các patterns middleware +- [Resilience Patterns](./resilience-patterns.md) - Circuit breaker / Circuit breaker + +## Tài Nguyên + +- Skill Source: `.cursor/skills/api-gateway-advanced/SKILL.md` diff --git a/docs/vi/skills/api-versioning-strategy.md b/docs/vi/skills/api-versioning-strategy.md new file mode 100644 index 00000000..e851265c --- /dev/null +++ b/docs/vi/skills/api-versioning-strategy.md @@ -0,0 +1,46 @@ +# Chiến Lược Versioning API (API Versioning Strategy) + +API versioning strategies for GoodGo microservices including semantic versioning, backward compatibility patterns, API deprecation, version negotiation, and breaking changes handling. +> Các chiến lược versioning API cho GoodGo microservices bao gồm semantic versioning, backward compatibility patterns, API deprecation, version negotiation, và xử lý breaking changes. + +## Tổng Quan + +API versioning strategies enable managing API evolution while maintaining compatibility and clearly communicating changes to consumers. + +Các chiến lược versioning API cho phép quản lý sự phát triển API đồng thời duy trì tương thích và giao tiếp rõ ràng các thay đổi tới consumers. + +## Khi Nào Sử Dụng + +Use this skill when versioning APIs, handling breaking changes, or implementing deprecation. + +Sử dụng skill này khi versioning APIs, xử lý breaking changes, hoặc implement deprecation. + +## Các Patterns Chính + +### URL Path Versioning / Versioning Theo Đường Dẫn + +```typescript +// EN: Version routes +// VI: Routes theo version +router.use('/v1', v1Router); +router.use('/v2', v2Router); +``` + +### Deprecation / Deprecation + +```typescript +// EN: Deprecation headers +// VI: Headers deprecation +res.setHeader('Deprecation', 'true'); +res.setHeader('Sunset', '2024-12-31'); +``` + +## Best Practices / Thực Hành Tốt + +1. Choose strategy and be consistent / Chọn chiến lược và nhất quán +2. Use semantic versioning / Sử dụng semantic versioning +3. Always deprecate before removing / Luôn deprecate trước khi xóa + +## Tài Nguyên + +- Skill Source: `.cursor/skills/api-versioning-strategy/SKILL.md` diff --git a/docs/vi/skills/cicd-advanced-patterns.md b/docs/vi/skills/cicd-advanced-patterns.md new file mode 100644 index 00000000..d613e9ba --- /dev/null +++ b/docs/vi/skills/cicd-advanced-patterns.md @@ -0,0 +1,48 @@ +# CI/CD Patterns Nâng Cao (CI/CD Advanced Patterns) + +Advanced CI/CD patterns for GoodGo microservices including blue-green deployments, canary releases, automated rollback, deployment verification, and progressive delivery. +> Các patterns CI/CD nâng cao cho GoodGo microservices bao gồm blue-green deployments, canary releases, automated rollback, deployment verification, và progressive delivery. + +## Tổng Quan + +Advanced CI/CD patterns enable safe, zero-downtime deployments with blue-green, canary releases, automated rollbacks, and deployment verification. + +Các patterns CI/CD nâng cao cho phép deployments an toàn, zero-downtime với blue-green, canary releases, automated rollbacks, và deployment verification. + +## Khi Nào Sử Dụng + +Use this skill when implementing advanced deployment strategies, automated rollbacks, or progressive delivery. + +Sử dụng skill này khi implement các chiến lược deployment nâng cao, automated rollbacks, hoặc progressive delivery. + +## Các Patterns Chính + +### Blue-Green Deployment / Blue-Green Deployment + +```yaml +# EN: Switch between blue and green environments +# VI: Chuyển đổi giữa blue và green environments +spec: + selector: + version: blue # EN: Switch to green / VI: Chuyển sang green +``` + +### Canary Deployment / Canary Deployment + +```yaml +# EN: Gradual rollout to subset of users +# VI: Rollout dần dần tới subset users +route: + - destination: { subset: canary } + weight: 10 # EN: 10% traffic / VI: 10% traffic +``` + +## Best Practices / Thực Hành Tốt + +1. Use blue-green for zero-downtime / Sử dụng blue-green cho zero-downtime +2. Use canary for gradual rollouts / Sử dụng canary cho rollouts dần dần +3. Always have rollback plan / Luôn có kế hoạch rollback + +## Tài Nguyên + +- Skill Source: `.cursor/skills/cicd-advanced-patterns/SKILL.md` diff --git a/docs/vi/skills/configuration-management.md b/docs/vi/skills/configuration-management.md new file mode 100644 index 00000000..fc9147ce --- /dev/null +++ b/docs/vi/skills/configuration-management.md @@ -0,0 +1,45 @@ +# Quản Lý Cấu Hình (Configuration Management) + +Configuration management patterns for GoodGo microservices including feature flags, dynamic configuration reloading, environment-specific configurations, and secrets management. +> Các patterns quản lý cấu hình cho GoodGo microservices bao gồm feature flags, dynamic configuration reloading, environment-specific configurations, và secrets management. + +## Tổng Quan + +Configuration management patterns enable flexible, secure, and environment-aware configuration handling with feature flags, dynamic reloading, and secrets management. + +Các patterns quản lý cấu hình cho phép xử lý cấu hình linh hoạt, an toàn và nhận biết environment với feature flags, dynamic reloading, và quản lý secrets. + +## Khi Nào Sử Dụng + +Use this skill when implementing feature flags, managing configurations, or handling secrets. + +Sử dụng skill này khi implement feature flags, quản lý configurations, hoặc xử lý secrets. + +## Các Patterns Chính + +### Feature Flags / Feature Flags + +```typescript +// EN: Check if feature is enabled +// VI: Kiểm tra xem feature có được bật không +const enabled = await featureFlagService.isEnabled('new-feature', userId); +``` + +### Dynamic Configuration / Configuration Động + +```typescript +// EN: Auto-reload configuration +// VI: Tự động tải lại configuration +await configService.load(); +configService.startAutoReload(60000); +``` + +## Best Practices / Thực Hành Tốt + +1. Always validate configuration / Luôn validate configuration +2. Never commit secrets / Không bao giờ commit secrets +3. Use feature flags for rollouts / Sử dụng feature flags cho rollouts + +## Tài Nguyên + +- Skill Source: `.cursor/skills/configuration-management/SKILL.md` diff --git a/docs/vi/skills/data-consistency-patterns.md b/docs/vi/skills/data-consistency-patterns.md new file mode 100644 index 00000000..747a500e --- /dev/null +++ b/docs/vi/skills/data-consistency-patterns.md @@ -0,0 +1,84 @@ +# Patterns Nhất Quán Dữ Liệu (Data Consistency Patterns) + +Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services. +> Các patterns nhất quán dữ liệu cho distributed microservices bao gồm Saga patterns, distributed transactions, eventual consistency, compensation, và idempotency. Sử dụng khi xử lý distributed transactions, implement eventual consistency, hoặc quản lý đồng bộ dữ liệu giữa các services. + +## Tổng Quan + +Data consistency in distributed systems requires different approaches than traditional ACID transactions. This skill covers Saga patterns for distributed transactions, idempotency for retries, optimistic locking for conflicts, and eventual consistency strategies. + +Nhất quán dữ liệu trong hệ thống phân tán yêu cầu các cách tiếp cận khác với ACID transactions truyền thống. Skill này bao gồm Saga patterns cho distributed transactions, idempotency cho retries, optimistic locking cho conflicts, và các chiến lược eventual consistency. + +## Khi Nào Sử Dụng + +Use this skill when: +- Implementing distributed transactions across multiple services +- Handling eventual consistency in microservices +- Implementing Saga patterns for distributed workflows +- Designing compensation strategies + +Sử dụng skill này khi: +- Implement distributed transactions qua nhiều services +- Xử lý eventual consistency trong microservices +- Implement Saga patterns cho distributed workflows +- Thiết kế chiến lược compensation + +## Khái Niệm Chính + +### ACID vs BASE + +**ACID (Truyền Thống)**: Atomicity, Consistency, Isolation, Durability + +**BASE (Phân Tán)**: Basic Availability, Soft state, Eventual consistency + +### Các Mô Hình Nhất Quán + +- **Strong Consistency**: Tất cả nodes thấy cùng dữ liệu cùng lúc +- **Eventual Consistency**: Hệ thống trở nên nhất quán theo thời gian +- **Weak Consistency**: Không đảm bảo về thời điểm nhất quán + +## Các Patterns Chính + +### Saga Orchestrator Pattern + +```typescript +// EN: Centralized orchestrator coordinates steps +// VI: Orchestrator tập trung điều phối các bước +const saga = new SagaOrchestrator(); +await saga.execute({ + sagaId: 'saga_123', + steps: [ + { name: 'create-order', execute: createOrder, compensate: cancelOrder }, + { name: 'reserve-inventory', execute: reserveInventory, compensate: releaseInventory }, + ], +}); +``` + +### Idempotency / Idempotency + +```typescript +// EN: Execute with idempotency check +// VI: Thực thi với idempotency check +await idempotencyHandler.execute( + idempotencyKey, + async () => await userService.create(data) +); +``` + +## Best Practices / Thực Hành Tốt + +1. **Design Compensations**: Mỗi step cần compensation / Every step needs compensation +2. **Idempotent Steps**: Làm cho steps idempotent cho retries / Make steps idempotent for retries +3. **Conflict Resolution**: Định nghĩa chiến lược giải quyết / Define resolution strategies +4. **Monitoring**: Theo dõi saga execution / Track saga execution + +## Skills Liên Quan + +- [Event-Driven Architecture](./event-driven-architecture.md) - Event patterns / Các patterns event +- [Error Handling Patterns](./error-handling-patterns.md) - Error handling / Xử lý lỗi +- [Database & Prisma](./database-prisma.md) - Database patterns / Các patterns database + +## Tài Nguyên + +- [Saga Pattern](https://microservices.io/patterns/data/saga.html) - Saga pattern overview +- Skill Source: `.cursor/skills/data-consistency-patterns/SKILL.md` - Source file đầy đủ diff --git a/docs/vi/skills/event-driven-architecture.md b/docs/vi/skills/event-driven-architecture.md new file mode 100644 index 00000000..8193a90d --- /dev/null +++ b/docs/vi/skills/event-driven-architecture.md @@ -0,0 +1,381 @@ +# Kiến Trúc Hướng Sự Kiện (Event-Driven Architecture) + +Event-driven architecture patterns with Apache Kafka for GoodGo microservices. Use when implementing async communication, event publishing/consuming, event sourcing, CQRS, or integrating event streams with HTTP endpoints. +> Các patterns kiến trúc hướng sự kiện với Apache Kafka cho GoodGo microservices. Sử dụng khi implement giao tiếp bất đồng bộ, publish/consume events, event sourcing, CQRS, hoặc tích hợp event streams với HTTP endpoints. + +## Tổng Quan + +Event-Driven Architecture (EDA) enables asynchronous communication between services using Apache Kafka as the message broker. This pattern decouples services, improves scalability, and enables event sourcing, CQRS, and reactive systems. GoodGo platform uses Kafka for high-throughput event streaming and integrates with Traefik to expose events via SSE/WebSocket endpoints. + +Kiến trúc hướng sự kiện (EDA) cho phép giao tiếp bất đồng bộ giữa các services sử dụng Apache Kafka làm message broker. Pattern này tách biệt các services, cải thiện khả năng mở rộng, và cho phép event sourcing, CQRS, và các hệ thống reactive. Nền tảng GoodGo sử dụng Kafka cho event streaming hiệu suất cao và tích hợp với Traefik để expose events qua SSE/WebSocket endpoints. + +## Khi Nào Sử Dụng + +Use this skill when: +- Implementing asynchronous communication between services +- Decoupling services for better scalability +- Publishing domain events for downstream consumers +- Consuming events from other services +- Implementing event sourcing patterns +- Implementing CQRS (Command Query Responsibility Segregation) +- Exposing event streams via HTTP (SSE/WebSocket) +- Handling eventual consistency across services +- Building reactive systems that respond to changes +- Integrating with Apache Kafka message broker + +Sử dụng skill này khi: +- Implement giao tiếp bất đồng bộ giữa các services +- Tách biệt services để cải thiện khả năng mở rộng +- Publish domain events cho downstream consumers +- Consume events từ các services khác +- Implement event sourcing patterns +- Implement CQRS (Command Query Responsibility Segregation) +- Expose event streams qua HTTP (SSE/WebSocket) +- Xử lý eventual consistency giữa các services +- Xây dựng reactive systems phản hồi với các thay đổi +- Tích hợp với Apache Kafka message broker + +## Khái Niệm Chính + +### Event-Driven vs Request-Response + +**Request-Response (Synchronous / Đồng Bộ):** +- Client waits for response / Client đợi response +- Tight coupling between services / Liên kết chặt chẽ giữa các services +- Blocking operations / Các thao tác blocking +- Immediate consistency / Nhất quán ngay lập tức +- Use Traefik API Gateway for HTTP/REST / Sử dụng Traefik API Gateway cho HTTP/REST + +**Event-Driven (Asynchronous / Bất Đồng Bộ):** +- Fire-and-forget publishing / Publish fire-and-forget +- Loose coupling between services / Liên kết lỏng lẻo giữa các services +- Non-blocking operations / Các thao tác non-blocking +- Eventual consistency / Nhất quán cuối cùng +- Use Kafka for message broker / Sử dụng Kafka cho message broker + +### Kafka Fundamentals / Các Khái Niệm Cơ Bản về Kafka + +- **Topics**: Named streams of events (e.g., `user.created`, `order.placed`) / Các luồng sự kiện được đặt tên +- **Partitions**: Physical division of topics for parallelism and scaling / Chia nhỏ vật lý của topics để song song hóa và mở rộng +- **Consumer Groups**: Groups of consumers that work together to process events / Các nhóm consumers làm việc cùng nhau để xử lý events +- **Producers**: Services that publish events to topics / Services phát hành events tới topics +- **Consumers**: Services that subscribe to topics and process events / Services đăng ký topics và xử lý events + +### Traefik Integration / Tích Hợp Traefik + +Traefik serves dual purpose: +- **API Gateway**: Routes synchronous HTTP/REST requests / Định tuyến các request HTTP/REST đồng bộ +- **Event Streaming Gateway**: Routes SSE/WebSocket connections to event streaming endpoints / Định tuyến các kết nối SSE/WebSocket tới event streaming endpoints + +Services publish events to Kafka, then expose SSE/WebSocket endpoints that consume from Kafka for HTTP clients. + +Services publish events vào Kafka, sau đó expose SSE/WebSocket endpoints consume từ Kafka cho HTTP clients. + +## Các Patterns Chính + +### Event Publishing / Phát Hành Events + +```typescript +// src/core/events/event-publisher.ts +import { producer } from '../config/kafka.config'; +import { logger } from '@goodgo/logger'; +import { v4 as uuidv4 } from 'uuid'; + +export class EventPublisher { + /** + * EN: Publish event to Kafka topic + * VI: Phát hành event tới Kafka topic + */ + async publish( + topic: string, + event: Omit, + options?: { partitionKey?: string } + ): Promise { + const fullEvent: T = { + ...event, + eventId: uuidv4(), + timestamp: new Date().toISOString(), + source: this.serviceName, + } as T; + + await producer.send({ + topic, + messages: [{ + key: options?.partitionKey || fullEvent.eventId, + value: JSON.stringify(fullEvent), + headers: { + 'event-type': event.eventType, + 'event-version': event.eventVersion, + }, + }], + }); + } +} +``` + +**Tham Khảo**: `.cursor/skills/event-driven-architecture/SKILL.md` + +### Event Consuming / Tiêu Thụ Events + +```typescript +// src/core/events/event-consumer.ts +import { kafka } from '../config/kafka.config'; + +export class EventConsumer { + private handlers: Map = new Map(); + + /** + * EN: Register event handler + * VI: Đăng ký event handler + */ + on(eventType: string, handler: EventHandler): void { + if (!this.handlers.has(eventType)) { + this.handlers.set(eventType, []); + } + this.handlers.get(eventType)!.push(handler); + } + + /** + * EN: Start consuming from topics + * VI: Bắt đầu consume từ topics + */ + async start(topics: string[]): Promise { + await this.consumer.connect(); + await this.consumer.subscribe({ topics, fromBeginning: false }); + + await this.consumer.run({ + eachMessage: async ({ topic, partition, message }) => { + const event: BaseEvent = JSON.parse(message.value?.toString() || '{}'); + const handlers = this.handlers.get(event.eventType) || []; + await Promise.all(handlers.map(h => h.handle(event))); + }, + }); + } +} +``` + +### Outbox Pattern / Pattern Outbox + +The Outbox pattern ensures transactional event publishing by storing events in the database within the same transaction as the business data. + +Pattern Outbox đảm bảo publish events trong transaction bằng cách lưu events trong database trong cùng transaction với business data. + +```typescript +// EN: Store event in database within transaction +// VI: Lưu event vào database trong transaction +await prisma.outboxEvent.create({ + data: { + eventType: 'user.created', + eventData: userData, + topic: 'user.created', + status: 'PENDING', + }, +}); + +// EN: Separate process publishes from outbox to Kafka +// VI: Process riêng publish từ outbox tới Kafka +async function processOutbox() { + const events = await prisma.outboxEvent.findMany({ + where: { status: 'PENDING' }, + }); + + for (const event of events) { + await eventPublisher.publish(event.topic, event.eventData); + await prisma.outboxEvent.update({ + where: { id: event.id }, + data: { status: 'PUBLISHED' }, + }); + } +} +``` + +### SSE Endpoint / Endpoint SSE + +Server-Sent Events (SSE) allows clients to receive event streams via HTTP. + +Server-Sent Events (SSE) cho phép clients nhận event streams qua HTTP. + +```typescript +// src/modules/events/events.controller.ts +async streamEvents(req: Request, res: Response): Promise { + res.setHeader('Content-Type', 'text/event-stream'); + res.setHeader('Cache-Control', 'no-cache'); + res.setHeader('Connection', 'keep-alive'); + + const topic = req.query.topic as string; + const consumer = kafka.consumer({ groupId: `sse-${Date.now()}` }); + + await consumer.connect(); + await consumer.subscribe({ topic, fromBeginning: false }); + + await consumer.run({ + eachMessage: async ({ message }) => { + const event = JSON.parse(message.value?.toString() || '{}'); + res.write(`data: ${JSON.stringify(event)}\n\n`); + }, + }); + + req.on('close', async () => { + await consumer.disconnect(); + }); +} +``` + +## Cấu Trúc Event / Event Structure + +```typescript +interface BaseEvent { + eventId: string; // EN: Unique event identifier / VI: Định danh event duy nhất + eventType: string; // EN: Event type (e.g., "user.created") / VI: Loại event + eventVersion: string; // EN: Schema version / VI: Phiên bản schema + timestamp: string; // EN: ISO 8601 timestamp / VI: Timestamp ISO 8601 + source: string; // EN: Service that published the event / VI: Service phát hành event + correlationId?: string; // EN: Request correlation ID / VI: Correlation ID của request + traceId?: string; // EN: Distributed tracing ID / VI: ID phân tán tracing + data: unknown; // EN: Event payload / VI: Payload của event +} +``` + +## Best Practices / Thực Hành Tốt + +### Event Naming Conventions / Quy Ước Đặt Tên Events + +- **Event Type**: `{domain}.{action}.v{version}` (e.g., `user.created.v1`) / Loại Event: `{domain}.{action}.v{version}` +- **Topic**: `{domain}.{entity}.{action}` (e.g., `user.created`) / Topic: `{domain}.{entity}.{action}` +- Use lowercase with dots as separators / Sử dụng chữ thường với dấu chấm làm phân cách +- Keep names descriptive and consistent / Giữ tên mô tả và nhất quán + +### Partition Key Selection / Lựa Chọn Partition Key + +- Use entity ID for ordering guarantees (same entity → same partition) / Sử dụng entity ID để đảm bảo thứ tự +- Use correlation ID for request tracing / Sử dụng correlation ID để trace request +- Use user ID for user-scoped events / Sử dụng user ID cho events phạm vi user +- Avoid high-cardinality keys (distributes evenly) / Tránh keys có độ phân tán cao + +### Event Ordering Guarantees / Đảm Bảo Thứ Tự Events + +- Kafka guarantees ordering **per partition** / Kafka đảm bảo thứ tự **theo partition** +- Use partition key to ensure related events go to same partition / Sử dụng partition key để đảm bảo events liên quan cùng partition +- Events in different partitions have no ordering guarantee / Events ở các partitions khác nhau không có đảm bảo thứ tự +- Don't rely on global ordering across all events / Không phụ thuộc vào thứ tự toàn cục + +### Error Handling / Xử Lý Lỗi + +- Implement Dead Letter Queue (DLQ) for failed events / Implement DLQ cho events failed +- Use retry with exponential backoff / Sử dụng retry với exponential backoff +- Log all event processing failures / Ghi log tất cả lỗi xử lý events +- Monitor consumer lag and DLQ size / Giám sát consumer lag và kích thước DLQ + +### Observability / Khả Năng Quan Sát + +- Log all published and consumed events / Ghi log tất cả events đã publish và consume +- Track metrics: events published/consumed, processing duration, consumer lag / Theo dõi metrics: events published/consumed, thời gian xử lý, consumer lag +- Add distributed tracing to event flows / Thêm distributed tracing vào event flows +- Include correlation IDs for request tracking / Bao gồm correlation IDs để theo dõi request + +## Infrastructure Setup / Thiết Lập Hạ Tầng + +### Docker Compose (Local / Cục Bộ) + +```yaml +services: + kafka: + image: confluentinc/cp-kafka:7.4.0 + ports: + - "9092:9092" + environment: + KAFKA_BROKER_ID: 1 + KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 + + schema-registry: + image: confluentinc/cp-schema-registry:7.4.0 + ports: + - "8081:8081" + environment: + SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:9092 +``` + +**Tham Khảo**: `deployments/local/docker-compose.yml` + +## Testing / Kiểm Thử + +### Unit Testing / Kiểm Thử Đơn Vị + +```typescript +import { EventPublisher } from '../event-publisher'; +import { producer } from '../../config/kafka.config'; + +jest.mock('../../config/kafka.config'); + +describe('EventPublisher', () => { + it('should publish event successfully', async () => { + const publisher = new EventPublisher(); + const mockSend = jest.fn().mockResolvedValue({}); + (producer.send as jest.Mock) = mockSend; + + await publisher.publish('user.created', { + eventType: 'user.created', + eventVersion: '1.0.0', + data: { userId: '123' }, + }); + + expect(mockSend).toHaveBeenCalled(); + }); +}); +``` + +### Integration Testing / Kiểm Thử Tích Hợp + +Use Kafka test containers for integration testing: + +Sử dụng Kafka test containers cho integration testing: + +```typescript +import { KafkaContainer } from '@testcontainers/kafka'; + +describe('Event Flow E2E', () => { + let kafkaContainer: StartedKafkaContainer; + + beforeAll(async () => { + kafkaContainer = await new KafkaContainer().start(); + process.env.KAFKA_BROKERS = kafkaContainer.getBootstrapServer(); + }); + + it('should publish and consume event', async () => { + // EN: Test implementation + // VI: Implementation test + }); +}); +``` + +## Use Cases / Các Trường Hợp Sử Dụng + +### User Created Event Flow / Luồng User Created Event + +1. Auth Service creates user in database / Auth Service tạo user trong database +2. Publishes `user.created` event to Kafka / Publish event `user.created` tới Kafka +3. Notification Service consumes event and sends welcome email / Notification Service consume event và gửi email chào mừng +4. Analytics Service consumes event and updates metrics / Analytics Service consume event và cập nhật metrics + +### Order Processing with Multiple Consumers / Xử Lý Order với Nhiều Consumers + +1. Order Service publishes `order.placed` event / Order Service publish event `order.placed` +2. Payment Service processes payment / Payment Service xử lý thanh toán +3. Inventory Service reserves items / Inventory Service dự trữ items +4. Notification Service sends confirmation / Notification Service gửi xác nhận + +## Skills Liên Quan + +- [Resilience Patterns](./resilience-patterns.md) - Circuit breaker, retry patterns / Circuit breaker, các patterns retry +- [Error Handling Patterns](./error-handling-patterns.md) - Error handling best practices / Best practices về error handling +- [Observability & Monitoring](./observability-monitoring.md) - Logging, metrics, tracing / Logging, metrics, tracing +- [Middleware Patterns](./middleware-patterns.md) - SSE endpoint middleware / Middleware SSE endpoint +- [Project Rules](./project-rules.md) - GoodGo coding standards / Tiêu chuẩn coding GoodGo + +## Tài Nguyên + +- [KafkaJS Documentation](https://kafka.js.org/) - Node.js Kafka client +- [Confluent Schema Registry](https://docs.confluent.io/platform/current/schema-registry/index.html) - Schema versioning +- [Kafka Best Practices](https://kafka.apache.org/documentation/#best_practices) - Official Kafka documentation / Tài liệu chính thức Kafka +- Skill Source: `.cursor/skills/event-driven-architecture/SKILL.md` - Source file đầy đủ / Full source file diff --git a/docs/vi/skills/infrastructure-as-code.md b/docs/vi/skills/infrastructure-as-code.md new file mode 100644 index 00000000..e40045e1 --- /dev/null +++ b/docs/vi/skills/infrastructure-as-code.md @@ -0,0 +1,50 @@ +# Infrastructure as Code + +Infrastructure as Code patterns for GoodGo platform including Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, and multi-environment management. +> Các patterns Infrastructure as Code cho nền tảng GoodGo bao gồm Terraform modules, Kubernetes operators, infrastructure testing, GitOps workflows, và multi-environment management. + +## Tổng Quan + +Infrastructure as Code enables managing infrastructure through code, providing version control, reproducibility, and automation. + +Infrastructure as Code cho phép quản lý infrastructure qua code, cung cấp version control, reproducibility, và automation. + +## Khi Nào Sử Dụng + +Use this skill when managing infrastructure, implementing GitOps, or creating reusable modules. + +Sử dụng skill này khi quản lý infrastructure, implement GitOps, hoặc tạo các modules tái sử dụng. + +## Các Patterns Chính + +### Terraform Modules / Terraform Modules + +```hcl +# EN: Reusable module +# VI: Module tái sử dụng +module "postgresql" { + source = "../../modules/postgresql" + database_name = "goodgo" +} +``` + +### GitOps / GitOps + +```yaml +# EN: Automated sync from Git +# VI: Đồng bộ tự động từ Git +spec: + syncPolicy: + automated: + prune: true +``` + +## Best Practices / Thực Hành Tốt + +1. Keep infrastructure in version control / Giữ infrastructure trong version control +2. Create reusable modules / Tạo modules tái sử dụng +3. Test before applying / Test trước khi apply + +## Tài Nguyên + +- Skill Source: `.cursor/skills/infrastructure-as-code/SKILL.md` diff --git a/docs/vi/skills/inter-service-communication.md b/docs/vi/skills/inter-service-communication.md new file mode 100644 index 00000000..7d779689 --- /dev/null +++ b/docs/vi/skills/inter-service-communication.md @@ -0,0 +1,196 @@ +# Giao Tiếp Giữa Các Services (Inter-Service Communication) + +Inter-service communication patterns for GoodGo microservices including gRPC, GraphQL, service-to-service authentication, protocol selection, and client patterns. Use when implementing service-to-service calls, choosing communication protocols, or building service clients. +> Các patterns giao tiếp giữa services cho GoodGo microservices bao gồm gRPC, GraphQL, service-to-service authentication, lựa chọn protocol, và client patterns. Sử dụng khi implement service-to-service calls, chọn communication protocols, hoặc xây dựng service clients. + +## Tổng Quan + +Inter-service communication enables services to work together in a microservices architecture. GoodGo supports multiple protocols: HTTP/REST (current standard), gRPC (for high performance), and GraphQL (for flexible queries). This skill covers patterns for implementing resilient, secure, and performant inter-service communication. + +Giao tiếp giữa các services cho phép các services làm việc cùng nhau trong kiến trúc microservices. GoodGo hỗ trợ nhiều protocols: HTTP/REST (chuẩn hiện tại), gRPC (cho hiệu suất cao), và GraphQL (cho queries linh hoạt). Skill này bao gồm các patterns để implement giao tiếp giữa services có khả năng phục hồi, bảo mật và hiệu suất cao. + +## Khi Nào Sử Dụng + +Use this skill when: +- Implementing service-to-service communication +- Choosing between REST, gRPC, or GraphQL protocols +- Setting up gRPC services and clients +- Implementing GraphQL services and resolvers +- Implementing service-to-service authentication +- Building resilient service clients with circuit breakers + +Sử dụng skill này khi: +- Implement giao tiếp giữa các services +- Chọn giữa REST, gRPC, hoặc GraphQL protocols +- Setup gRPC services và clients +- Implement GraphQL services và resolvers +- Implement service-to-service authentication +- Xây dựng service clients có khả năng phục hồi với circuit breakers + +## Khái Niệm Chính + +### Các Lựa Chọn Protocol + +**HTTP/REST:** +- Dễ đọc, dễ debug / Human-readable, easy to debug +- Tương thích browser / Browser-compatible +- Semantics HTTP chuẩn / Standard HTTP semantics +- JSON payloads +- Tốt cho external APIs / Good for external APIs + +**gRPC:** +- Binary protocol (Protocol Buffers) +- Hiệu suất cao, độ trễ thấp / High performance, low latency +- Hỗ trợ streaming / Streaming support +- Strong typing với .proto files +- Dựa trên HTTP/2 / HTTP/2 based + +**GraphQL:** +- Ngôn ngữ query linh hoạt / Flexible query language +- Single endpoint +- Client kiểm soát data fetching / Client-controlled data fetching +- Strong typing với schema + +### Hướng Dẫn Lựa Chọn Protocol + +Chọn protocol dựa trên use case, yêu cầu hiệu suất, chuyên môn team, và nhu cầu ecosystem. + +Choose protocol based on use case, performance requirements, team expertise, and ecosystem needs. + +## Các Patterns Chính + +### HTTP/REST Service Client + +```typescript +// EN: Service client với circuit breaker +// VI: Service client với circuit breaker +import { ServiceClient } from '../../core/clients/service-client'; + +const notificationClient = new ServiceClient({ + baseURL: process.env.NOTIFICATION_SERVICE_URL || 'http://notification-service:5003', + serviceName: 'notification-service', + timeout: 5000, + enableCircuitBreaker: true, +}); + +// EN: Usage +// VI: Sử dụng +await notificationClient.post('/api/v1/notifications', { + userId, + message, +}); +``` + +### gRPC Service + +```typescript +// EN: gRPC server +// VI: gRPC server +import { UserGrpcServer } from './user.grpc.service'; + +const grpcServer = new UserGrpcServer(userService); +grpcServer.start(50051); + +// EN: gRPC client +// VI: gRPC client +const userGrpcClient = new GrpcClient({ + protoPath: './proto/user_service.proto', + packageName: 'goodgo.user.v1', + serviceName: 'UserService', + serverUrl: 'localhost:50051', +}); + +const user = await userGrpcClient.call('getUser', { user_id: '123' }); +``` + +### GraphQL Service + +```typescript +// EN: GraphQL client +// VI: GraphQL client +const userGraphQLClient = new GraphQLServiceClient({ + endpoint: 'http://user-service:5002/graphql', +}); + +const GET_USER_QUERY = ` + query GetUser($id: ID!) { + user(id: $id) { + id + email + name + } + } +`; + +const user = await userGraphQLClient.query(GET_USER_QUERY, { id: '123' }); +``` + +### Service-to-Service Authentication + +```typescript +// EN: Internal auth middleware +// VI: Internal auth middleware +import { internalAuthMiddleware } from '../../middlewares/internal-auth.middleware'; + +router.use('/internal', internalAuthMiddleware); + +// EN: Client tự động thêm auth header +// VI: Client automatically adds auth header +const client = new ServiceClient({ + baseURL: 'http://service:5000', + serviceName: 'service', +}); +// X-Service-Auth header được thêm tự động +``` + +## Best Practices / Thực Hành Tốt + +### Protocol Selection / Lựa Chọn Protocol + +- **REST**: External APIs, browser clients, simple CRUD / APIs ngoài, browser clients, CRUD đơn giản +- **gRPC**: Internal services, high performance, streaming / Services nội bộ, hiệu suất cao, streaming +- **GraphQL**: Complex queries, mobile apps, flexible data / Queries phức tạp, mobile apps, data linh hoạt + +### Performance / Hiệu Suất + +- Use connection pooling / Sử dụng connection pooling +- Enable HTTP keep-alive / Bật HTTP keep-alive +- Set appropriate timeouts / Thiết lập timeout phù hợp +- Implement circuit breakers / Implement circuit breakers + +### Security / Bảo Mật + +- Always authenticate internal calls / Luôn xác thực internal calls +- Use TLS/mTLS / Sử dụng TLS/mTLS +- Store secrets securely / Lưu secrets an toàn +- Implement rate limiting / Implement rate limiting + +### Observability / Khả Năng Quan Sát + +- Log with correlation IDs / Ghi log với correlation IDs +- Track metrics (duration, success rate) / Theo dõi metrics +- Add distributed tracing / Thêm distributed tracing +- Monitor service health / Giám sát sức khỏe service + +## Testing / Kiểm Thử + +```typescript +// EN: Mock service client +// VI: Mock service client +const mockClient = createMockServiceClient(); +mockClient.get.mockResolvedValue({ id: '123' }); +``` + +## Skills Liên Quan + +- [Resilience Patterns](./resilience-patterns.md) - Circuit breaker, retry patterns / Circuit breaker, các patterns retry +- [Security](./security.md) - Authentication, authorization / Authentication, authorization +- [API Design](./api-design.md) - RESTful API patterns / RESTful API patterns +- [Project Rules](./project-rules.md) - GoodGo coding standards / Tiêu chuẩn coding GoodGo + +## Tài Nguyên + +- [gRPC Documentation](https://grpc.io/docs/) - Tài liệu gRPC chính thức +- [GraphQL Documentation](https://graphql.org/learn/) - Tài liệu GraphQL +- [Protocol Buffers](https://developers.google.com/protocol-buffers) - Hướng dẫn Protocol Buffers +- Skill Source: `.cursor/skills/inter-service-communication/SKILL.md` - Source file đầy đủ diff --git a/docs/vi/skills/performance-optimization.md b/docs/vi/skills/performance-optimization.md new file mode 100644 index 00000000..79f0359c --- /dev/null +++ b/docs/vi/skills/performance-optimization.md @@ -0,0 +1,52 @@ +# Tối Ưu Hiệu Suất (Performance Optimization) + +Performance optimization patterns for GoodGo microservices including database query optimization, memory leak detection, profiling, connection pooling, and caching strategies. +> Các patterns tối ưu hiệu suất cho GoodGo microservices bao gồm tối ưu database queries, phát hiện memory leaks, profiling, connection pooling, và caching strategies. + +## Tổng Quan + +Performance optimization patterns help identify and fix performance bottlenecks, optimize database queries, detect memory leaks, and improve overall application performance. + +Các patterns tối ưu hiệu suất giúp xác định và sửa các nút cổ chai hiệu suất, tối ưu database queries, phát hiện memory leaks, và cải thiện hiệu suất ứng dụng tổng thể. + +## Khi Nào Sử Dụng + +Use this skill when optimizing performance, profiling applications, or detecting bottlenecks. + +Sử dụng skill này khi tối ưu hiệu suất, profiling ứng dụng, hoặc phát hiện bottlenecks. + +## Các Patterns Chính + +### Database Query Optimization / Tối Ưu Database Queries + +```typescript +// EN: Avoid N+1 queries +// VI: Tránh N+1 queries +const users = await userRepository.findAll({ + include: { orders: true }, // EN: Single query / VI: Single query +}); +``` + +### Memory Profiling / Memory Profiling + +```typescript +// EN: Monitor memory usage +// VI: Giám sát memory usage +const profiler = new MemoryProfiler(); +profiler.start(); +``` + +## Best Practices / Thực Hành Tốt + +1. Use indexes, avoid N+1 queries / Sử dụng indexes, tránh N+1 queries +2. Monitor memory usage / Giám sát memory usage +3. Cache frequently accessed data / Cache dữ liệu thường truy cập + +## Skills Liên Quan + +- [Caching Patterns](./caching-patterns.md) - Caching strategies / Chiến lược caching +- [Observability & Monitoring](./observability-monitoring.md) - Monitoring / Giám sát + +## Tài Nguyên + +- Skill Source: `.cursor/skills/performance-optimization/SKILL.md` diff --git a/docs/vi/skills/service-discovery-registry.md b/docs/vi/skills/service-discovery-registry.md new file mode 100644 index 00000000..9b2ccd54 --- /dev/null +++ b/docs/vi/skills/service-discovery-registry.md @@ -0,0 +1,45 @@ +# Service Discovery & Registry + +Service discovery and registry patterns for GoodGo microservices including service registry, health check orchestration, load balancing strategies, and service mesh integration. +> Các patterns service discovery và registry cho GoodGo microservices bao gồm service registry, health check orchestration, load balancing strategies, và tích hợp service mesh. + +## Tổng Quan + +Service discovery and registry patterns enable services to find and communicate with each other dynamically, with health monitoring and load balancing. + +Các patterns service discovery và registry cho phép services tìm và giao tiếp với nhau một cách động, với giám sát health và load balancing. + +## Khi Nào Sử Dụng + +Use this skill when implementing service discovery, managing service registry, or orchestrating health checks. + +Sử dụng skill này khi implement service discovery, quản lý service registry, hoặc điều phối health checks. + +## Các Patterns Chính + +### Kubernetes DNS Discovery / Service Discovery DNS Kubernetes + +```typescript +// EN: Use Kubernetes DNS +// VI: Sử dụng Kubernetes DNS +const serviceUrl = `http://user-service.production.svc.cluster.local`; +``` + +### Service Registry / Service Registry + +```typescript +// EN: Register and discover services +// VI: Đăng ký và tìm services +await serviceRegistry.register(serviceInfo); +const service = await serviceRegistry.discover('user-service'); +``` + +## Best Practices / Thực Hành Tốt + +1. Use Kubernetes DNS / Sử dụng Kubernetes DNS +2. Implement health checks / Implement health checks +3. Use service registry / Sử dụng service registry + +## Tài Nguyên + +- Skill Source: `.cursor/skills/service-discovery-registry/SKILL.md`