pos-system/docs/en/skills/microservices-development-process.md

---
name: microservices-development-process
description: Standard development process for creating and maintaining microservices in GoodGo platform. Use when creating new services, migrating services, refactoring services, or planning service implementations.
---

# Microservices Development Process

## When to Use This Skill

Use this skill when:
- Creating a new microservice from scratch
- Migrating or refactoring an existing service
- Planning service implementation with multiple phases
- Ensuring comprehensive coverage of all development aspects
- Need structured approach to service development

## Development Process Overview

The microservices development process follows these phases:
1. **Planning & Impact Analysis** - Define scope, impact, dependencies
2. **Foundation Setup** - Service structure, configs, infrastructure
3. **Core Implementation** - Business logic, APIs, data layer
4. **Integration** - Routes, middleware, external services
5. **Testing** - Unit, integration, E2E tests
6. **Documentation** - API docs, README, guides
7. **Cleanup & Verification** - Remove temporary files, verify completeness
8. **Deployment** - Staging deployment, production deployment

### Process Flow Diagram

This diagram shows the complete 8-phase development process with decision points and feedback loops.

```mermaid
graph TD
    Start([Start: New Service Requirements]) --> Phase1[Phase 1: Planning & Impact Analysis]
    Phase1 --> ImpactCheck{Impact Analysis<br/>Complete?}
    ImpactCheck -->|No| Phase1
    ImpactCheck -->|Yes| Phase2[Phase 2: Foundation Setup]
    
    Phase2 --> FoundationCheck{Service Starts<br/>& Health Check Passes?}
    FoundationCheck -->|No| Phase2
    FoundationCheck -->|Yes| Phase3[Phase 3: Core Implementation]
    
    Phase3 --> ImplementationCheck{Business Logic<br/>Implemented?}
    ImplementationCheck -->|No| Phase3
    ImplementationCheck -->|Yes| Phase4[Phase 4: Integration]
    
    Phase4 --> IntegrationCheck{Routes & Middleware<br/>Working?}
    IntegrationCheck -->|No| Phase4
    IntegrationCheck -->|Yes| Phase5[Phase 5: Testing]
    
    Phase5 --> TestCheck{Tests Pass<br/>& Coverage Met?}
    TestCheck -->|No| Phase5
    TestCheck -->|Yes| Phase6[Phase 6: Documentation]
    
    Phase6 --> DocCheck{Docs<br/>Complete?}
    DocCheck -->|No| Phase6
    DocCheck -->|Yes| Phase7[Phase 7: Cleanup & Verification]
    
    Phase7 --> VerificationCheck{All Checks<br/>Pass?}
    VerificationCheck -->|No| Phase7
    VerificationCheck -->|Yes| Phase8[Phase 8: Deployment]
    
    Phase8 --> DeployCheck{Staging<br/>Deployed?}
    DeployCheck -->|No| Phase8
    DeployCheck -->|Yes| Production{Deploy to<br/>Production?}
    Production -->|Yes| ProdDeploy[Production Deployment]
    Production -->|No| Complete([Complete])
    ProdDeploy --> Complete
    
    style Phase1 fill:#e1f5ff
    style Phase2 fill:#fff4e1
    style Phase3 fill:#f0e1ff
    style Phase4 fill:#e1ffe1
    style Phase5 fill:#ffe1e1
    style Phase6 fill:#e1ffff
    style Phase7 fill:#fff0e1
    style Phase8 fill:#ffe1f5
    style Complete fill:#d4edda
```

### Detailed Phase Flow

This diagram breaks down the tasks within each phase and shows the sequential flow between phases.

```mermaid
graph LR
    subgraph Planning["Phase 1: Planning"]
        P1A[Define Scope] --> P1B[Impact Analysis]
        P1B --> P1C[Dependencies Map]
        P1C --> P1D[Acceptance Criteria]
    end
    
    subgraph Foundation["Phase 2: Foundation"]
        F2A[Copy Template] --> F2B[Configure Package]
        F2B --> F2C[Setup Database]
        F2C --> F2D[Configure Docker]
        F2D --> F2E[Setup Traefik]
    end
    
    subgraph Implementation["Phase 3: Implementation"]
        I3A[DTOs] --> I3B[Repository]
        I3B --> I3C[Service]
        I3C --> I3D[Controller]
        I3D --> I3E[Module]
    end
    
    subgraph Integration["Phase 4: Integration"]
        IN4A[Register Routes] --> IN4B[Setup Middleware]
        IN4B --> IN4C[External Services]
        IN4C --> IN4D[Health Checks]
    end
    
    subgraph Testing["Phase 5: Testing"]
        T5A[Unit Tests] --> T5B[Integration Tests]
        T5B --> T5C[E2E Tests]
        T5C --> T5D[Coverage Check]
    end
    
    subgraph Documentation["Phase 6: Documentation"]
        D6A[README] --> D6B[API Docs]
        D6B --> D6C[Architecture Docs]
    end
    
    subgraph Cleanup["Phase 7: Cleanup"]
        C7A[Remove Temp Files] --> C7B[Update References]
        C7B --> C7C[Verify Everything]
    end
    
    subgraph Deployment["Phase 8: Deployment"]
        DEP8A[Staging] --> DEP8B[Verification]
        DEP8B --> DEP8C[Production]
    end
    
    Planning --> Foundation
    Foundation --> Implementation
    Implementation --> Integration
    Integration --> Testing
    Testing --> Documentation
    Documentation --> Cleanup
    Cleanup --> Deployment
    
    style Planning fill:#e1f5ff
    style Foundation fill:#fff4e1
    style Implementation fill:#f0e1ff
    style Integration fill:#e1ffe1
    style Testing fill:#ffe1e1
    style Documentation fill:#e1ffff
    style Cleanup fill:#fff0e1
    style Deployment fill:#ffe1f5
```

## Phase 1: Planning & Impact Analysis

### Scope Definition

Define clearly:
- **Service Purpose**: What business capability does it provide?
- **API Surface**: What endpoints are needed?
- **Data Models**: What data structures are required?
- **Dependencies**: What services/packages does it depend on?
- **Breaking Changes**: Any backward compatibility concerns?

### Impact Analysis Checklist

Before starting implementation, identify all affected areas:

**Files to Create:**
- [ ] Service directory: `services/service-name/`
- [ ] Prisma schema: `services/service-name/prisma/schema.prisma`
- [ ] Dockerfile: `services/service-name/Dockerfile`
- [ ] Service README: `services/service-name/README.md`

**Files to Update:**
- [ ] Root `package.json` workspace config
- [ ] `deployments/local/docker-compose.yml` - Add service
- [ ] `infra/traefik/dynamic/routes.yml` - Add routes
- [ ] `.github/workflows/ci-*.yml` - Add CI workflow (if needed)
- [ ] Documentation: `docs/en/guides/`, `docs/vi/guides/`
- [ ] Scripts: `scripts/db/*.sh`, `scripts/dev/*.sh` (if service-specific)

**Infrastructure Changes:**
- [ ] Database: New schema/tables
- [ ] Redis: New cache keys/patterns (if needed)
- [ ] Traefik: New routes and services
- [ ] Observability: New service metrics/traces

**Dependencies:**
- [ ] External: Database, Redis, third-party APIs
- [ ] Internal: Shared packages (@goodgo/logger, @goodgo/types, etc.)
- [ ] Other Services: List dependent services

## Phase 2: Foundation Setup

### Service Structure Creation

**Template Usage:**
```bash
cp -r services/_template services/new-service-name
cd services/new-service-name
# Update package.json name to @goodgo/new-service-name
```

**Required Files:**
- Service structure from template
- `package.json` with correct name and dependencies
- `src/config/app.config.ts` - Configuration with Zod validation
- `.env.example` - Environment variables template
- `prisma/schema.prisma` - Database schema
- `Dockerfile` - Container configuration
- `jest.config.ts` - Test configuration

### Database Setup

```bash
# Create initial migration
cd services/service-name
pnpm prisma migrate dev --name init
pnpm prisma generate
```

### Docker & Infrastructure

**Docker Compose Integration:**
Add service to `deployments/local/docker-compose.yml` with:
- Build context and dockerfile
- Environment variables
- Traefik labels for routing
- Health check configuration

**Traefik Routes:**
Update `infra/traefik/dynamic/routes.yml` with:
- Router rules (PathPrefix)
- Service configuration
- Middleware chain (CORS, rate-limit, auth)

### Acceptance Criteria for Phase 2

- [ ] Service directory created from template
- [ ] `package.json` configured correctly
- [ ] Environment variables defined
- [ ] Prisma schema created and migration run
- [ ] Service starts: `pnpm dev` (health check passes)
- [ ] Docker build succeeds
- [ ] Service accessible via Traefik
- [ ] No TypeScript errors: `pnpm typecheck`

## Phase 3: Core Implementation

### Module Structure

Each feature module follows this pattern:
```
modules/feature-name/
├── feature.controller.ts    # HTTP handlers
├── feature.service.ts       # Business logic
├── feature.repository.ts    # Data access
├── feature.dto.ts          # Validation schemas (Zod)
├── feature.module.ts       # Module registration
└── index.ts                # Public exports
```

### Implementation Flow

This diagram shows the step-by-step implementation order for each feature module within Phase 3.

```mermaid
graph TD
    Start[Start Implementation] --> DTOs[1. Create DTOs<br/>Zod Validation Schemas]
    DTOs --> Repo[2. Create Repository<br/>Prisma Data Access]
    Repo --> Service[3. Create Service<br/>Business Logic]
    Service --> Controller[4. Create Controller<br/>HTTP Handlers]
    Controller --> Module[5. Create Module<br/>Wire Up Components]
    Module --> Test[Manual Testing]
    Test --> Pass{Tests Pass?}
    Pass -->|No| Repo
    Pass -->|Yes| Next[Next Feature Module]
    
    style DTOs fill:#e1f5ff
    style Repo fill:#fff4e1
    style Service fill:#f0e1ff
    style Controller fill:#e1ffe1
    style Module fill:#ffe1e1
```

### Implementation Order

1. **DTOs** - Zod schemas for request/response validation
2. **Repository** - Prisma-based data access, CRUD operations
3. **Service** - Business logic, error handling, validation
4. **Controller** - HTTP request handling, standardized responses
5. **Module** - Wire up components, export router

### Code Patterns

**Repository:** Extend base Repository, use Prisma client for data access
**Service:** Inject repository, implement business logic, use logger
**Controller:** Handle HTTP requests, validate with DTOs, call services
**Module:** Wire up dependencies, export router

### Acceptance Criteria for Phase 3

- [ ] All DTOs defined with Zod validation
- [ ] Repository methods implemented
- [ ] Service business logic implemented
- [ ] Controllers handle requests correctly
- [ ] Modules configured properly
- [ ] No TypeScript errors
- [ ] Manual API testing successful

## Phase 4: Integration

### Route Registration

Update `src/routes/index.ts`:
- Import feature modules
- Create router instances
- Register routes with path prefixes
- Mount to main app with `/api/v1/service-name` prefix

### Middleware Setup

**Required Middlewares (in order):**
1. Correlation middleware
2. Logging middleware
3. Metrics middleware
4. CORS middleware
5. Rate limiting middleware
6. Authentication middleware (if needed)
7. Error middleware (always last)

### External Service Integration

- HTTP clients: Use `@goodgo/http-client` for external APIs
- Redis caching: Implement cache patterns for frequently accessed data
- Error handling: Handle external service failures gracefully

### Acceptance Criteria for Phase 4

- [ ] All routes registered and accessible
- [ ] Middlewares applied in correct order
- [ ] Error handling works for all scenarios
- [ ] External services integrated (if any)
- [ ] Caching implemented (if needed)
- [ ] Health check endpoint works: `/health`

## Phase 5: Testing

### Test Structure

**Unit Tests:** Next to source files (`*.test.ts`), mock all dependencies
**Integration Tests:** `src/__tests__/`, test component interactions
**E2E Tests:** `src/__tests__/*.e2e.ts`, test full API workflows

### Test Coverage Targets

- Minimum: 70% coverage (branches, functions, lines, statements)
- Critical paths: 90%+ coverage
- Repositories: 80%+ coverage
- Services: 80%+ coverage
- Controllers: 70%+ coverage

### Testing Checklist

**Unit Tests:**
- [ ] Repository tests: All CRUD operations
- [ ] Service tests: Business logic, error handling
- [ ] Controller tests: Request/response handling
- [ ] DTO tests: Validation rules

**Integration Tests:**
- [ ] Module integration: Controller → Service → Repository
- [ ] Database operations: Real Prisma client with test DB
- [ ] Middleware chain: Request flow through middlewares

**E2E Tests:**
- [ ] API endpoints: Full request/response cycle
- [ ] Authentication: Protected routes
- [ ] Error scenarios: 400, 401, 403, 404, 500
- [ ] Health checks: /health endpoint

### Acceptance Criteria for Phase 5

- [ ] All unit tests pass: `pnpm test`
- [ ] Integration tests pass
- [ ] E2E tests pass
- [ ] Coverage meets thresholds: `pnpm test:coverage`
- [ ] No test warnings or errors
- [ ] Tests run in CI pipeline successfully

## Phase 6: Documentation

### Required Documentation

**Service README:**
- Service overview (bilingual EN/VI)
- Features list
- Prerequisites
- Quick start guide
- Configuration reference (environment variables table)
- API endpoints overview
- Development guide
- Testing instructions

**API Documentation:**
- Swagger/OpenAPI spec: `src/docs/swagger.ts`
- Document all endpoints
- Request/response schemas
- Examples

**Architecture Documentation (if complex):**
- `ARCHITECTURE.en.md` / `ARCHITECTURE.vi.md`
- System design, data flow, component interactions

### Documentation Checklist

- [ ] README is comprehensive and bilingual
- [ ] Swagger docs accessible: `/api-docs`
- [ ] All endpoints appear in Swagger
- [ ] Examples are clear and accurate
- [ ] Environment variables documented
- [ ] Architecture docs created (if needed)

### Acceptance Criteria for Phase 6

- [ ] README is comprehensive and bilingual
- [ ] Swagger docs accessible: `/api-docs`
- [ ] All endpoints documented with examples
- [ ] Documentation reviewed and accurate

## Phase 7: Cleanup & Verification

### Verification Process Flow

This diagram illustrates the cleanup and verification workflow for Phase 7, including the decision point for migrations and the comprehensive verification steps.

```mermaid
graph TD
    Start[Start Cleanup] --> Remove[Remove Temporary Files]
    Remove --> Update{Is Migration?}
    Update -->|Yes| RefUpdate[Update References<br/>grep & replace]
    Update -->|No| Verify[Run Verification]
    RefUpdate --> Verify
    
    Verify --> TypeCheck[TypeScript Check]
    TypeCheck --> LintCheck[Lint Check]
    LintCheck --> TestCheck[Test Check]
    TestCheck --> BuildCheck[Build Check]
    BuildCheck --> DockerCheck[Docker Build]
    DockerCheck --> HealthCheck[Health Check]
    HealthCheck --> TraefikCheck[Traefik Check]
    TraefikCheck --> AllPass{All Pass?}
    
    AllPass -->|No| Fix[Fix Issues]
    Fix --> Verify
    AllPass -->|Yes| Complete[Phase Complete]
    
    style Remove fill:#ffe1e1
    style RefUpdate fill:#fff4e1
    style Verify fill:#e1ffe1
    style Complete fill:#d4edda
```

### Cleanup Checklist

**Remove Temporary Files:**
- [ ] Remove backup directories (e.g., `service-name.backup/`)
- [ ] Remove temporary status files (e.g., `*_STATUS.md`, `*_CHECKLIST.md`)
- [ ] Remove debug/scratch files
- [ ] Clean up unused imports
- [ ] Remove commented-out code

**Reference Updates (for migrations/renames):**
```bash
# Find all references
grep -r "old-service-name" . --exclude-dir=node_modules --exclude-dir=.git

# Update checklist:
- [ ] Package names: `@goodgo/old-name` → `@goodgo/new-name`
- [ ] Service paths: `services/old-name` → `services/new-name`
- [ ] Docker images: `goodgo/old-name` → `goodgo/new-name`
- [ ] Deployment names: `old-name` → `new-name`
- [ ] Environment variables updated
- [ ] CI/CD workflows updated
- [ ] Scripts updated (if needed)
- [ ] Documentation updated (except historical context)
```

### Verification Steps

**Comprehensive Verification:**
```bash
# 1. Service starts successfully
pnpm dev && curl http://localhost:5000/health

# 2. Type checking passes
pnpm typecheck

# 3. Linting passes
pnpm lint

# 4. Tests pass with coverage
pnpm test && pnpm test:coverage

# 5. Build succeeds
pnpm build

# 6. Docker build succeeds
docker build -t service-name .

# 7. Service accessible via Traefik
curl http://localhost/api/v1/service-name/health

# 8. No broken references (if migration)
grep -r "old-reference" . --exclude-dir=node_modules
```

### Final Verification Checklist

**Code Quality:**
- [ ] No TypeScript errors
- [ ] No linting errors
- [ ] No unused imports/variables
- [ ] Code follows project conventions
- [ ] Comments are clear (bilingual if needed)

**Functionality:**
- [ ] Service starts without errors
- [ ] Health check works
- [ ] All API endpoints functional
- [ ] Database operations work
- [ ] External integrations work (if any)

**Testing:**
- [ ] All tests pass
- [ ] Coverage meets requirements
- [ ] E2E tests verify full workflows

**Documentation:**
- [ ] README is complete and accurate
- [ ] API documentation is up-to-date
- [ ] Code comments are helpful

**Infrastructure:**
- [ ] Docker image builds
- [ ] Service works in Docker Compose
- [ ] Traefik routes configured correctly
- [ ] Environment variables documented

**Cleanup:**
- [ ] Temporary files removed
- [ ] All references updated (if migration)
- [ ] No orphaned files

### Acceptance Criteria for Phase 7

- [ ] All cleanup tasks completed
- [ ] All verification steps pass
- [ ] No broken references or links
- [ ] Code is production-ready
- [ ] Documentation is complete

## Phase 8: Deployment

### Staging Deployment

**Pre-deployment Checklist:**
- [ ] Database migrations tested: `pnpm prisma migrate deploy`
- [ ] Environment variables configured in staging
- [ ] Kubernetes manifests reviewed
- [ ] Secrets configured in Kubernetes
- [ ] Health checks configured

**Deployment Steps:**
```bash
# 1. Build and push Docker image
docker build -t goodgo/service-name:latest .
docker push goodgo/service-name:latest

# 2. Apply Kubernetes configs
kubectl apply -f deployments/staging/kubernetes/service-name.yaml
kubectl apply -f deployments/staging/kubernetes/service-name-configmap.yaml

# 3. Wait for rollout
kubectl rollout status deployment/service-name -n staging

# 4. Verify deployment
kubectl get pods -n staging -l app=service-name
curl https://staging-api.example.com/api/v1/service-name/health
```

### Production Deployment

**Pre-production Checklist:**
- [ ] Staging tests passed
- [ ] Database backup created
- [ ] Rollback plan documented
- [ ] Monitoring dashboards ready
- [ ] Alerting configured

### Acceptance Criteria for Phase 8

- [ ] Service deployed to staging successfully
- [ ] All staging tests pass
- [ ] Monitoring shows healthy metrics
- [ ] Production deployment completed (if applicable)
- [ ] Post-deployment verification successful

## Rollback Strategy

### When to Rollback

- Critical errors in staging/production
- Performance degradation
- Data integrity issues
- Security vulnerabilities discovered

### Rollback Steps

```bash
# 1. Identify previous working version
kubectl rollout history deployment/service-name -n staging

# 2. Rollback to previous version
kubectl rollout undo deployment/service-name -n staging

# 3. Verify rollback
kubectl rollout status deployment/service-name -n staging

# 4. Database rollback (if needed)
# Revert migrations if schema changes were made
```

## Best Practices Summary

1. **Always Plan First**: Complete impact analysis before coding
2. **Follow Phases**: Don't skip verification steps
3. **Test Early**: Write tests alongside implementation
4. **Document as You Go**: Don't leave documentation for the end
5. **Clean Up Regularly**: Remove temporary files during development
6. **Verify Comprehensively**: Use checklists to ensure nothing is missed
7. **Plan for Rollback**: Always have a rollback strategy

## Common Pitfalls to Avoid

1. **Skipping Impact Analysis**: Leads to missing updates in scripts/configs
2. **No Verification Steps**: Misses broken references or incomplete implementation
3. **Deferring Cleanup**: Accumulates technical debt
4. **Incomplete Testing**: Missing edge cases and error scenarios
5. **Poor Documentation**: Makes maintenance difficult
6. **No Rollback Plan**: Difficult to recover from failures

## Resources

- [Project Rules](../project-rules/SKILL.md) - Architecture and conventions
- [API Design](../api-design/SKILL.md) - API design patterns
- [Testing Patterns](../testing-patterns/SKILL.md) - Testing best practices
- [Documentation](../documentation/SKILL.md) - Documentation guidelines
- [Database Prisma](../database-prisma/SKILL.md) - Prisma patterns
- Service Template: `services/_template/`