pos-system/microservices/.claude/POS_DEPLOYMENT_STATE.md

GoodGo POS System Deployment State - Comprehensive Analysis

Ngày tạo: 2026-04-09 | Cập nhật lần cuối: 2026-04-11
Working Directory: /Users/velikho/Desktop/WORKING/pos-system
Project: GoodGo Platform - Monorepo with 26 microservices

---

Executive Summary

The GoodGo platform is a enterprise-scale microservices POS system built on:

• .NET 10 backend (C# 14, clean architecture + CQRS)
• PostgreSQL 16 (per-service databases)
• Kubernetes (RKE2) for staging/production deployment
• Docker Compose for local development
• Multi-vertical support: POS, F&B, retail, spa, karaoke

Deployment Strategy:

• Local: Docker Compose (single-machine development)
• Staging: Kubernetes with Neon PostgreSQL (self-hosted on K8s)
• Production: Kubernetes with Neon PostgreSQL (cloud)

Current Staging Live Trạng thái (2026-04-11)

Component	Trạng thái	Details
DNS	✅ Live	api.techbi.org + platform.techbi.org → 212.28.186.239
TLS	✅ Valid	Let's Encrypt, expires Jul 2026
Harbor Registry	✅ 25 images	harbor.techbi.org/goodgo/*
K8s Services	✅ 23/25 running	1 replica each, iam-service needs resources
Neon PostgreSQL	✅ Running	Self-hosted in neon namespace, NodePort 30992
CI/CD	✅ Gitea Actions	Parallel Kaniko builds → Harbor → K8s deploy
Redis	✅ Running	In-cluster, port 6379
RabbitMQ	✅ Running	In-cluster, port 5672

Cluster Nodes (3-node RKE2)

Node	Role	IP	CPU	Memory
vmi3082489	control-plane	212.28.186.239	6 cores	12 GB
vmi3202282	worker	185.225.232.65	6 cores	12 GB
vmi3202283	worker	185.225.233.97	6 cores	12 GB

Note

: DNS points to control plane 212.28.186.239 where ingress-nginx can resolve cluster DNS and route to ClusterIPs. Worker nodes have hostNetwork issue preventing ClusterIP routing from ingress pods.

---

1. Kubernetes Manifests & Deployments

Location

deployments/
├── staging/kubernetes/        # 35 YAML files (namespace: staging)
├── production/kubernetes/     # 14 YAML files (namespace: production)
└── local/
    ├── docker-compose.yml
    └── kubernetes/            # Local K8s test manifests

Staging Kubernetes Services (35 total)

Core POS Services (8):

• iam-service, merchant-service, order-service, fnb-engine
• catalog-service, inventory-service, wallet-service, booking-service

Engagement Services (5):

• promotion-service, membership-service, chat-service, social-service, mission-service

Advertising Services (5):

• ads-manager-service, ads-serving-service, ads-billing-service
• ads-tracking-service, ads-analytics-service

Marketing Integrations (4):

• mkt-facebook-service, mkt-whatsapp-service, mkt-x-service, mkt-zalo-service

Utilities:

• storage-service, mining-service

Infrastructure:

• rabbitmq, redis, redis-sentinel, minio
• ingress, namespace, network-policy
• configmap, secrets, act-runner-rbac, gitea-sync-cronjob

Production Kubernetes Services (14 total)

Reduced subset - only core services:

• Core 8 services + redis + infrastructure (ingress, namespace, configmap, secrets)

Strategy: Production uses core services only for stability/performance

---

2. Configuration & Secrets Management

ConfigMap Configuration

File: deployments/staging/kubernetes/configmap.yaml

Key Settings:

Category	Variables	Staging Value	Production Value
Environment	ASPNETCORE_ENVIRONMENT	Staging	Production
Service Port	ASPNETCORE_URLS	http://+:8080	http://+:8080
JWT Authority	Jwt__Authority	https://api.techbi.org	http://iam-service:8080
JWT Audience	Jwt__Audience	goodgo-api	goodgo-api
JWT HTTPS	Jwt__RequireHttpsMetadata	true	true
Redis Host	Redis__Host	redis	redis
Redis Port	Redis__Port	6379	6379
MinIO Bucket	Storage__MinIO__BucketName	goodgo-staging	goodgo-prod
CORS Origins	Cors__AllowedOrigins	platform.techbi.org, api.techbi.org	pos.goodgo.vn, goodgo.vn
Log Level	Serilog__MinimumLevel__Default	Information	Warning
Swagger	Features__SwaggerEnabled	true	false

Secrets Management

File: deployments/staging/kubernetes/secrets.yaml

Contains PLACEHOLDER values only - real secrets in:

• Kubernetes kubectl create secret commands
• GitHub Secrets (CI/CD)
• External-secrets operator
• Sealed-secrets (GitOps)

Secrets Inventory (35 total entries):

Secret Loại	Count	Examples
JWT Keys	2	Jwt__Secret, Jwt__RefreshSecret
Database URLs	23	One per service (iam_service, merchant_service, ...)
Redis	2	Redis__Password, ConnectionStrings__Redis
MinIO	3	AccessKey, SecretKey, Endpoint
RabbitMQ	2	Username, Password
IdentityServer	1	IssuerUri

Connection String Format:

Host=db-host;Port=30992;Database=[service_name];
Username=cloud_admin;Password=CHANGE_ME;
SSL Mode=Prefer

---

3. Database Migrations

Migration Locations (22 services)

services/[service-name]-net/src/[ServiceName].Infrastructure/
├── Migrations/
│   ├── yyyyMMddHHmmss_Name.cs
│   ├── yyyyMMddHHmmss_Name.Designer.cs
│   └── [ServiceName]ContextModelSnapshot.cs
└── Data/
    └── DataSeeder.cs (optional)

Example: Order Service Migrations

20260117175742_InitialOrder.cs
20260305004928_AddTableIdAndDiscountFields.cs
20260306175520_PhaseTwo.cs

Services with Migrations (All 22 .NET services):

iam-service, merchant-service, order-service, fnb-engine, catalog-service, inventory-service, wallet-service, booking-service, promotion-service, membership-service, chat-service, social-service, mission-service, mining-service, storage-service, ads-manager-service, ads-serving-service, ads-billing-service, ads-tracking-service, ads-analytics-service, mkt-zalo-service, mkt-facebook-service

Migration Execution

# Polyglot migration script
./scripts/db/migrate.sh

# Manual per-service
dotnet ef database update --project services/[service-name]-net

---

4. Documentation

Documentation Structure

docs/
├── README.md
├── production-checklist.md      (82-item deployment checklist)
├── adr/                         (Architecture Decision Records)
├── audit/                       (19 role-based audit reports)
├── en/ & vi/                    (English & Vietnamese docs)
│   ├── architecture/            (8 architecture docs)
│   ├── guides/                  (9 deployment guides)
│   ├── skills/                  (15 skill docs)
│   ├── runbooks/                (incident response, rollback)
│   └── templates/               (architecture, dotnet, nodejs)

Key Documents

Document	Mục đích	Updated
README.md	Project overview & quick start	Current
CLAUDE.md	Agent configuration & full architecture	Current
ROADMAP.md	Development phases & features	Current
production-checklist.md	82-item deployment checklist	2026-03-06
CTO_DEPLOYMENT_REPORT.md	Deployment analysis	2026-03-14
CTO_FIX_TRACKER.md	Bug fixes & tracking	2026-03-13

Kiến Trúc Documentation

1. system-design.md - Overall architecture
2. microservices-communication.md - Service-to-service patterns
3. event-driven-architecture.md - RabbitMQ event patterns
4. multi-vertical-architecture.md - POS multi-vertical
5. caching-architecture.md - Redis caching
6. data-consistency-patterns.md - Database consistency
7. observability-architecture.md - Monitoring/logging
8. security-architecture.md - Auth/encryption/rate limiting
9. iam-proposal.md - Identity service design

---

5. Infrastructure Configuration

Local Development

File: deployments/local/docker-compose.yml (1349 lines)

Services:

• All 26 .NET microservices
• PostgreSQL 16 + Redis 7 + RabbitMQ 3
• MinIO (S3-compatible storage)
• Traefik v3 (API gateway)
• Full observability stack (Prometheus, Grafana, Loki, Promtail)

Hạ Tầng Directories

infra/
├── docker/                  # Dev/Prod Docker Compose
├── databases/              # PostgreSQL + Redis + Neon
├── observability/          # Prometheus, Grafana, Loki, Promtail
│   ├── prometheus/         # Rules & config
│   ├── grafana/           # Dashboards & datasources
│   ├── loki/              # Log aggregation
│   ├── alertmanager/      # Alert routing
│   └── promtail/          # Log shipper
└── traefik/               # API Gateway
    ├── traefik.yml        # Main config
    └── dynamic/           # Routes, middleware, services

---

6. Database Architecture

Per-Service Database Pattern

Each service has its own PostgreSQL database:

iam-service → iam_service
merchant-service → merchant_service
order-service → order_service
fnb-engine → fnb_engine
... (23 total services)

Cơ Sở Dữ Liệu Providers

Environment	Provider	Details
Local	PostgreSQL 16 (Docker)	Single instance
Staging	Neon PostgreSQL (cloud)	Branching, PITR, serverless
Production	Neon PostgreSQL (cloud)	HA, failover, autoscaling

---

7. Service Architecture Pattern

Clean Architecture + CQRS

ServiceName/
├── src/
│   ├── ServiceName.API/
│   │   ├── Application/ (Commands, Queries, Validations, Behaviors)
│   │   ├── Controllers/ ([ApiVersion("1.0")])
│   │   └── Program.cs (DI + middleware)
│   ├── ServiceName.Domain/
│   │   ├── AggregatesModel/ (Entity + IAggregateRoot)
│   │   ├── SeedWork/ (Entity, IRepository, IUnitOfWork, ValueObject, Enumeration)
│   │   └── Events/ (Domain events, Exceptions)
│   └── ServiceName.Infrastructure/
│       ├── Persistence/ (DbContext, IUnitOfWork)
│       ├── EntityConfigurations/ (Fluent API, snake_case)
│       ├── Repositories/
│       ├── Migrations/ (EF Core migrations)
│       └── DependencyInjection.cs
└── tests/
    ├── UnitTests/ (xUnit + Moq + FluentAssertions)
    └── FunctionalTests/ (WebApplicationFactory)

Key Patterns

• Commands: record VerbEntityCommand(...) : IRequest<Result>
• Queries: record GetEntityQuery(...) : IRequest<Result>
• Handlers: class VerbEntityCommandHandler : IRequestHandler<>
• Validators: class VerbEntityCommandValidator : AbstractValidator<>
• Repositories: Interface in Domain, Implementation in Infrastructure

---

8. Tech Stack

Layer	Technology	Version
Runtime	.NET Core	10.0
Language	C#	14
Framework	ASP.NET Core	10.0
CQRS	MediatR	12.4+
ORM	Entity Framework Core	10
Validation	FluentValidation	11
Logging	Serilog	Latest
Caching	Redis	7
Data Access	Dapper	Latest
Resilience	Polly	Latest
Frontend	Blazor WASM + MudBlazor	10.0 + 8.15
Mobile	.NET MAUI / SwiftUI	Latest
Database	PostgreSQL	16 (Neon)
Message Broker	RabbitMQ	3
Storage	MinIO	S3-compatible
Container Orchestration	Kubernetes (RKE2)	Latest
Container Registry	Harbor	harbor.techbi.org/goodgo/*
CI/CD	Gitea Actions + Kaniko	Parallel batch builds
API Gateway	Nginx Ingress Controller	Latest
Monitoring	Prometheus + Grafana + Loki	Latest
CI/CD	Gitea Actions + Kaniko	Parallel batch builds
Monorepo	pnpm 8 + Turborepo	Latest

---

9. Deployment Environments

Local Development

• Docker Compose (single machine)
• All 26 services + infrastructure
• PostgreSQL local
• Full observability stack
• HTTP via Traefik

Staging

• Kubernetes (RKE2) multi-node
• 35 services (full platform)
• Neon PostgreSQL (cloud)
• Domain: api.staging.goodgo.vn
• Features: Swagger enabled, detailed errors
• Logging: Information level
• JWT Authority: https://api.techbi.org
• Secrets: kubectl + GitHub Actions

Production

• Kubernetes (RKE2) ≥3 nodes
• 14 services (core only)
• Neon PostgreSQL (cloud)
• Domain: goodgo.vn, pos.goodgo.vn
• Features: Swagger disabled, no detailed errors
• Logging: Warning level
• JWT Authority: iam-service (internal)
• Secrets: sealed-secrets / external-secrets operator
• Security: Network policies, rate limiting, RBAC

---

10. Production Deployment Checklist

From: docs/production-checklist.md (82 items)

Pre-Deployment (11)

• E2E tests passing
• Security audit completed
• Database migrations reviewed
• Secrets rotated
• SSL/TLS certificates ready
• DNS records configured
• CDN configured
• Backup strategy verified
• Load testing completed
• Rollback plan approved

Hạ Tầng (13)

• K8s cluster ≥3 nodes
• Namespace created
• Resource limits configured
• HPA (2-10 replicas)
• PersistentVolumeClaims
• Ingress + TLS configured
• Network policies enforced
• Node affinity rules

Per-Service (12)

• Docker image tagged with SHA
• Image pushed to Docker Hub
• Environment variables in Secrets
• Health checks responding
• Database migrated
• Seed data loaded
• Connection strings configured
• Redis/RabbitMQ configured
• Logging level configured

Monitoring (8)

• Prometheus scraping
• Grafana dashboards loaded
• Alert rules active
• Alert notifications configured
• Loki receiving logs
• Dashboard access restricted

Security (17)

• JWT keys rotated
• OIDC discovery endpoint live
• Token expiry configured
• CORS configured
• HTTPS enforced
• Security headers configured
• Rate limiting configured
• RLS policies applied
• No secrets in ConfigMap

Post-Deployment (20)

• Smoke tests (IAM login, Merchant shop, Order flow)
• FnB kitchen flow tested
• Wallet/VNPay tested
• Multi-browser session tested
• EOD report tested
• Error rates < 0.1% (5xx)
• p95 latency < 500ms
• SignalR connections stable
• Grafana dashboards live
• Alert rules working

---

11. Key Files Summary

File	Lines	Mục đích
deployments/local/docker-compose.yml	1349	Local dev environment
CLAUDE.md	500+	Agent config & architecture
ROADMAP.md	600+	Development phases
docs/production-checklist.md	186	Deployment checklist
README.md	130	Project overview
CTO_DEPLOYMENT_REPORT.md	250+	Deployment analysis

---

12. Critical Observations

Strengths ✓

• Comprehensive Kubernetes infrastructure
• Database per service (true microservices)
• Clean architecture across all services
• Extensive documentation (English + Vietnamese)
• Security-first design (secrets, RBAC, rate limiting)
• Production checklist (82 items)
• Cloud-ready (Neon PostgreSQL)

Considerations ⚠

• 23 database URLs (each needs GitHub Secret)
• 26 services in staging (complex management)
• JWT authority differs per environment
• CORS origins must be updated per environment
• Secrets rotation requires manual process

Deployment Strategy

• Staging: Full 26 services (development focus)
• Production: Core 8 services (performance focus)

---

13. Conclusion

The GoodGo POS system is a production-grade microservices platform with:

• ✓ Comprehensive Kubernetes deployment
• ✓ 26 specialized services
• ✓ Robust database isolation
• ✓ Complete observability
• ✓ Security-focused configuration
• ✓ Extensive documentation
• ✓ Clear staging → production path

Trạng thái: Mature, well-documented system ready for production operation.