- Updated the architecture documentation to enhance clarity with detailed diagrams and descriptions for the GoodGo Microservices Platform. - Revised the .NET and Node.js template documentation to reflect new naming conventions, project structures, and setup instructions for local development. - Improved the guide documentation with verification checklists, troubleshooting steps, and real-world examples to assist developers in deploying and managing services effectively. - Ensured bilingual support in documentation to enhance accessibility for a wider audience.
472 lines
15 KiB
Markdown
472 lines
15 KiB
Markdown
# Kiến trúc Hướng Sự kiện
|
|
|
|
> Kiến trúc hướng sự kiện cho giao tiếp bất đồng bộ sử dụng Apache Kafka
|
|
|
|
## Sơ đồ Tổng quan
|
|
|
|
```mermaid
|
|
graph TD
|
|
subgraph "Event Producers"
|
|
IAM[IAM Service]
|
|
Service1[Service A]
|
|
end
|
|
|
|
subgraph "Event Broker"
|
|
Kafka[Apache Kafka]
|
|
Topics[Topics: user.events, auth.events]
|
|
end
|
|
|
|
subgraph "Event Consumers"
|
|
Consumer1[Notification Service]
|
|
Consumer2[Audit Service]
|
|
end
|
|
|
|
IAM -->|Publish| Kafka
|
|
Service1 -->|Publish| Kafka
|
|
Kafka --> Topics
|
|
Topics -->|Subscribe| Consumer1
|
|
Topics -->|Subscribe| Consumer2
|
|
|
|
style IAM fill:#5E35B1,stroke:#4527A0,color:#ffffff
|
|
style Service1 fill:#5E35B1,stroke:#4527A0,color:#ffffff
|
|
style Kafka fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style Topics fill:#FB8C00,stroke:#EF6C00,color:#ffffff
|
|
style Consumer1 fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
style Consumer2 fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
```
|
|
|
|
## Mô tả Kiến trúc
|
|
|
|
Nền tảng GoodGo triển khai Kiến trúc Hướng Sự kiện (EDA) cho giao tiếp bất đồng bộ giữa microservices.
|
|
|
|
> [!IMPORTANT]
|
|
> **Trạng thái hiện tại**: Event Bus với RabbitMQ/Kafka chưa được triển khai. Hiện tại sử dụng MediatR cho domain events nội bộ trong mỗi service.
|
|
|
|
**Nguyên tắc Cốt lõi**:
|
|
1. **Event-First Design**: Mọi thay đổi trạng thái phát ra domain events
|
|
2. **Loose Coupling**: Services giao tiếp qua events (nội bộ hoặc message broker)
|
|
3. **Eventual Consistency**: Chấp nhận inconsistency tạm thời
|
|
4. **CQRS Pattern**: Tách biệt read/write operations với MediatR
|
|
|
|
**Công nghệ Hiện tại**:
|
|
- **MediatR** - Domain events nội bộ service
|
|
- **Entity Framework Core** - Domain event dispatch qua DbContext
|
|
|
|
**Công nghệ Planned**:
|
|
- **RabbitMQ + MassTransit** - Inter-service events (Roadmap)
|
|
- **Outbox Pattern** - Reliable event publishing
|
|
|
|
## Luồng Sự kiện
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Producer as IAM Service
|
|
participant Kafka as Kafka Broker
|
|
participant Consumer as Notification Service
|
|
|
|
Producer->>Kafka: Publish Event (user.created)
|
|
Kafka->>Consumer: Deliver Event
|
|
Consumer->>Consumer: Process Event
|
|
Consumer-->>Kafka: Acknowledge
|
|
```
|
|
|
|
**Các Bước**: Publish → Distribute → Consume → Retry (nếu thất bại) → DLQ (sau retry tối đa) → Acknowledge
|
|
|
|
## Cấu trúc Sự kiện
|
|
|
|
### Domain Events (MediatR - Hiện tại)
|
|
|
|
```csharp
|
|
// EN: Base domain event interface
|
|
// VI: Interface domain event cơ bản
|
|
public interface IDomainEvent : INotification
|
|
{
|
|
Guid EventId { get; }
|
|
DateTime OccurredOn { get; }
|
|
}
|
|
|
|
// EN: Example domain event
|
|
// VI: Ví dụ domain event
|
|
public record UserCreatedEvent : IDomainEvent
|
|
{
|
|
public Guid EventId { get; } = Guid.NewGuid();
|
|
public DateTime OccurredOn { get; } = DateTime.UtcNow;
|
|
|
|
public required Guid UserId { get; init; }
|
|
public required string Email { get; init; }
|
|
public required string FirstName { get; init; }
|
|
}
|
|
|
|
// EN: Event handler
|
|
// VI: Handler xử lý event
|
|
public class UserCreatedEventHandler : INotificationHandler<UserCreatedEvent>
|
|
{
|
|
private readonly ILogger<UserCreatedEventHandler> _logger;
|
|
|
|
public async Task Handle(UserCreatedEvent notification, CancellationToken ct)
|
|
{
|
|
_logger.LogInformation("User created: {UserId}, {Email}",
|
|
notification.UserId, notification.Email);
|
|
|
|
// TODO: Send welcome email, create profile, etc.
|
|
}
|
|
}
|
|
```
|
|
|
|
## Kafka Topics
|
|
|
|
```mermaid
|
|
graph LR
|
|
UserCreated[user.created<br/>Partitions: 3]
|
|
AuthLogin[auth.login.success<br/>Partitions: 5]
|
|
AuditEvents[audit.events<br/>Partitions: 10]
|
|
|
|
style UserCreated fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style AuthLogin fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
style AuditEvents fill:#E53935,stroke:#C62828,color:#ffffff
|
|
```
|
|
|
|
**Quy ước Đặt tên**: `{domain}.{action}.{version}`
|
|
|
|
**Ví dụ**:
|
|
- `user.created.v1`
|
|
- `auth.login.success.v1`
|
|
- `audit.event.logged.v1`
|
|
|
|
## Xử lý Lỗi
|
|
|
|
```mermaid
|
|
graph TD
|
|
Event[Event] --> Process[Process]
|
|
Process -->|Success| Ack[Acknowledge]
|
|
Process -->|Failure| Retry[Retry 3x]
|
|
Retry -->|Max Retries| DLQ[Dead Letter Queue]
|
|
DLQ --> Alert[Alert Team]
|
|
|
|
style Event fill:#757575,stroke:#616161,color:#ffffff
|
|
style Process fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style Ack fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
style Retry fill:#FB8C00,stroke:#EF6C00,color:#ffffff
|
|
style DLQ fill:#E53935,stroke:#C62828,color:#ffffff
|
|
style Alert fill:#E53935,stroke:#C62828,color:#ffffff
|
|
```
|
|
|
|
**Chiến lược**:
|
|
1. Retry với exponential backoff (100ms → 200ms → 400ms)
|
|
2. Tối đa 3 lần thử
|
|
3. Chuyển sang DLQ sau retry tối đa
|
|
4. Xem xét thủ công và xử lý lại
|
|
|
|
## Bối cảnh Hệ thống
|
|
|
|
```mermaid
|
|
C4Context
|
|
title Sơ đồ Bối cảnh Event-Driven Architecture
|
|
|
|
System(iam, "IAM Service", "Event producer")
|
|
System(service_a, "Service A", "Event producer")
|
|
System(notification, "Notification Service", "Event consumer")
|
|
System(audit, "Audit Service", "Event consumer")
|
|
|
|
System_Ext(kafka, "Apache Kafka", "Event streaming platform")
|
|
System_Ext(registry, "Schema Registry", "Schema management")
|
|
System_Ext(monitoring, "Monitoring", "Kafka metrics & alerts")
|
|
|
|
Rel(iam, kafka, "Publishes events", "Kafka Protocol")
|
|
Rel(service_a, kafka, "Publishes events", "Kafka Protocol")
|
|
Rel(kafka, notification, "Delivers events", "Kafka Protocol")
|
|
Rel(kafka, audit, "Delivers events", "Kafka Protocol")
|
|
Rel(kafka, registry, "Validates schemas", "HTTP")
|
|
Rel(kafka, monitoring, "Sends metrics", "JMX")
|
|
```
|
|
|
|
**Mô tả Các Thành phần**:
|
|
- **Producers**: IAM Service và các services khác publish domain events
|
|
- **Kafka**: Event broker trung tâm, quản lý topics và partitions
|
|
- **Consumers**: Notification và Audit services consume events
|
|
- **Schema Registry**: Quản lý và validate Avro schemas
|
|
- **Monitoring**: Thu thập metrics từ Kafka cluster
|
|
|
|
## Đặc điểm Hiệu suất
|
|
|
|
| Chỉ số | Mục tiêu | Ghi chú |
|
|
|-----------------|-------------------|-----------------|
|
|
| **Event Publish Latency (P95)** | < 10ms | Fire-and-forget, async |
|
|
| **Event Delivery Latency (P95)** | < 100ms | End-to-end from publish to consume |
|
|
| **Throughput** | 10,000 events/s | Per topic, scalable with partitions |
|
|
| **Consumer Lag** | < 1000 messages | Per partition, monitored |
|
|
| **Event Size** | < 1MB | Recommended max size |
|
|
| **Retention** | 7 days | Default, configurable per topic |
|
|
| **Replication Factor** | 3 | For fault tolerance |
|
|
|
|
**Tối ưu hóa Hiệu suất**:
|
|
- **Batch Publishing**: Group multiple events để giảm network overhead
|
|
- **Compression**: Sử dụng Snappy hoặc LZ4 compression
|
|
- **Partitioning**: Phân chia topics thành multiple partitions cho parallel processing
|
|
- **Consumer Groups**: Multiple consumers trong cùng group để scale horizontally
|
|
- **Async Publishing**: Fire-and-forget pattern, không block request handlers
|
|
|
|
## Cân nhắc Bảo mật
|
|
|
|
**Mã hóa Sự kiện**:
|
|
- TLS in-transit cho tất cả Kafka connections
|
|
- Optional payload encryption cho sensitive data
|
|
- End-to-end encryption với custom encryption layer
|
|
|
|
**Kiểm soát Truy cập**:
|
|
- Kafka ACLs (Access Control Lists) per topic
|
|
- SASL/SCRAM authentication cho producers và consumers
|
|
- Separate credentials cho mỗi service
|
|
- Principle of least privilege - chỉ grant quyền cần thiết
|
|
|
|
**Xác thực Schema**:
|
|
- Avro schemas trong Schema Registry
|
|
- Schema evolution với backward/forward compatibility
|
|
- Reject events không match schema
|
|
|
|
**Kiểm toán**:
|
|
- Log tất cả event publishes và consumes
|
|
- Correlation IDs để trace event flow
|
|
- Retention policy cho audit logs (7 years)
|
|
|
|
**Lưu trữ Dữ liệu**:
|
|
- Default 7 days retention
|
|
- Configurable per topic
|
|
- Automatic deletion sau retention period
|
|
- Compliance với GDPR (right to erasure)
|
|
|
|
## Triển khai
|
|
|
|
```mermaid
|
|
graph TD
|
|
subgraph "Kafka Cluster"
|
|
subgraph "Brokers"
|
|
Broker1[Kafka Broker 1<br/>Leader for partitions 0,3,6]
|
|
Broker2[Kafka Broker 2<br/>Leader for partitions 1,4,7]
|
|
Broker3[Kafka Broker 3<br/>Leader for partitions 2,5,8]
|
|
end
|
|
|
|
subgraph "Coordination"
|
|
ZK[Zookeeper Ensemble<br/>3 nodes]
|
|
end
|
|
|
|
Broker1 --> ZK
|
|
Broker2 --> ZK
|
|
Broker3 --> ZK
|
|
end
|
|
|
|
subgraph "Producers"
|
|
IAM[IAM Service]
|
|
ServiceA[Service A]
|
|
end
|
|
|
|
subgraph "Consumers"
|
|
Notification[Notification Service<br/>Consumer Group: notifications]
|
|
Audit[Audit Service<br/>Consumer Group: audit]
|
|
end
|
|
|
|
IAM --> Broker1
|
|
IAM --> Broker2
|
|
IAM --> Broker3
|
|
|
|
ServiceA --> Broker1
|
|
ServiceA --> Broker2
|
|
ServiceA --> Broker3
|
|
|
|
Broker1 --> Notification
|
|
Broker2 --> Notification
|
|
Broker3 --> Notification
|
|
|
|
Broker1 --> Audit
|
|
Broker2 --> Audit
|
|
Broker3 --> Audit
|
|
|
|
style Broker1 fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style Broker2 fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style Broker3 fill:#1E88E5,stroke:#1565C0,color:#ffffff
|
|
style ZK fill:#8E24AA,stroke:#7B1FA2,color:#ffffff
|
|
style IAM fill:#5E35B1,stroke:#4527A0,color:#ffffff
|
|
style ServiceA fill:#5E35B1,stroke:#4527A0,color:#ffffff
|
|
style Notification fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
style Audit fill:#43A047,stroke:#2E7D32,color:#ffffff
|
|
```
|
|
|
|
### Chiến lược Triển khai
|
|
|
|
**Cấu hình Kafka Cluster**:
|
|
- **Brokers**: 3 brokers minimum (5 for production)
|
|
- **Replication Factor**: 3 (for fault tolerance)
|
|
- **Min In-Sync Replicas**: 2 (ensure data durability)
|
|
- **Partitions**: 3-10 per topic (based on throughput needs)
|
|
- **Zookeeper**: 3-node ensemble (for coordination)
|
|
|
|
**Phân bổ Tài nguyên**:
|
|
| Component | CPU | Memory | Disk |
|
|
|-----------|-----|--------|------|
|
|
| **Kafka Broker** | 2 cores | 4GB RAM | 100GB SSD |
|
|
| **Zookeeper** | 1 core | 2GB RAM | 20GB SSD |
|
|
| **Schema Registry** | 500m | 1GB RAM | 10GB |
|
|
|
|
**Cấu hình Topic**:
|
|
```yaml
|
|
user.created:
|
|
partitions: 3
|
|
replication-factor: 3
|
|
retention-ms: 604800000 # 7 days
|
|
compression-type: snappy
|
|
|
|
auth.login.success:
|
|
partitions: 5
|
|
replication-factor: 3
|
|
retention-ms: 604800000
|
|
compression-type: snappy
|
|
|
|
audit.events:
|
|
partitions: 10
|
|
replication-factor: 3
|
|
retention-ms: 220752000000 # 7 years
|
|
compression-type: lz4
|
|
```
|
|
|
|
**Tính Sẵn sàng Cao**:
|
|
- Multiple brokers với partition replication
|
|
- Automatic leader election khi broker fails
|
|
- Consumer group rebalancing
|
|
- Monitoring và alerting cho broker health
|
|
|
|
## Giám sát & Khả năng quan sát
|
|
|
|
### Chỉ số Chính
|
|
|
|
**Kafka Broker Metrics**:
|
|
- `kafka_server_brokertopicmetrics_messagesinpersec` - Messages in/sec
|
|
- `kafka_server_brokertopicmetrics_bytesinpersec` - Bytes in/sec
|
|
- `kafka_server_brokertopicmetrics_bytesoutpersec` - Bytes out/sec
|
|
- `kafka_controller_kafkacontroller_activecontrollercount` - Active controller
|
|
- `kafka_server_replicamanager_underreplicatedpartitions` - Under-replicated partitions
|
|
|
|
**Consumer Metrics**:
|
|
- `kafka_consumer_fetch_manager_records_lag_max` - Max consumer lag
|
|
- `kafka_consumer_fetch_manager_records_consumed_rate` - Records consumed/sec
|
|
- `kafka_consumer_coordinator_commit_latency_avg` - Commit latency
|
|
|
|
**Producer Metrics**:
|
|
- `kafka_producer_record_send_total` - Total records sent
|
|
- `kafka_producer_record_error_total` - Total send errors
|
|
- `kafka_producer_request_latency_avg` - Request latency
|
|
|
|
**Application Metrics**:
|
|
```typescript
|
|
// Custom metrics cho event processing
|
|
const eventPublished = new Counter({
|
|
name: 'events_published_total',
|
|
help: 'Total events published',
|
|
labelNames: ['event_type', 'topic']
|
|
});
|
|
|
|
const eventConsumed = new Counter({
|
|
name: 'events_consumed_total',
|
|
help: 'Total events consumed',
|
|
labelNames: ['event_type', 'topic', 'consumer_group']
|
|
});
|
|
|
|
const eventProcessingDuration = new Histogram({
|
|
name: 'event_processing_duration_seconds',
|
|
help: 'Event processing duration',
|
|
labelNames: ['event_type'],
|
|
buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5]
|
|
});
|
|
```
|
|
|
|
**Quy tắc Cảnh báo**:
|
|
```yaml
|
|
# High consumer lag
|
|
- alert: HighConsumerLag
|
|
expr: kafka_consumer_fetch_manager_records_lag_max > 10000
|
|
for: 5m
|
|
severity: warning
|
|
annotations:
|
|
summary: "High consumer lag detected"
|
|
description: "Consumer lag is {{ $value }} messages"
|
|
|
|
# Broker down
|
|
- alert: KafkaBrokerDown
|
|
expr: kafka_server_kafkaserver_brokerstate != 3
|
|
for: 1m
|
|
severity: critical
|
|
annotations:
|
|
summary: "Kafka broker is down"
|
|
|
|
# Under-replicated partitions
|
|
- alert: UnderReplicatedPartitions
|
|
expr: kafka_server_replicamanager_underreplicatedpartitions > 0
|
|
for: 5m
|
|
severity: warning
|
|
annotations:
|
|
summary: "Under-replicated partitions detected"
|
|
|
|
# Offline partitions
|
|
- alert: OfflinePartitions
|
|
expr: kafka_controller_kafkacontroller_offlinepartitionscount > 0
|
|
for: 1m
|
|
severity: critical
|
|
annotations:
|
|
summary: "Offline partitions detected"
|
|
```
|
|
|
|
**Dashboards**:
|
|
- Kafka Cluster Overview (brokers, topics, partitions)
|
|
- Producer Performance (throughput, latency, errors)
|
|
- Consumer Performance (lag, throughput, errors)
|
|
- Topic Metrics (messages/sec, bytes/sec, retention)
|
|
|
|
**Logging**:
|
|
```typescript
|
|
// Structured logging cho events
|
|
logger.info('Event published', {
|
|
eventId: event.eventId,
|
|
eventType: event.eventType,
|
|
topic: 'user.created',
|
|
correlationId: event.correlationId
|
|
});
|
|
|
|
logger.info('Event consumed', {
|
|
eventId: event.eventId,
|
|
eventType: event.eventType,
|
|
topic: 'user.created',
|
|
consumerGroup: 'notifications',
|
|
processingTime: duration
|
|
});
|
|
```
|
|
|
|
|
|
## Tài liệu Liên quan
|
|
|
|
- [System Design](./system-design.md) - Kiến trúc tổng thể
|
|
- [IAM Architecture](./iam-proposal.md) - Triển khai Event sourcing
|
|
|
|
---
|
|
|
|
**Cập nhật Lần cuối**: 2026-01-14
|
|
**Tác giả**: GoodGo Architecture Team
|
|
|
|
## Mẹo Nhanh
|
|
|
|
### Bảng Màu Mermaid
|
|
|
|
| Loại Node | Màu Nền | Màu Viền | Màu Chữ | Sử dụng |
|
|
|-----------|------------|--------------|------------|-------|
|
|
| **Core/Broker** | `#1E88E5` (Blue) | `#1565C0` | `#ffffff` | Kafka Brokers, Main Components |
|
|
| **Topic/Data** | `#FB8C00` (Orange) | `#EF6C00` | `#ffffff` | Topics, Queues, Data Stores |
|
|
| **Success/Safe** | `#43A047` (Green) | `#2E7D32` | `#ffffff` | Successful flows, Safe states |
|
|
| **Error/Danger** | `#E53935` (Red) | `#C62828` | `#ffffff` | Errors, DLQ, Critical issues |
|
|
| **Coordination** | `#8E24AA` (Purple) | `#7B1FA2` | `#ffffff` | Zookeeper, Orchestrators |
|
|
|
|
### Các Chỉ báo Trực quan
|
|
|
|
- 🔄 **Retry Loop**: Chỉ báo thử lại tự động
|
|
- ⚠️ **DLQ/Warning**: Đường dẫn xử lý lỗi
|
|
- 📝 **Log/Audit**: Điểm ghi log
|
|
- 🔐 **Lock/Auth**: Kiểm tra bảo mật
|
|
|