2640b351c3
- Added request/response flow diagrams to api-design and api-gateway-advanced skills for better visualization of processes. - Introduced configuration loading flow in configuration-management skill to clarify the configuration process. - Included error propagation flow in error-handling-patterns skill to illustrate error handling across layers. - Enhanced various skills with additional diagrams to improve understanding of complex concepts. These updates aim to provide clearer guidance and improve the overall documentation experience for developers.
379 lines
13 KiB
Markdown
379 lines
13 KiB
Markdown
# Patterns Nhất Quán Dữ Liệu (Data Consistency Patterns)
|
|
|
|
Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services.
|
|
> Các patterns nhất quán dữ liệu cho distributed microservices bao gồm Saga patterns, distributed transactions, eventual consistency, compensation, và idempotency. Sử dụng khi xử lý distributed transactions, implement eventual consistency, hoặc quản lý đồng bộ dữ liệu giữa các services.
|
|
|
|
## Tổng Quan
|
|
|
|
Data consistency in distributed systems requires different approaches than traditional ACID transactions. This skill covers Saga patterns for distributed transactions, idempotency for retries, optimistic locking for conflicts, and eventual consistency strategies.
|
|
|
|
Nhất quán dữ liệu trong hệ thống phân tán yêu cầu các cách tiếp cận khác với ACID transactions truyền thống. Skill này bao gồm Saga patterns cho distributed transactions, idempotency cho retries, optimistic locking cho conflicts, và các chiến lược eventual consistency.
|
|
|
|
## Khi Nào Sử Dụng
|
|
|
|
Use this skill when:
|
|
- Implementing distributed transactions across multiple services
|
|
- Handling eventual consistency in microservices
|
|
- Implementing Saga patterns for distributed workflows
|
|
- Designing compensation strategies
|
|
|
|
Sử dụng skill này khi:
|
|
- Implement distributed transactions qua nhiều services
|
|
- Xử lý eventual consistency trong microservices
|
|
- Implement Saga patterns cho distributed workflows
|
|
- Thiết kế chiến lược compensation
|
|
|
|
## Khái Niệm Chính
|
|
|
|
### ACID vs BASE
|
|
|
|
**ACID (Truyền Thống)**: Atomicity, Consistency, Isolation, Durability
|
|
|
|
**BASE (Phân Tán)**: Basic Availability, Soft state, Eventual consistency
|
|
|
|
### Các Mô Hình Nhất Quán
|
|
|
|
- **Strong Consistency**: Tất cả nodes thấy cùng dữ liệu cùng lúc
|
|
- **Eventual Consistency**: Hệ thống trở nên nhất quán theo thời gian
|
|
- **Weak Consistency**: Không đảm bảo về thời điểm nhất quán
|
|
|
|
## Các Patterns Chính
|
|
|
|
### Saga Orchestrator Pattern
|
|
|
|
```typescript
|
|
// EN: Centralized orchestrator coordinates steps
|
|
// VI: Orchestrator tập trung điều phối các bước
|
|
const saga = new SagaOrchestrator();
|
|
await saga.execute({
|
|
sagaId: 'saga_123',
|
|
steps: [
|
|
{ name: 'create-order', execute: createOrder, compensate: cancelOrder },
|
|
{ name: 'reserve-inventory', execute: reserveInventory, compensate: releaseInventory },
|
|
],
|
|
});
|
|
```
|
|
|
|
#### Luồng Điều Phối Saga (Saga Orchestration Flow)
|
|
|
|
Biểu đồ sau minh họa cách Saga orchestrator thực thi các bước tuần tự và xử lý compensation khi thất bại:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant Orchestrator
|
|
participant Step1 as Step 1: Create Order
|
|
participant Step2 as Step 2: Reserve Inventory
|
|
participant Step3 as Step 3: Process Payment
|
|
|
|
Client->>Orchestrator: Execute Saga
|
|
Orchestrator->>Step1: Execute Step 1
|
|
Step1-->>Orchestrator: Success (Order Created)
|
|
Orchestrator->>Step2: Execute Step 2
|
|
Step2-->>Orchestrator: Success (Inventory Reserved)
|
|
Orchestrator->>Step3: Execute Step 3
|
|
Step3-->>Orchestrator: Failure (Payment Failed)
|
|
Orchestrator->>Step2: Compensate Step 2
|
|
Step2-->>Orchestrator: Compensation Complete
|
|
Orchestrator->>Step1: Compensate Step 1
|
|
Step1-->>Orchestrator: Compensation Complete
|
|
Orchestrator-->>Client: Saga Failed (Compensated)
|
|
```
|
|
|
|
#### Luồng Compensation (Compensation Flow)
|
|
|
|
Khi một bước thất bại, orchestrator sẽ compensate tất cả các bước đã hoàn thành trước đó theo thứ tự ngược lại:
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
Start([Saga Execution Starts]) --> Step1[Execute Step 1]
|
|
Step1 -->|Success| Step2[Execute Step 2]
|
|
Step1 -->|Failure| Fail1[Saga Failed<br/>No Compensation Needed]
|
|
|
|
Step2 -->|Success| Step3[Execute Step 3]
|
|
Step2 -->|Failure| Comp1[Compensate Step 1]
|
|
|
|
Step3 -->|Success| Complete([Saga Completed])
|
|
Step3 -->|Failure| Comp2[Compensate Step 2]
|
|
|
|
Comp2 --> Comp1
|
|
Comp1 --> Fail2[Saga Failed<br/>All Steps Compensated]
|
|
|
|
style Start fill:#e1f5ff
|
|
style Complete fill:#d4edda
|
|
style Fail1 fill:#f8d7da
|
|
style Fail2 fill:#f8d7da
|
|
style Comp1 fill:#fff3cd
|
|
style Comp2 fill:#fff3cd
|
|
```
|
|
|
|
#### Luồng Eventual Consistency (Eventual Consistency Flow)
|
|
|
|
Biểu đồ này cho thấy cách dữ liệu trở nên nhất quán giữa các services theo thời gian thông qua việc truyền events:
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph ServiceA[Service A: Write Model]
|
|
Write[Write Operation] --> EventStore[Event Store]
|
|
EventStore --> Publish[Publish Event]
|
|
end
|
|
|
|
subgraph EventBus[Event Bus]
|
|
Publish --> Queue[Event Queue]
|
|
end
|
|
|
|
subgraph ServiceB[Service B: Read Model]
|
|
Queue --> Consume[Consume Event]
|
|
Consume --> Update[Update Read Model]
|
|
Update --> Consistent[Eventually Consistent]
|
|
end
|
|
|
|
subgraph ServiceC[Service C: Read Model]
|
|
Queue --> Consume2[Consume Event]
|
|
Consume2 --> Update2[Update Read Model]
|
|
Update2 --> Consistent2[Eventually Consistent]
|
|
end
|
|
|
|
style Write fill:#e1f5ff
|
|
style Consistent fill:#d4edda
|
|
style Consistent2 fill:#d4edda
|
|
style Queue fill:#fff3cd
|
|
```
|
|
|
|
### Saga Choreography Pattern / Pattern Saga Choreography
|
|
|
|
Trong choreography, các services phản ứng với events mà không cần coordinator tập trung:
|
|
|
|
```typescript
|
|
// EN: Services react to events
|
|
// VI: Các services phản ứng với events
|
|
eventConsumer.on('order.created', async (event) => {
|
|
await inventoryService.reserve(event.data.items);
|
|
await eventPublisher.publish('inventory.reserved', {...});
|
|
});
|
|
```
|
|
|
|
#### Luồng Saga Choreography (Saga Choreography Flow)
|
|
|
|
Biểu đồ sau cho thấy cách các services phối hợp thông qua events trong pattern choreography:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant OrderService
|
|
participant EventBus
|
|
participant InventoryService
|
|
participant PaymentService
|
|
participant NotificationService
|
|
|
|
OrderService->>EventBus: Publish order.created
|
|
EventBus->>InventoryService: order.created event
|
|
InventoryService->>InventoryService: Reserve Inventory
|
|
InventoryService->>EventBus: Publish inventory.reserved
|
|
EventBus->>PaymentService: inventory.reserved event
|
|
PaymentService->>PaymentService: Process Payment
|
|
PaymentService->>EventBus: Publish payment.processed
|
|
EventBus->>NotificationService: payment.processed event
|
|
NotificationService->>NotificationService: Send Confirmation
|
|
|
|
Note over InventoryService,PaymentService: If payment fails,<br/>compensation events are published
|
|
PaymentService->>EventBus: Publish payment.failed
|
|
EventBus->>InventoryService: payment.failed event
|
|
InventoryService->>InventoryService: Release Inventory
|
|
EventBus->>OrderService: payment.failed event
|
|
OrderService->>OrderService: Cancel Order
|
|
```
|
|
|
|
### Idempotency / Idempotency
|
|
|
|
Idempotency đảm bảo các operations có thể được retry an toàn mà không có side effects:
|
|
|
|
```typescript
|
|
// EN: Execute with idempotency check
|
|
// VI: Thực thi với idempotency check
|
|
await idempotencyHandler.execute(
|
|
idempotencyKey,
|
|
async () => await userService.create(data)
|
|
);
|
|
```
|
|
|
|
#### Luồng Idempotency (Idempotency Flow)
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
Request[Client Request] --> Check{Idempotency Key<br/>Exists?}
|
|
Check -->|Yes| Return[Return Cached Result]
|
|
Check -->|No| Execute[Execute Operation]
|
|
Execute --> Store[Store Result with Key]
|
|
Store --> Return2[Return Result]
|
|
|
|
Return --> Client[Client Response]
|
|
Return2 --> Client
|
|
|
|
style Check fill:#fff3cd
|
|
style Return fill:#d4edda
|
|
style Return2 fill:#d4edda
|
|
```
|
|
|
|
### Optimistic Locking / Khóa Lạc Quan
|
|
|
|
Optimistic locking ngăn chặn lost updates bằng cách sử dụng version fields:
|
|
|
|
```typescript
|
|
// EN: Update with version check
|
|
// VI: Cập nhật với kiểm tra version
|
|
await optimisticLockService.updateWithLock(
|
|
repository,
|
|
id,
|
|
(current) => ({ ...current, name: newName })
|
|
);
|
|
```
|
|
|
|
#### Luồng Optimistic Locking (Optimistic Locking Flow)
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client1
|
|
participant Client2
|
|
participant Service
|
|
participant DB[(Database)]
|
|
|
|
Client1->>Service: Read Entity (version=1)
|
|
Service->>DB: SELECT * WHERE id=123
|
|
DB-->>Service: Entity (version=1)
|
|
Service-->>Client1: Entity Data
|
|
|
|
Client2->>Service: Read Entity (version=1)
|
|
Service->>DB: SELECT * WHERE id=123
|
|
DB-->>Service: Entity (version=1)
|
|
Service-->>Client2: Entity Data
|
|
|
|
Client1->>Service: Update (version=1)
|
|
Service->>DB: UPDATE WHERE id=123 AND version=1
|
|
DB-->>Service: Success (version=2)
|
|
|
|
Client2->>Service: Update (version=1)
|
|
Service->>DB: UPDATE WHERE id=123 AND version=1
|
|
DB-->>Service: Conflict (version=2 exists)
|
|
Service-->>Client2: OptimisticLockError
|
|
Note over Client2: Retry with new version
|
|
```
|
|
|
|
### CQRS Pattern / Pattern CQRS
|
|
|
|
Command Query Responsibility Segregation tách biệt read và write operations để tối ưu hiệu suất và khả năng mở rộng.
|
|
|
|
#### Luồng Kiến Trúc CQRS (CQRS Architecture Flow)
|
|
|
|
Biểu đồ sau minh họa cách CQRS tách biệt write path và read path:
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph WritePath[Write Path]
|
|
Command[Command] --> WriteModel[Write Model<br/>Normalized]
|
|
WriteModel --> Event[Domain Event]
|
|
Event --> EventStore[(Event Store)]
|
|
end
|
|
|
|
subgraph ReadPath[Read Path]
|
|
Query[Query] --> ReadModel[Read Model<br/>Denormalized]
|
|
ReadModel --> Response[Query Response]
|
|
end
|
|
|
|
subgraph Sync[Eventual Consistency]
|
|
EventStore --> EventHandler[Event Handler]
|
|
EventHandler --> Projection[Projection]
|
|
Projection --> ReadModel
|
|
end
|
|
|
|
style WritePath fill:#e1f5ff
|
|
style ReadPath fill:#d4edda
|
|
style Sync fill:#fff3cd
|
|
style EventStore fill:#f8d7da
|
|
```
|
|
|
|
```typescript
|
|
// EN: Write path: Command handler
|
|
// VI: Write path: Command handler
|
|
await commandHandler.handle({
|
|
type: 'CREATE_ORDER',
|
|
data: { userId, items }
|
|
});
|
|
|
|
// EN: Read path: Optimized query
|
|
// VI: Read path: Query tối ưu
|
|
const orders = await readModel.findOrdersByUser(userId);
|
|
```
|
|
|
|
### Outbox Pattern / Pattern Outbox
|
|
|
|
Outbox pattern đảm bảo event publishing đáng tin cậy bằng cách lưu events trong cùng database transaction với business data.
|
|
|
|
#### Luồng Outbox Pattern (Outbox Pattern Flow)
|
|
|
|
Biểu đồ này cho thấy cách Outbox pattern đảm bảo việc gửi events:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant Service
|
|
participant DB[(Database)]
|
|
participant OutboxTable[(Outbox Table)]
|
|
participant Processor[Outbox Processor]
|
|
participant EventBus[Event Bus]
|
|
|
|
Client->>Service: Business Operation
|
|
Service->>DB: Begin Transaction
|
|
Service->>DB: Update Business Data
|
|
Service->>OutboxTable: Insert Event (same transaction)
|
|
Service->>DB: Commit Transaction
|
|
|
|
Note over Service,OutboxTable: Event stored atomically<br/>with business data
|
|
|
|
Processor->>OutboxTable: Poll for Unpublished Events
|
|
OutboxTable-->>Processor: Return Events
|
|
Processor->>EventBus: Publish Event
|
|
EventBus-->>Processor: Publish Confirmed
|
|
Processor->>OutboxTable: Mark as Published
|
|
|
|
Note over Processor,EventBus: Background processor<br/>ensures delivery
|
|
```
|
|
|
|
```typescript
|
|
// EN: Execute business operation and store event in same transaction
|
|
// VI: Thực thi business operation và lưu event trong cùng transaction
|
|
await prisma.$transaction(async (tx) => {
|
|
// Business operation
|
|
await tx.order.create({ data: orderData });
|
|
|
|
// Store event in outbox (same transaction)
|
|
await tx.outboxEvent.create({
|
|
data: {
|
|
eventType: 'order.created',
|
|
payload: orderData,
|
|
status: 'pending'
|
|
}
|
|
});
|
|
});
|
|
|
|
// EN: Background processor publishes events from outbox
|
|
// VI: Background processor publish events từ outbox
|
|
```
|
|
|
|
## Best Practices / Thực Hành Tốt
|
|
|
|
1. **Design Compensations**: Mỗi step cần compensation / Every step needs compensation
|
|
2. **Idempotent Steps**: Làm cho steps idempotent cho retries / Make steps idempotent for retries
|
|
3. **Conflict Resolution**: Định nghĩa chiến lược giải quyết / Define resolution strategies
|
|
4. **Monitoring**: Theo dõi saga execution / Track saga execution
|
|
|
|
## Skills Liên Quan
|
|
|
|
- [Event-Driven Architecture](./event-driven-architecture.md) - Event patterns / Các patterns event
|
|
- [Error Handling Patterns](./error-handling-patterns.md) - Error handling / Xử lý lỗi
|
|
- [Database & Prisma](./database-prisma.md) - Database patterns / Các patterns database
|
|
|
|
## Tài Nguyên
|
|
|
|
- [Saga Pattern](https://microservices.io/patterns/data/saga.html) - Saga pattern overview
|
|
- Skill Source: `.cursor/skills/data-consistency-patterns/SKILL.md` - Source file đầy đủ
|