pos-system/docs/en/skills/data-consistency-patterns.md

name, description

name	description
data-consistency-patterns	Data consistency patterns for distributed microservices including Saga patterns, distributed transactions, eventual consistency, compensation, and idempotency. Use when handling distributed transactions, implementing eventual consistency, or managing data synchronization across services.

Data Consistency Patterns

When to Use This Skill

Use this skill when:

• Implementing distributed transactions across multiple services
• Handling eventual consistency in microservices
• Implementing Saga patterns for distributed workflows
• Designing compensation strategies for failed transactions
• Implementing idempotent operations
• Managing data synchronization across services
• Handling conflict resolution

Core Concepts

ACID vs BASE

ACID (Traditional): Atomicity, Consistency, Isolation, Durability

BASE (Distributed): Basic Availability, Soft state, Eventual consistency

Consistency Models

• Strong Consistency: All nodes see same data at same time
• Eventual Consistency: System becomes consistent over time
• Weak Consistency: No guarantees about when consistency occurs

Key Patterns

Saga Orchestrator Pattern

Saga Orchestration Flow

The following diagram illustrates how a Saga orchestrator executes steps sequentially and handles compensation on failure:

sequenceDiagram
    participant Client
    participant Orchestrator
    participant Step1 as Step 1: Create Order
    participant Step2 as Step 2: Reserve Inventory
    participant Step3 as Step 3: Process Payment

    Client->>Orchestrator: Execute Saga
    Orchestrator->>Step1: Execute Step 1
    Step1-->>Orchestrator: Success (Order Created)
    Orchestrator->>Step2: Execute Step 2
    Step2-->>Orchestrator: Success (Inventory Reserved)
    Orchestrator->>Step3: Execute Step 3
    Step3-->>Orchestrator: Failure (Payment Failed)
    Orchestrator->>Step2: Compensate Step 2
    Step2-->>Orchestrator: Compensation Complete
    Orchestrator->>Step1: Compensate Step 1
    Step1-->>Orchestrator: Compensation Complete
    Orchestrator-->>Client: Saga Failed (Compensated)

Compensation Flow

When a step fails, the orchestrator compensates all previously completed steps in reverse order:

flowchart TD
    Start([Saga Execution Starts]) --> Step1[Execute Step 1]
    Step1 -->|Success| Step2[Execute Step 2]
    Step1 -->|Failure| Fail1[Saga Failed<br/>No Compensation Needed]
    
    Step2 -->|Success| Step3[Execute Step 3]
    Step2 -->|Failure| Comp1[Compensate Step 1]
    
    Step3 -->|Success| Complete([Saga Completed])
    Step3 -->|Failure| Comp2[Compensate Step 2]
    
    Comp2 --> Comp1
    Comp1 --> Fail2[Saga Failed<br/>All Steps Compensated]
    
    style Start fill:#e1f5ff
    style Complete fill:#d4edda
    style Fail1 fill:#f8d7da
    style Fail2 fill:#f8d7da
    style Comp1 fill:#fff3cd
    style Comp2 fill:#fff3cd

Eventual Consistency Flow

This diagram shows how data becomes consistent across services over time through event propagation:

flowchart LR
    subgraph ServiceA[Service A: Write Model]
        Write[Write Operation] --> EventStore[Event Store]
        EventStore --> Publish[Publish Event]
    end
    
    subgraph EventBus[Event Bus]
        Publish --> Queue[Event Queue]
    end
    
    subgraph ServiceB[Service B: Read Model]
        Queue --> Consume[Consume Event]
        Consume --> Update[Update Read Model]
        Update --> Consistent[Eventually Consistent]
    end
    
    subgraph ServiceC[Service C: Read Model]
        Queue --> Consume2[Consume Event]
        Consume2 --> Update2[Update Read Model]
        Update2 --> Consistent2[Eventually Consistent]
    end
    
    style Write fill:#e1f5ff
    style Consistent fill:#d4edda
    style Consistent2 fill:#d4edda
    style Queue fill:#fff3cd

// Centralized orchestrator coordinates steps
const saga = new SagaOrchestrator();
await saga.execute({
  sagaId: 'saga_123',
  steps: [
    { name: 'create-order', execute: createOrder, compensate: cancelOrder },
    { name: 'reserve-inventory', execute: reserveInventory, compensate: releaseInventory },
    { name: 'process-payment', execute: chargePayment, compensate: refundPayment },
  ],
  data: {},
  status: 'pending',
});

Saga Choreography Pattern

In choreography, services react to events without a central coordinator:

// Services react to events
eventConsumer.on('order.created', async (event) => {
  await inventoryService.reserve(event.data.items);
  await eventPublisher.publish('inventory.reserved', {...});
});

Saga Choreography Flow

The following diagram shows how services coordinate through events in a choreography pattern:

sequenceDiagram
    participant OrderService
    participant EventBus
    participant InventoryService
    participant PaymentService
    participant NotificationService

    OrderService->>EventBus: Publish order.created
    EventBus->>InventoryService: order.created event
    InventoryService->>InventoryService: Reserve Inventory
    InventoryService->>EventBus: Publish inventory.reserved
    EventBus->>PaymentService: inventory.reserved event
    PaymentService->>PaymentService: Process Payment
    PaymentService->>EventBus: Publish payment.processed
    EventBus->>NotificationService: payment.processed event
    NotificationService->>NotificationService: Send Confirmation
    
    Note over InventoryService,PaymentService: If payment fails,<br/>compensation events are published
    PaymentService->>EventBus: Publish payment.failed
    EventBus->>InventoryService: payment.failed event
    InventoryService->>InventoryService: Release Inventory
    EventBus->>OrderService: payment.failed event
    OrderService->>OrderService: Cancel Order

Idempotency

Idempotency ensures operations can be safely retried without side effects:

// Execute operation with idempotency check
await idempotencyHandler.execute(
  idempotencyKey,
  async () => await userService.create(data)
);

Idempotency Flow

flowchart TD
    Request[Client Request] --> Check{Idempotency Key<br/>Exists?}
    Check -->|Yes| Return[Return Cached Result]
    Check -->|No| Execute[Execute Operation]
    Execute --> Store[Store Result with Key]
    Store --> Return2[Return Result]
    
    Return --> Client[Client Response]
    Return2 --> Client
    
    style Check fill:#fff3cd
    style Return fill:#d4edda
    style Return2 fill:#d4edda

Optimistic Locking

Optimistic locking prevents lost updates using version fields:

// Update with version check
await optimisticLockService.updateWithLock(
  repository,
  id,
  (current) => ({ ...current, name: newName })
);

Optimistic Locking Flow

sequenceDiagram
    participant Client1
    participant Client2
    participant Service
    participant DB[(Database)]

    Client1->>Service: Read Entity (version=1)
    Service->>DB: SELECT * WHERE id=123
    DB-->>Service: Entity (version=1)
    Service-->>Client1: Entity Data
    
    Client2->>Service: Read Entity (version=1)
    Service->>DB: SELECT * WHERE id=123
    DB-->>Service: Entity (version=1)
    Service-->>Client2: Entity Data
    
    Client1->>Service: Update (version=1)
    Service->>DB: UPDATE WHERE id=123 AND version=1
    DB-->>Service: Success (version=2)
    
    Client2->>Service: Update (version=1)
    Service->>DB: UPDATE WHERE id=123 AND version=1
    DB-->>Service: Conflict (version=2 exists)
    Service-->>Client2: OptimisticLockError
    Note over Client2: Retry with new version

CQRS Pattern

Command Query Responsibility Segregation separates read and write operations for optimized performance and scalability.

CQRS Architecture Flow

The following diagram illustrates how CQRS separates write and read paths:

flowchart TB
    subgraph WritePath[Write Path]
        Command[Command] --> WriteModel[Write Model<br/>Normalized]
        WriteModel --> Event[Domain Event]
        Event --> EventStore[(Event Store)]
    end
    
    subgraph ReadPath[Read Path]
        Query[Query] --> ReadModel[Read Model<br/>Denormalized]
        ReadModel --> Response[Query Response]
    end
    
    subgraph Sync[Eventual Consistency]
        EventStore --> EventHandler[Event Handler]
        EventHandler --> Projection[Projection]
        Projection --> ReadModel
    end
    
    style WritePath fill:#e1f5ff
    style ReadPath fill:#d4edda
    style Sync fill:#fff3cd
    style EventStore fill:#f8d7da

// Write path: Command handler
await commandHandler.handle({
  type: 'CREATE_ORDER',
  data: { userId, items }
});

// Read path: Optimized query
const orders = await readModel.findOrdersByUser(userId);

Outbox Pattern

The Outbox pattern ensures reliable event publishing by storing events in the same database transaction as business data.

Outbox Pattern Flow

This diagram shows how the Outbox pattern guarantees event delivery:

sequenceDiagram
    participant Client
    participant Service
    participant DB[(Database)]
    participant OutboxTable[(Outbox Table)]
    participant Processor[Outbox Processor]
    participant EventBus[Event Bus]

    Client->>Service: Business Operation
    Service->>DB: Begin Transaction
    Service->>DB: Update Business Data
    Service->>OutboxTable: Insert Event (same transaction)
    Service->>DB: Commit Transaction
    
    Note over Service,OutboxTable: Event stored atomically<br/>with business data
    
    Processor->>OutboxTable: Poll for Unpublished Events
    OutboxTable-->>Processor: Return Events
    Processor->>EventBus: Publish Event
    EventBus-->>Processor: Publish Confirmed
    Processor->>OutboxTable: Mark as Published
    
    Note over Processor,EventBus: Background processor<br/>ensures delivery

// Execute business operation and store event in same transaction
await prisma.$transaction(async (tx) => {
  // Business operation
  await tx.order.create({ data: orderData });
  
  // Store event in outbox (same transaction)
  await tx.outboxEvent.create({
    data: {
      eventType: 'order.created',
      payload: orderData,
      status: 'pending'
    }
  });
});

// Background processor publishes events from outbox

Best Practices

1. Design Compensations: Every step needs compensation
2. Idempotent Steps: Make steps idempotent for retries
3. Conflict Resolution: Define resolution strategies
4. Monitoring: Track saga execution and consistency lag
5. Read Models: Use separate read models for queries (CQRS)

Resources

• Saga Pattern
• Event-Driven Architecture
• Error Handling Patterns
• Skill Source: .cursor/skills/data-consistency-patterns/SKILL.md