---
trigger: always_on
---

# Data Consistency Patterns

## When to Use This Skill

Use this skill when:
- Implementing distributed transactions across multiple services
- Handling eventual consistency in microservices
- Implementing Saga patterns for distributed workflows
- Designing compensation strategies for failed transactions
- Implementing idempotent operations
- Managing data synchronization across services
- Handling conflict resolution
- Implementing optimistic locking strategies
- Building event sourcing systems
- Ensuring data integrity in distributed systems

## Core Concepts

### ACID vs BASE

**ACID (Traditional Databases):**
- Atomicity: All or nothing
- Consistency: Data always valid
- Isolation: Concurrent transactions isolated
- Durability: Committed changes persist

**BASE (Distributed Systems):**
- Basic Availability: System available most of the time
- Soft state: State may change over time
- Eventual consistency: Consistency achieved eventually

### Consistency Models

| Model | Description | Use Case |
|-------|-------------|----------|
| **Strong** | All nodes see same data at same time | Financial transactions |
| **Weak** | No guarantees about when consistency occurs | Caching |
| **Eventual** | System becomes consistent over time | Read models, analytics |
| **Causal** | Related operations maintain order | User sessions |

### Distributed Transaction Challenges

- Network partitions
- Service failures
- Clock synchronization
- Partial failures
- Two-Phase Commit (2PC) limitations

## Saga Pattern

### Orchestration vs Choreography

| Approach | Central Control | Resilience | Complexity | Best For |
|----------|-----------------|------------|------------|----------|
| **Orchestration** | Yes (single coordinator) | Lower (SPOF) | Easier to debug | Complex workflows |
| **Choreography** | No (event-driven) | Higher | Harder to trace | Simple flows, loose coupling |

### Saga Execution Flow

```
Execute: Step1 -> Step2 -> Step3 -> Complete
            |        |        |
            v        v        v
Compensate: <- Step2.undo <- Step1.undo (on failure)
```

### Key Saga Interfaces

```typescript
interface SagaStep {
  name: string;
  execute: () => Promise<any>;
  compensate: (context: any) => Promise<void>;
  retry?: number;
}

interface SagaContext {
  sagaId: string;
  steps: SagaStep[];
  currentStep: number;
  data: Record<string, any>;
  status: 'pending' | 'running' | 'completed' | 'compensating' | 'failed';
}
```

See [./references/REFERENCE.md](./references/REFERENCE.md) for full Saga orchestrator implementation.

## Idempotency Pattern

Ensures operations can be safely retried without side effects.

### Key Concepts

- **Idempotency Key**: Unique identifier for each operation
- **Result Storage**: Cache results to return on duplicates
- **TTL**: Time-to-live for idempotency records

### Pattern

```typescript
const key = `${operation}:${userId}:${hash(requestData)}`;
await idempotencyHandler.execute(key, () => operation());
```

See [./references/REFERENCE.md](./references/REFERENCE.md) for full implementation.

## Optimistic Locking

Prevents lost updates in concurrent scenarios using version fields.

### Pattern

```typescript
await prisma.entity.update({
  where: { id, version: currentVersion },
  data: { ...updates, version: { increment: 1 } }
});
```

### Prisma Schema

```prisma
model Entity {
  id      String @id @default(cuid())
  version Int    @default(1)
  // ... other fields
}
```

See [./references/REFERENCE.md](./references/REFERENCE.md) for full optimistic locking service.

## CQRS Pattern

Command Query Responsibility Segregation separates read and write operations.

### Architecture

```
Write Path: Command -> Write Model -> Event -> Event Store
Read Path:  Query -> Read Model (denormalized, optimized for reads)
```

### Key Benefits

- Optimized read models for query performance
- Independent scaling of read/write operations
- Eventual consistency between models

See [./references/REFERENCE.md](./references/REFERENCE.md) for implementation details.

## Conflict Resolution Strategies

| Strategy | Description | Use Case |
|----------|-------------|----------|
| **Last Write Wins** | Latest timestamp wins | Simple scenarios |
| **First Write Wins** | Earliest timestamp wins | Preserving original data |
| **Merge** | Combine both versions | Non-conflicting fields |
| **Manual** | Store for human review | Critical data conflicts |

## Outbox Pattern

Ensures reliable event publishing by storing events in the same transaction as business data.

### Flow

1. Execute business operation in transaction
2. Store event in outbox table (same transaction)
3. Background processor publishes and marks as sent

See [./references/REFERENCE.md](./references/REFERENCE.md) for implementation.

## Best Practices

### Saga Pattern
- Design compensations for every step
- Make steps idempotent for retries
- Set timeouts for saga execution
- Monitor saga execution and compensation
- Choose orchestration for complex workflows, choreography for simple ones

### Idempotency
- Generate keys from request data
- Store keys with results
- Set appropriate TTL for records
- Regularly clean expired records

### Eventual Consistency
- Accept that consistency is eventual
- Use separate read models for queries
- Define resolution strategies upfront
- Monitor consistency lag

### Optimistic Locking
- Add version fields to entities
- Implement retry with exponential backoff
- Handle conflicts gracefully
- Use for read-heavy workloads

## Common Mistakes

### No Compensation Logic
```typescript
// BAD: No compensation
steps: [{ execute: () => createOrder() }]

// GOOD: Always define compensation
steps: [{
  execute: () => createOrder(),
  compensate: (ctx) => cancelOrder(ctx.orderId)
}]
```

### Missing Idempotency
```typescript
// BAD: Creates duplicate on retry
await createPayment(orderId);

// GOOD: Idempotent check
const existing = await findPayment(idempotencyKey);
if (existing) return existing;
await createPayment(orderId);
```

### Ignoring Partial Failures
```typescript
// BAD: No error handling
await step1(); await step2(); await step3();

// GOOD: Saga orchestration
await sagaOrchestrator.execute(context);
```

### No Version Field
```prisma
// GOOD: Add version for optimistic locking
model Entity {
  version Int @default(1)
}
```

## Quick Reference

| Pattern | Use Case | Complexity |
|---------|----------|------------|
| **Saga (Orchestrated)** | Complex multi-step workflows | High |
| **Saga (Choreography)** | Simple event-driven flows | Medium |
| **Outbox Pattern** | Guaranteed event publishing | Medium |
| **Idempotency** | Retry-safe operations | Low |
| **Optimistic Lock** | Concurrent updates | Low |
| **CQRS** | Read/write optimization | High |
| **Dead Letter Queue** | Failed message handling | Medium |

## Resources

### Internal References
- [Detailed Code Examples](./references/REFERENCE.md) - Full implementations for all patterns
- [Event-Driven Architecture](../event-driven-architecture/SKILL.md) - Event patterns
- [Error Handling Patterns](../error-handling-patterns/SKILL.md) - Error handling
- [Database & Prisma](../database-prisma/SKILL.md) - Database patterns
- [Project Rules](../project-rules/SKILL.md) - GoodGo coding standards

### External Resources
- [Saga Pattern](https://microservices.io/patterns/data/saga.html) - Saga pattern overview
- [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html) - Event sourcing pattern
- [CQRS Pattern](https://martinfowler.com/bliki/CQRS.html) - CQRS pattern