16 KiB
16 KiB
name, description, compatibility, metadata
| name | description | compatibility | metadata | ||||
|---|---|---|---|---|---|---|---|
| outbox-pattern | Transactional Outbox Pattern - đảm bảo atomicity khi publish events. Use for reliable messaging, at-least-once delivery, và event consistency. | .NET 10+, EF Core, MassTransit, PostgreSQL |
|
Outbox Pattern / Transactional Outbox
Pattern đảm bảo tính nhất quán khi publish events từ microservices.
When to Use This Skill / Khi Nào Sử Dụng
Use this skill when:
- Publishing integration events reliably / Publish integration events tin cậy
- Ensuring at-least-once delivery / Đảm bảo gửi ít nhất một lần
- Avoiding dual-write problem / Tránh vấn đề dual-write
- Distributed transactions not feasible / Không khả thi dùng distributed transactions
Core Concepts / Khái Niệm Cốt Lõi
The Dual-Write Problem / Vấn Đề Dual-Write
┌─────────────────────────────────────────────────────────────┐
│ ❌ DUAL-WRITE PROBLEM │
├─────────────────────────────────────────────────────────────┤
│ │
│ Service A │
│ ┌─────────────────────────────────────────────┐ │
│ │ 1. await _db.SaveChangesAsync(); ✅ │ │
│ │ 2. await _bus.PublishAsync(event); ❌ FAIL! │ │
│ └─────────────────────────────────────────────┘ │
│ │
│ Result: DB updated BUT message NOT sent = INCONSISTENCY │
│ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│ ✅ OUTBOX PATTERN │
├─────────────────────────────────────────────────────────────┤
│ │
│ Service A │
│ ┌─────────────────────────────────────────────┐ │
│ │ BEGIN TRANSACTION │ │
│ │ 1. INSERT INTO Orders ... │ │
│ │ 2. INSERT INTO Outbox (event) │ │
│ │ COMMIT │ │
│ └─────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────┐ │
│ │ Outbox Processor (Background) │ │
│ │ 1. SELECT * FROM Outbox WHERE Sent = false │ │
│ │ 2. Publish to Message Bus │ │
│ │ 3. UPDATE Outbox SET Sent = true │ │
│ └─────────────────────────────────────────────┘ │
│ │
│ Result: ATOMIC - Both or Neither │
│ │
└─────────────────────────────────────────────────────────────┘
Outbox Flow / Luồng Outbox
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Command │───▶│ Handler │───▶│ DB + Outbox│───▶│ Processor │
│ (API) │ │ (Domain) │ │ (Same Tx) │ │ (Background)│
└─────────────┘ └─────────────┘ └─────────────┘ └──────┬──────┘
│
▼
┌─────────────┐
│ Message Bus │
│ (RabbitMQ) │
└─────────────┘
Key Components / Thành Phần Chính
| Component | Purpose | Implementation |
|---|---|---|
| Outbox Table | Store pending events | EF Core entity |
| Outbox Processor | Publish pending events | BackgroundService |
| Idempotency | Prevent duplicates | Event ID tracking |
Key Patterns / Mẫu Chính
Outbox Entity / Entity Outbox
/// <summary>
/// EN: Outbox message entity for transactional messaging.
/// VI: Entity outbox message cho transactional messaging.
/// </summary>
public class OutboxMessage
{
public Guid Id { get; set; }
public string EventType { get; set; } = default!;
public string Payload { get; set; } = default!;
public DateTime CreatedAt { get; set; }
public DateTime? ProcessedAt { get; set; }
public bool IsProcessed { get; set; }
public int RetryCount { get; set; }
public string? Error { get; set; }
}
DbContext Configuration / Cấu Hình DbContext
/// <summary>
/// EN: DbContext with outbox support.
/// VI: DbContext với hỗ trợ outbox.
/// </summary>
public class OrderDbContext : DbContext
{
public DbSet<Order> Orders => Set<Order>();
public DbSet<OutboxMessage> OutboxMessages => Set<OutboxMessage>();
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<OutboxMessage>(entity =>
{
entity.ToTable("OutboxMessages");
entity.HasKey(e => e.Id);
entity.Property(e => e.EventType)
.HasMaxLength(500)
.IsRequired();
entity.Property(e => e.Payload)
.HasColumnType("jsonb")
.IsRequired();
entity.HasIndex(e => new { e.IsProcessed, e.CreatedAt })
.HasFilter("\"IsProcessed\" = false");
});
}
}
Unit of Work with Outbox / Unit of Work với Outbox
/// <summary>
/// EN: Unit of Work that saves domain events to outbox.
/// VI: Unit of Work lưu domain events vào outbox.
/// </summary>
public interface IUnitOfWork
{
Task<int> SaveChangesAsync(CancellationToken ct = default);
}
public class UnitOfWork : IUnitOfWork
{
private readonly OrderDbContext _context;
private readonly ILogger<UnitOfWork> _logger;
public UnitOfWork(OrderDbContext context, ILogger<UnitOfWork> logger)
{
_context = context;
_logger = logger;
}
public async Task<int> SaveChangesAsync(CancellationToken ct = default)
{
// EN: Convert domain events to outbox messages
// VI: Chuyển domain events thành outbox messages
var aggregates = _context.ChangeTracker
.Entries<IAggregateRoot>()
.Where(e => e.Entity.DomainEvents.Any())
.Select(e => e.Entity)
.ToList();
foreach (var aggregate in aggregates)
{
foreach (var domainEvent in aggregate.DomainEvents)
{
var outboxMessage = new OutboxMessage
{
Id = Guid.NewGuid(),
EventType = domainEvent.GetType().AssemblyQualifiedName!,
Payload = JsonSerializer.Serialize(domainEvent, domainEvent.GetType()),
CreatedAt = DateTime.UtcNow,
IsProcessed = false
};
_context.OutboxMessages.Add(outboxMessage);
}
aggregate.ClearDomainEvents();
}
var result = await _context.SaveChangesAsync(ct);
_logger.LogDebug("Saved {Count} changes with outbox messages", result);
return result;
}
}
Outbox Processor / Outbox Processor
/// <summary>
/// EN: Background service that processes outbox messages.
/// VI: Background service xử lý outbox messages.
/// </summary>
public class OutboxProcessor : BackgroundService
{
private readonly IServiceScopeFactory _scopeFactory;
private readonly ILogger<OutboxProcessor> _logger;
private readonly TimeSpan _pollingInterval = TimeSpan.FromSeconds(5);
private const int BatchSize = 100;
private const int MaxRetries = 5;
public OutboxProcessor(
IServiceScopeFactory scopeFactory,
ILogger<OutboxProcessor> logger)
{
_scopeFactory = scopeFactory;
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken ct)
{
_logger.LogInformation("Outbox Processor started");
while (!ct.IsCancellationRequested)
{
try
{
await ProcessPendingMessagesAsync(ct);
}
catch (Exception ex)
{
_logger.LogError(ex, "Error processing outbox messages");
}
await Task.Delay(_pollingInterval, ct);
}
}
private async Task ProcessPendingMessagesAsync(CancellationToken ct)
{
using var scope = _scopeFactory.CreateScope();
var context = scope.ServiceProvider.GetRequiredService<OrderDbContext>();
var publishEndpoint = scope.ServiceProvider.GetRequiredService<IPublishEndpoint>();
var messages = await context.OutboxMessages
.Where(m => !m.IsProcessed && m.RetryCount < MaxRetries)
.OrderBy(m => m.CreatedAt)
.Take(BatchSize)
.ToListAsync(ct);
foreach (var message in messages)
{
try
{
var eventType = Type.GetType(message.EventType);
if (eventType == null)
{
_logger.LogWarning("Unknown event type: {EventType}", message.EventType);
continue;
}
var @event = JsonSerializer.Deserialize(message.Payload, eventType);
await publishEndpoint.Publish(@event!, eventType, ct);
message.IsProcessed = true;
message.ProcessedAt = DateTime.UtcNow;
_logger.LogDebug("Published outbox message {Id}", message.Id);
}
catch (Exception ex)
{
message.RetryCount++;
message.Error = ex.Message;
_logger.LogWarning(ex, "Failed to publish message {Id}, retry {Retry}",
message.Id, message.RetryCount);
}
}
await context.SaveChangesAsync(ct);
}
}
Command Handler with Outbox / Command Handler với Outbox
/// <summary>
/// EN: Command handler that raises domain events saved to outbox.
/// VI: Command handler phát domain events được lưu vào outbox.
/// </summary>
public class CreateOrderCommandHandler : IRequestHandler<CreateOrderCommand, OrderResult>
{
private readonly IOrderRepository _repository;
private readonly ILogger<CreateOrderCommandHandler> _logger;
public CreateOrderCommandHandler(
IOrderRepository repository,
ILogger<CreateOrderCommandHandler> logger)
{
_repository = repository;
_logger = logger;
}
public async Task<OrderResult> Handle(CreateOrderCommand request, CancellationToken ct)
{
var order = new Order(request.UserId, request.ShippingAddress);
foreach (var item in request.Items)
{
order.AddItem(item.ProductId, item.Quantity, item.UnitPrice);
}
// EN: Domain events are raised inside aggregate
// VI: Domain events được phát trong aggregate
await _repository.AddAsync(order, ct);
// EN: SaveChanges converts domain events to outbox messages
// VI: SaveChanges chuyển domain events thành outbox messages
await _repository.UnitOfWork.SaveChangesAsync(ct);
_logger.LogInformation("Order created: {OrderId}", order.Id);
return new OrderResult(order.Id);
}
}
Common Mistakes / Lỗi Thường Gặp
1. Publishing Before Saving
// ❌ BAD: Publish before DB commit
await _publishEndpoint.Publish(new OrderCreatedEvent(order.Id));
await _dbContext.Orders.AddAsync(order);
await _dbContext.SaveChangesAsync();
// If SaveChanges fails, message is already sent!
// ✅ GOOD: Use outbox pattern
order.AddDomainEvent(new OrderCreatedEvent(order.Id));
await _dbContext.Orders.AddAsync(order);
await _dbContext.SaveChangesAsync(); // Saves both order AND outbox in same transaction
2. No Idempotency in Consumers
// ❌ BAD: No idempotency check
public async Task Consume(ConsumeContext<OrderCreatedEvent> context)
{
await _service.ProcessOrder(context.Message.OrderId);
// May process duplicate messages!
}
// ✅ GOOD: Idempotent consumer
public async Task Consume(ConsumeContext<OrderCreatedEvent> context)
{
var eventId = context.MessageId;
if (await _processedMessages.ExistsAsync(eventId))
return; // Already processed
await _service.ProcessOrder(context.Message.OrderId);
await _processedMessages.MarkAsync(eventId);
}
3. No Retry Limit
// ❌ BAD: Infinite retries
var messages = await context.OutboxMessages
.Where(m => !m.IsProcessed) // Will keep retrying failed messages forever
.ToListAsync();
// ✅ GOOD: Retry limit with dead letter
var messages = await context.OutboxMessages
.Where(m => !m.IsProcessed && m.RetryCount < MaxRetries)
.ToListAsync();
// Move failed messages to dead letter after max retries
Quick Reference / Tham Chiếu Nhanh
Outbox Table Schema
| Column | Type | Purpose |
|---|---|---|
| Id | UUID | Primary key |
| EventType | VARCHAR(500) | Full type name for deserialization |
| Payload | JSONB | Serialized event data |
| CreatedAt | TIMESTAMP | When event was created |
| ProcessedAt | TIMESTAMP | When event was published |
| IsProcessed | BOOLEAN | Processing status |
| RetryCount | INT | Failed attempts |
| Error | TEXT | Last error message |
Delivery Guarantees
| Pattern | Guarantee | Trade-off |
|---|---|---|
| Direct publish | Best-effort | May lose messages |
| Outbox | At-least-once | May duplicate messages |
| Inbox + Outbox | Exactly-once | More complexity |
Resources / Tài Nguyên
- Detailed Examples - Full code examples
- Event Sourcing - Event-based persistence
- Inter-service Communication - MassTransit
- SAGA Pattern - Distributed transactions
- Repository Pattern - Unit of Work