Merge feat/goo-223-export-caps-streaming into master
Some checks failed
Security Scanning / Security Gate (push) Has been cancelled
CI / Lint → Typecheck → Test → Build (22) (push) Failing after 9s
CI / E2E Tests (push) Has been skipped
CI / AI Services (Python) — Smoke (push) Failing after 4s
Deploy / Build API Image (push) Failing after 21s
Deploy / Build Web Image (push) Failing after 13s
Deploy / Build AI Services Image (push) Failing after 9s
Deploy / Deploy to Staging (push) Has been skipped
Deploy / Deploy to Production (push) Has been skipped
Deploy / Rollback Staging (push) Has been skipped
Deploy / Rollback Production (push) Has been skipped
E2E Tests / Playwright E2E (push) Failing after 10s
Deploy / Smoke Test Staging (push) Has been skipped
Deploy / Smoke Test Production (push) Has been skipped
Backup Verification / Backup Restore Verification (push) Failing after 11s
CodeQL Analysis / CodeQL (javascript-typescript) (push) Failing after 1m1s
Security Scanning / Trivy Scan — API Image (push) Failing after 1m54s
Security Scanning / Trivy Scan — Web Image (push) Failing after 53s
Security Scanning / Trivy Scan — AI Services Image (push) Failing after 47s
Security Scanning / Trivy Filesystem Scan (push) Failing after 39s
Security Scanning / Dependency Audit (pnpm) (push) Failing after 11m9s

Brings GOO-172 Phase 0 RFC-004 foundations onto master:
- libs/contracts/events/ (envelope, event-types, schemas)
- apps/api/src/modules/shared/infrastructure/event-bus/
- apps/api/src/modules/shared/infrastructure/outbox/
- prisma/migrations/20260423140000_add_event_outbox/

Also includes GOO-222 (POI COUNT collapse) and GOO-223 (export-user-data caps + streaming).

Unblocks GOO-174 Phase 2 work that depends on Phase 0 contracts being on master.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Ho Ngoc Hai
2026-04-24 14:32:18 +07:00
36 changed files with 1568 additions and 98 deletions

View File

@@ -0,0 +1,50 @@
# Observability — Read-Model / Projector (RFC-003 Phase 0)
Grafana dashboards and wiring notes for the read-model observability stack
introduced in [GOO-192](/GOO/issues/GOO-192) under [GOO-94](/GOO/issues/GOO-94) §6 Phase 0.
## Metrics
All metrics live in the existing NestJS `metrics/` module
(`apps/api/src/modules/metrics/`) and are scraped via the standard `/metrics`
endpoint.
| Metric | Type | Labels | Purpose |
| --------------------------------------- | --------- | --------- | --------------------------------------------------------- |
| `read_model_projector_lag_seconds` | Gauge | `handler` | Seconds between latest source event and projector cursor. |
| `read_model_refresh_duration_seconds` | Histogram | `view` | Duration of read-model / materialised view refreshes. |
| `read_model_reconciliation_drift_total` | Counter | `model` | Count of drift discrepancies found during reconciliation. |
### Emit points
Inject `MetricsService` and call:
```ts
metrics.setProjectorLag(handler, lagSeconds);
metrics.recordReadModelRefresh(view, durationSeconds);
metrics.recordReconciliationDrift(model, count?);
```
## Dashboard
- File: `read-models-dashboard.json` (Grafana schema v38).
- Import into Grafana (`Dashboards → Import → Upload JSON`), pick the Prometheus
data source.
- Variables: `handler`, `view`, `model` — derived from Prometheus label values.
- Panels:
1. Projector lag by handler (time series + thresholded)
2. Max projector lag (stat, RAG 30s / 120s)
3. Refresh duration p50/p95 by view
4. Refresh throughput (refreshes/sec) by view
5. Reconciliation drift rate by model (15m rate)
6. Total drift events in last 24h (stat, RAG 1 / 10)
## Local verification
```bash
pnpm --filter @goodgo/api dev
curl -s http://localhost:3001/metrics | grep read_model_
```
All three metric families should appear with `# HELP` / `# TYPE` headers even
before any samples are recorded.

View File

@@ -0,0 +1,77 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"graphTooltip": 1,
"id": null,
"uid": "goodgo-read-models",
"title": "GoodGo · Read-Model Observability (RFC-003 Phase 0)",
"tags": ["goodgo", "rfc-003", "read-models", "observability"],
"timezone": "browser",
"schemaVersion": 38,
"version": 1,
"refresh": "30s",
"time": { "from": "now-6h", "to": "now" },
"templating": {
"list": [
{ "name": "datasource", "type": "datasource", "query": "prometheus", "current": { "text": "Prometheus", "value": "Prometheus" } },
{ "name": "handler", "type": "query", "datasource": "${datasource}", "query": "label_values(read_model_projector_lag_seconds, handler)", "includeAll": true, "multi": true, "refresh": 2 },
{ "name": "view", "type": "query", "datasource": "${datasource}", "query": "label_values(read_model_refresh_duration_seconds_bucket, view)", "includeAll": true, "multi": true, "refresh": 2 },
{ "name": "model", "type": "query", "datasource": "${datasource}", "query": "label_values(read_model_reconciliation_drift_total, model)", "includeAll": true, "multi": true, "refresh": 2 }
]
},
"panels": [
{
"id": 1, "type": "timeseries", "title": "Projector lag (seconds) — by handler",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 0, "y": 0 },
"fieldConfig": { "defaults": { "unit": "s", "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 30 }, { "color": "red", "value": 120 }] } } },
"targets": [{ "expr": "read_model_projector_lag_seconds{handler=~\"$handler\"}", "legendFormat": "{{handler}}", "refId": "A" }]
},
{
"id": 2, "type": "stat", "title": "Max projector lag (current)",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 12, "y": 0 },
"fieldConfig": { "defaults": { "unit": "s", "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 30 }, { "color": "red", "value": 120 }] } } },
"options": { "reduceOptions": { "calcs": ["lastNotNull"] } },
"targets": [{ "expr": "max(read_model_projector_lag_seconds{handler=~\"$handler\"})", "refId": "A" }]
},
{
"id": 3, "type": "timeseries", "title": "Refresh duration p50/p95 — by view",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 0, "y": 8 },
"fieldConfig": { "defaults": { "unit": "s" } },
"targets": [
{ "expr": "histogram_quantile(0.95, sum by (view, le) (rate(read_model_refresh_duration_seconds_bucket{view=~\"$view\"}[5m])))", "legendFormat": "p95 · {{view}}", "refId": "A" },
{ "expr": "histogram_quantile(0.50, sum by (view, le) (rate(read_model_refresh_duration_seconds_bucket{view=~\"$view\"}[5m])))", "legendFormat": "p50 · {{view}}", "refId": "B" }
]
},
{
"id": 4, "type": "timeseries", "title": "Refresh throughput (refreshes/sec) — by view",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 12, "y": 8 },
"fieldConfig": { "defaults": { "unit": "ops" } },
"targets": [{ "expr": "sum by (view) (rate(read_model_refresh_duration_seconds_count{view=~\"$view\"}[5m]))", "legendFormat": "{{view}}", "refId": "A" }]
},
{
"id": 5, "type": "timeseries", "title": "Reconciliation drift rate — by model",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 0, "y": 16 },
"fieldConfig": { "defaults": { "unit": "ops" } },
"targets": [{ "expr": "sum by (model) (rate(read_model_reconciliation_drift_total{model=~\"$model\"}[15m]))", "legendFormat": "{{model}}", "refId": "A" }]
},
{
"id": 6, "type": "stat", "title": "Total drift events (last 24h)",
"datasource": "${datasource}", "gridPos": { "h": 8, "w": 12, "x": 12, "y": 16 },
"fieldConfig": { "defaults": { "unit": "short", "thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }, { "color": "yellow", "value": 1 }, { "color": "red", "value": 10 }] } } },
"options": { "reduceOptions": { "calcs": ["lastNotNull"] } },
"targets": [{ "expr": "sum by (model) (increase(read_model_reconciliation_drift_total{model=~\"$model\"}[24h]))", "legendFormat": "{{model}}", "refId": "A" }]
}
]
}

View File

@@ -17,6 +17,7 @@
"@aws-sdk/client-s3": "^3.1026.0", "@aws-sdk/client-s3": "^3.1026.0",
"@aws-sdk/s3-request-presigner": "^3.1026.0", "@aws-sdk/s3-request-presigner": "^3.1026.0",
"@goodgo/mcp-servers": "workspace:*", "@goodgo/mcp-servers": "workspace:*",
"@goodgo/contracts-events": "workspace:*",
"@nest-lab/throttler-storage-redis": "^1.2.0", "@nest-lab/throttler-storage-redis": "^1.2.0",
"@nestjs/bullmq": "^11.0.4", "@nestjs/bullmq": "^11.0.4",
"@nestjs/common": "^11.0.0", "@nestjs/common": "^11.0.0",

View File

@@ -4,11 +4,16 @@ import {
PrismaNeighborhoodScoreService, PrismaNeighborhoodScoreService,
} from '../services/neighborhood-score.service'; } from '../services/neighborhood-score.service';
// Helper: build the flat $queryRaw row list that countPOIs expects.
function makePoiRows(counts: Record<string, number>) {
return Object.entries(counts).map(([type, n]) => ({ type, count: BigInt(n) }));
}
describe('NeighborhoodScoreServiceImpl', () => { describe('NeighborhoodScoreServiceImpl', () => {
let service: NeighborhoodScoreServiceImpl; let service: NeighborhoodScoreServiceImpl;
let mockPrisma: { let mockPrisma: {
neighborhoodScore: { findUnique: ReturnType<typeof vi.fn>; upsert: ReturnType<typeof vi.fn> }; neighborhoodScore: { findUnique: ReturnType<typeof vi.fn>; upsert: ReturnType<typeof vi.fn> };
pOI: { count: ReturnType<typeof vi.fn> }; $queryRaw: ReturnType<typeof vi.fn>;
}; };
let mockLogger: { log: ReturnType<typeof vi.fn> }; let mockLogger: { log: ReturnType<typeof vi.fn> };
@@ -18,7 +23,7 @@ describe('NeighborhoodScoreServiceImpl', () => {
findUnique: vi.fn(), findUnique: vi.fn(),
upsert: vi.fn(), upsert: vi.fn(),
}, },
pOI: { count: vi.fn() }, $queryRaw: vi.fn(),
}; };
mockLogger = { log: vi.fn() }; mockLogger = { log: vi.fn() };
@@ -60,44 +65,45 @@ describe('NeighborhoodScoreServiceImpl', () => {
}); });
describe('calculateAndSave', () => { describe('calculateAndSave', () => {
it('calculates scores from POI counts and upserts', async () => { it('issues exactly one DB query and calculates scores correctly', async () => {
// Simulate POI counts: education=15 (max), healthcare=4 (50%), transport=6 (50%), mockPrisma.$queryRaw.mockResolvedValue(
// shopping=5 (50%), greenery=3 (50%), safety=2 (50%) makePoiRows({
const poiCountsByCategory = [15, 4, 6, 5, 3, 2]; SCHOOL: 10, UNIVERSITY: 5,
let callIndex = 0; HOSPITAL: 2, CLINIC: 2,
mockPrisma.pOI.count.mockImplementation(() => { METRO_STATION: 3, BUS_STOP: 3,
return Promise.resolve(poiCountsByCategory[callIndex++]!); MALL: 2, MARKET: 2, SUPERMARKET: 1,
}); PARK: 3,
POLICE_STATION: 1, FIRE_STATION: 1,
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => { }),
return Promise.resolve(create); );
}); mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
Promise.resolve(create),
);
const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh'); const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh');
// education: 15/15 * 10 = 10 → 10 * 20/10 = 20
// healthcare: 4/8 * 10 = 5 → 5 * 20/10 = 10
// transport: 6/12 * 10 = 5 → 5 * 20/10 = 10
// shopping: 5/10 * 10 = 5 → 5 * 15/10 = 7.5
// greenery: 3/6 * 10 = 5 → 5 * 15/10 = 7.5
// safety: 2/4 * 10 = 5 → 5 * 10/10 = 5
// total = 20 + 10 + 10 + 7.5 + 7.5 + 5 = 60
expect(result.educationScore).toBe(10); expect(result.educationScore).toBe(10);
expect(result.healthcareScore).toBe(5); expect(result.healthcareScore).toBe(5);
expect(result.totalScore).toBe(60); expect(result.totalScore).toBe(60);
// Assert single DB round-trip for all 6 categories
expect(mockPrisma.$queryRaw).toHaveBeenCalledTimes(1);
expect(mockPrisma.neighborhoodScore.upsert).toHaveBeenCalledTimes(1); expect(mockPrisma.neighborhoodScore.upsert).toHaveBeenCalledTimes(1);
}); });
it('caps category scores at 10', async () => { it('caps category scores at 10', async () => {
// All categories have way more POIs than max mockPrisma.$queryRaw.mockResolvedValue(
mockPrisma.pOI.count.mockResolvedValue(100); makePoiRows({
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => { SCHOOL: 100, UNIVERSITY: 100, HOSPITAL: 100, CLINIC: 100,
return Promise.resolve(create); METRO_STATION: 100, BUS_STOP: 100, MALL: 100, MARKET: 100,
}); SUPERMARKET: 100, PARK: 100, POLICE_STATION: 100, FIRE_STATION: 100,
}),
);
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
Promise.resolve(create),
);
const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh'); const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh');
// All scores capped at 10 → total = sum of weights = 100
expect(result.educationScore).toBe(10); expect(result.educationScore).toBe(10);
expect(result.healthcareScore).toBe(10); expect(result.healthcareScore).toBe(10);
expect(result.transportScore).toBe(10); expect(result.transportScore).toBe(10);
@@ -105,25 +111,27 @@ describe('NeighborhoodScoreServiceImpl', () => {
expect(result.greeneryScore).toBe(10); expect(result.greeneryScore).toBe(10);
expect(result.safetyScore).toBe(10); expect(result.safetyScore).toBe(10);
expect(result.totalScore).toBe(100); expect(result.totalScore).toBe(100);
expect(mockPrisma.$queryRaw).toHaveBeenCalledTimes(1);
}); });
it('returns 0 scores when no POIs exist', async () => { it('returns 0 scores when no POIs exist', async () => {
mockPrisma.pOI.count.mockResolvedValue(0); mockPrisma.$queryRaw.mockResolvedValue([]);
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => { mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
return Promise.resolve(create); Promise.resolve(create),
}); );
const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh'); const result = await service.calculateAndSave('Quận 1', 'Hồ Chí Minh');
expect(result.educationScore).toBe(0); expect(result.educationScore).toBe(0);
expect(result.totalScore).toBe(0); expect(result.totalScore).toBe(0);
expect(mockPrisma.$queryRaw).toHaveBeenCalledTimes(1);
}); });
it('logs the calculated score', async () => { it('logs the calculated score', async () => {
mockPrisma.pOI.count.mockResolvedValue(5); mockPrisma.$queryRaw.mockResolvedValue(makePoiRows({ SCHOOL: 5 }));
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => { mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
return Promise.resolve(create); Promise.resolve(create),
}); );
await service.calculateAndSave('Quận 1', 'Hồ Chí Minh'); await service.calculateAndSave('Quận 1', 'Hồ Chí Minh');
@@ -140,7 +148,7 @@ describe('HttpNeighborhoodScoreService', () => {
let prismaFallback: PrismaNeighborhoodScoreService; let prismaFallback: PrismaNeighborhoodScoreService;
let mockPrisma: { let mockPrisma: {
neighborhoodScore: { findUnique: ReturnType<typeof vi.fn>; upsert: ReturnType<typeof vi.fn> }; neighborhoodScore: { findUnique: ReturnType<typeof vi.fn>; upsert: ReturnType<typeof vi.fn> };
pOI: { count: ReturnType<typeof vi.fn> }; $queryRaw: ReturnType<typeof vi.fn>;
}; };
let mockLogger: { log: ReturnType<typeof vi.fn>; warn: ReturnType<typeof vi.fn> }; let mockLogger: { log: ReturnType<typeof vi.fn>; warn: ReturnType<typeof vi.fn> };
let mockAiClient: { scoreNeighborhood: ReturnType<typeof vi.fn> }; let mockAiClient: { scoreNeighborhood: ReturnType<typeof vi.fn> };
@@ -148,7 +156,7 @@ describe('HttpNeighborhoodScoreService', () => {
beforeEach(() => { beforeEach(() => {
mockPrisma = { mockPrisma = {
neighborhoodScore: { findUnique: vi.fn(), upsert: vi.fn() }, neighborhoodScore: { findUnique: vi.fn(), upsert: vi.fn() },
pOI: { count: vi.fn() }, $queryRaw: vi.fn(),
}; };
mockLogger = { log: vi.fn(), warn: vi.fn() }; mockLogger = { log: vi.fn(), warn: vi.fn() };
mockAiClient = { scoreNeighborhood: vi.fn() }; mockAiClient = { scoreNeighborhood: vi.fn() };
@@ -165,7 +173,7 @@ describe('HttpNeighborhoodScoreService', () => {
}); });
it('persists AI service response when scoreNeighborhood succeeds', async () => { it('persists AI service response when scoreNeighborhood succeeds', async () => {
mockPrisma.pOI.count.mockResolvedValue(6); mockPrisma.$queryRaw.mockResolvedValue(makePoiRows({ SCHOOL: 6 }));
mockAiClient.scoreNeighborhood.mockResolvedValue({ mockAiClient.scoreNeighborhood.mockResolvedValue({
district: 'Quận 1', district: 'Quận 1',
city: 'Hồ Chí Minh', city: 'Hồ Chí Minh',
@@ -179,7 +187,9 @@ describe('HttpNeighborhoodScoreService', () => {
poi_counts: { education: 6, healthcare: 6, transport: 6, shopping: 6, greenery: 6, safety: 6 }, poi_counts: { education: 6, healthcare: 6, transport: 6, shopping: 6, greenery: 6, safety: 6 },
algorithm_version: 'neighborhood-heuristic-v1', algorithm_version: 'neighborhood-heuristic-v1',
}); });
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => Promise.resolve(create)); mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
Promise.resolve(create),
);
const result = await httpService.calculateAndSave('Quận 1', 'Hồ Chí Minh'); const result = await httpService.calculateAndSave('Quận 1', 'Hồ Chí Minh');
@@ -187,12 +197,15 @@ describe('HttpNeighborhoodScoreService', () => {
expect(result.totalScore).toBe(71.2); expect(result.totalScore).toBe(71.2);
expect(result.educationScore).toBe(8.5); expect(result.educationScore).toBe(8.5);
expect(mockPrisma.neighborhoodScore.upsert).toHaveBeenCalledOnce(); expect(mockPrisma.neighborhoodScore.upsert).toHaveBeenCalledOnce();
expect(mockPrisma.$queryRaw).toHaveBeenCalledTimes(1);
}); });
it('falls back to prisma scoring when AI service throws', async () => { it('falls back to prisma scoring when AI service throws', async () => {
mockPrisma.pOI.count.mockResolvedValue(0); mockPrisma.$queryRaw.mockResolvedValue([]);
mockAiClient.scoreNeighborhood.mockRejectedValue(new Error('AI service down')); mockAiClient.scoreNeighborhood.mockRejectedValue(new Error('AI service down'));
mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }) => Promise.resolve(create)); mockPrisma.neighborhoodScore.upsert.mockImplementation(({ create }: { create: unknown }) =>
Promise.resolve(create),
);
const result = await httpService.calculateAndSave('Quận 7', 'Hồ Chí Minh'); const result = await httpService.calculateAndSave('Quận 7', 'Hồ Chí Minh');

View File

@@ -143,18 +143,26 @@ async function countPOIs(
district: string, district: string,
city: string, city: string,
): Promise<AiNeighborhoodPOICounts> { ): Promise<AiNeighborhoodPOICounts> {
const entries = await Promise.all( // Single GROUP BY query replaces 6x individual COUNT queries.
CATEGORY_KEYS.map(async (cat) => { const rows = await prisma.$queryRaw<{ type: POIType; count: bigint }[]>`
const count = await prisma.pOI.count({ SELECT "type", COUNT(*) AS count
where: { FROM "POI"
district, WHERE "district" = ${district} AND "city" = ${city}
city, GROUP BY "type"
type: { in: CATEGORY_POI_TYPES[cat] }, `;
},
}); const typeCountMap = new Map<POIType, number>();
return [cat, count] as const; for (const row of rows) {
}), typeCountMap.set(row.type, Number(row.count));
}
const entries = CATEGORY_KEYS.map((cat) => {
const total = CATEGORY_POI_TYPES[cat].reduce(
(sum, t) => sum + (typeCountMap.get(t) ?? 0),
0,
); );
return [cat, total] as const;
});
return Object.fromEntries(entries) as unknown as AiNeighborhoodPOICounts; return Object.fromEntries(entries) as unknown as AiNeighborhoodPOICounts;
} }

View File

@@ -1,7 +1,16 @@
import { PayloadTooLargeException } from '@nestjs/common';
import { NotFoundException } from '@modules/shared'; import { NotFoundException } from '@modules/shared';
import { ExportUserDataCommand } from '../commands/export-user-data/export-user-data.command'; import { ExportUserDataCommand } from '../commands/export-user-data/export-user-data.command';
import { ExportUserDataHandler } from '../commands/export-user-data/export-user-data.handler'; import { ExportUserDataHandler } from '../commands/export-user-data/export-user-data.handler';
async function readStream(stream: NodeJS.ReadableStream): Promise<string> {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk as string));
}
return Buffer.concat(chunks).toString('utf8');
}
describe('ExportUserDataHandler', () => { describe('ExportUserDataHandler', () => {
let handler: ExportUserDataHandler; let handler: ExportUserDataHandler;
@@ -17,7 +26,13 @@ describe('ExportUserDataHandler', () => {
transaction: { findMany: vi.fn() }, transaction: { findMany: vi.fn() },
}; };
const mockLogger = { log: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn(), verbose: vi.fn() }; const mockLogger = {
log: vi.fn(),
warn: vi.fn(),
error: vi.fn(),
debug: vi.fn(),
verbose: vi.fn(),
};
const sampleUser = { const sampleUser = {
id: 'user-1', id: 'user-1',
@@ -29,12 +44,25 @@ describe('ExportUserDataHandler', () => {
createdAt: new Date('2025-01-01'), createdAt: new Date('2025-01-01'),
}; };
function setupEmptyRelations() {
mockPrisma.agent.findUnique.mockResolvedValue(null);
mockPrisma.listing.findMany.mockResolvedValue([]);
mockPrisma.payment.findMany.mockResolvedValue([]);
mockPrisma.subscription.findFirst.mockResolvedValue(null);
mockPrisma.review.findMany.mockResolvedValue([]);
mockPrisma.inquiry.findMany.mockResolvedValue([]);
mockPrisma.savedSearch.findMany.mockResolvedValue([]);
mockPrisma.transaction.findMany.mockResolvedValue([]);
}
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
delete process.env['EXPORT_ROW_CAP'];
delete process.env['EXPORT_SIZE_CAP_MB'];
handler = new ExportUserDataHandler(mockPrisma as any, mockLogger as any); handler = new ExportUserDataHandler(mockPrisma as any, mockLogger as any);
}); });
it('exports all user data including relations', async () => { it('exports all user data including relations and returns a stream', async () => {
mockPrisma.user.findUnique.mockResolvedValue(sampleUser); mockPrisma.user.findUnique.mockResolvedValue(sampleUser);
mockPrisma.agent.findUnique.mockResolvedValue({ id: 'agent-1', userId: 'user-1' }); mockPrisma.agent.findUnique.mockResolvedValue({ id: 'agent-1', userId: 'user-1' });
mockPrisma.listing.findMany.mockResolvedValue([{ id: 'listing-1' }]); mockPrisma.listing.findMany.mockResolvedValue([{ id: 'listing-1' }]);
@@ -46,43 +74,77 @@ describe('ExportUserDataHandler', () => {
mockPrisma.transaction.findMany.mockResolvedValue([{ id: 'tx-1' }]); mockPrisma.transaction.findMany.mockResolvedValue([{ id: 'tx-1' }]);
const result = await handler.execute(new ExportUserDataCommand('user-1')); const result = await handler.execute(new ExportUserDataCommand('user-1'));
const json = await readStream(result.stream);
const parsed = JSON.parse(json);
expect(result.user).toEqual(sampleUser); expect(parsed.user).toMatchObject({ id: 'user-1' });
expect(result.agent).toEqual({ id: 'agent-1', userId: 'user-1' }); expect(parsed.agent).toEqual({ id: 'agent-1', userId: 'user-1' });
expect(result.listings).toHaveLength(1); expect(parsed.listings).toHaveLength(1);
expect(result.payments).toHaveLength(1); expect(parsed.payments).toHaveLength(1);
expect(result.subscription).toEqual({ id: 'sub-1', status: 'ACTIVE' }); expect(parsed.subscription).toEqual({ id: 'sub-1', status: 'ACTIVE' });
expect(result.reviews).toHaveLength(1); expect(parsed.reviews).toHaveLength(1);
expect(result.inquiries).toHaveLength(1); expect(parsed.inquiries).toHaveLength(1);
expect(result.savedSearches).toHaveLength(1); expect(parsed.savedSearches).toHaveLength(1);
expect(result.transactions).toHaveLength(1); expect(parsed.transactions).toHaveLength(1);
expect(result.truncated).toBe(false);
}); });
it('throws NotFoundException if user not found', async () => { it('throws NotFoundException if user not found', async () => {
mockPrisma.user.findUnique.mockResolvedValue(null); mockPrisma.user.findUnique.mockResolvedValue(null);
await expect( await expect(handler.execute(new ExportUserDataCommand('missing'))).rejects.toThrow(
handler.execute(new ExportUserDataCommand('missing')), NotFoundException,
).rejects.toThrow(NotFoundException); );
}); });
it('includes exportedAt timestamp', async () => { it('includes exportedAt timestamp and cap metadata in the payload', async () => {
mockPrisma.user.findUnique.mockResolvedValue(sampleUser); mockPrisma.user.findUnique.mockResolvedValue(sampleUser);
mockPrisma.agent.findUnique.mockResolvedValue(null); setupEmptyRelations();
mockPrisma.listing.findMany.mockResolvedValue([]);
mockPrisma.payment.findMany.mockResolvedValue([]);
mockPrisma.subscription.findFirst.mockResolvedValue(null);
mockPrisma.review.findMany.mockResolvedValue([]);
mockPrisma.inquiry.findMany.mockResolvedValue([]);
mockPrisma.savedSearch.findMany.mockResolvedValue([]);
mockPrisma.transaction.findMany.mockResolvedValue([]);
const before = new Date().toISOString(); const before = new Date().toISOString();
const result = await handler.execute(new ExportUserDataCommand('user-1')); const result = await handler.execute(new ExportUserDataCommand('user-1'));
const after = new Date().toISOString(); const after = new Date().toISOString();
const parsed = JSON.parse(await readStream(result.stream));
expect(result.exportedAt).toBeDefined(); expect(parsed.exportedAt).toBeDefined();
expect(result.exportedAt >= before).toBe(true); expect(parsed.exportedAt >= before).toBe(true);
expect(result.exportedAt <= after).toBe(true); expect(parsed.exportedAt <= after).toBe(true);
expect(typeof parsed.rowCap).toBe('number');
expect(typeof parsed.sizeCap).toBe('number');
});
it('applies row cap to each collection query', async () => {
process.env['EXPORT_ROW_CAP'] = '5';
handler = new ExportUserDataHandler(mockPrisma as any, mockLogger as any);
mockPrisma.user.findUnique.mockResolvedValue(sampleUser);
setupEmptyRelations();
await handler.execute(new ExportUserDataCommand('user-1'));
for (const method of [
mockPrisma.listing.findMany,
mockPrisma.payment.findMany,
mockPrisma.review.findMany,
mockPrisma.inquiry.findMany,
mockPrisma.savedSearch.findMany,
mockPrisma.transaction.findMany,
]) {
expect(method).toHaveBeenCalledWith(expect.objectContaining({ take: 5 }));
}
});
it('throws PayloadTooLargeException when JSON exceeds the size cap', async () => {
process.env['EXPORT_SIZE_CAP_MB'] = '0.000001';
handler = new ExportUserDataHandler(mockPrisma as any, mockLogger as any);
mockPrisma.user.findUnique.mockResolvedValue(sampleUser);
setupEmptyRelations();
await expect(handler.execute(new ExportUserDataCommand('user-1'))).rejects.toThrow(
PayloadTooLargeException,
);
expect(mockLogger.warn).toHaveBeenCalled();
}); });
}); });

View File

@@ -1,8 +1,14 @@
import { InternalServerErrorException } from '@nestjs/common'; import { HttpException, InternalServerErrorException, PayloadTooLargeException } from '@nestjs/common';
import { CommandHandler, type ICommandHandler } from '@nestjs/cqrs'; import { CommandHandler, type ICommandHandler } from '@nestjs/cqrs';
import { Readable } from 'node:stream';
import { LoggerService, PrismaService, DomainException, NotFoundException } from '@modules/shared'; import { LoggerService, PrismaService, DomainException, NotFoundException } from '@modules/shared';
import { ExportUserDataCommand } from './export-user-data.command'; import { ExportUserDataCommand } from './export-user-data.command';
/** Per-collection row cap. Override via EXPORT_ROW_CAP env var (default 10 000). */
const DEFAULT_ROW_CAP = 10_000;
/** Maximum total export size in megabytes. Override via EXPORT_SIZE_CAP_MB env var (default 100). */
const DEFAULT_SIZE_CAP_MB = 100;
export interface UserDataExport { export interface UserDataExport {
user: { user: {
id: string; id: string;
@@ -22,16 +28,34 @@ export interface UserDataExport {
savedSearches: unknown[]; savedSearches: unknown[];
transactions: unknown[]; transactions: unknown[];
exportedAt: string; exportedAt: string;
/** Effective row cap applied to each collection query. */
rowCap: number;
/** Effective size cap in bytes for the entire JSON payload. */
sizeCap: number;
}
export interface ExportUserDataResult {
/** Node.js Readable stream containing the UTF-8 encoded JSON payload. */
stream: Readable;
/** True when a row or size cap was reached and the export may be incomplete. */
truncated: boolean;
} }
@CommandHandler(ExportUserDataCommand) @CommandHandler(ExportUserDataCommand)
export class ExportUserDataHandler implements ICommandHandler<ExportUserDataCommand> { export class ExportUserDataHandler implements ICommandHandler<ExportUserDataCommand> {
private readonly rowCap: number;
private readonly sizeCapBytes: number;
constructor( constructor(
private readonly prisma: PrismaService, private readonly prisma: PrismaService,
private readonly logger: LoggerService, private readonly logger: LoggerService,
) {} ) {
this.rowCap = parseInt(process.env['EXPORT_ROW_CAP'] ?? String(DEFAULT_ROW_CAP), 10);
const sizeMb = parseFloat(process.env['EXPORT_SIZE_CAP_MB'] ?? String(DEFAULT_SIZE_CAP_MB));
this.sizeCapBytes = Math.floor(sizeMb * 1024 * 1024);
}
async execute(command: ExportUserDataCommand): Promise<UserDataExport> { async execute(command: ExportUserDataCommand): Promise<ExportUserDataResult> {
try { try {
const user = await this.prisma.user.findUnique({ const user = await this.prisma.user.findUnique({
where: { id: command.userId }, where: { id: command.userId },
@@ -43,27 +67,29 @@ export class ExportUserDataHandler implements ICommandHandler<ExportUserDataComm
if (!user) throw new NotFoundException('User', command.userId); if (!user) throw new NotFoundException('User', command.userId);
const rowCap = this.rowCap;
const [agent, listings, payments, subscription, reviews, inquiries, savedSearches, transactions] = const [agent, listings, payments, subscription, reviews, inquiries, savedSearches, transactions] =
await Promise.all([ await Promise.all([
this.prisma.agent.findUnique({ where: { userId: command.userId } }), this.prisma.agent.findUnique({ where: { userId: command.userId } }),
this.prisma.listing.findMany({ this.prisma.listing.findMany({
where: { sellerId: command.userId }, where: { sellerId: command.userId },
take: rowCap,
include: { property: { select: { title: true, address: true, district: true, city: true } } }, include: { property: { select: { title: true, address: true, district: true, city: true } } },
}), }),
this.prisma.payment.findMany({ this.prisma.payment.findMany({
where: { userId: command.userId }, where: { userId: command.userId },
take: rowCap,
select: { id: true, provider: true, type: true, amountVND: true, status: true, createdAt: true }, select: { id: true, provider: true, type: true, amountVND: true, status: true, createdAt: true },
}), }),
this.prisma.subscription.findFirst({ where: { userId: command.userId } }), this.prisma.subscription.findFirst({ where: { userId: command.userId } }),
this.prisma.review.findMany({ where: { userId: command.userId } }), this.prisma.review.findMany({ where: { userId: command.userId }, take: rowCap }),
this.prisma.inquiry.findMany({ where: { userId: command.userId } }), this.prisma.inquiry.findMany({ where: { userId: command.userId }, take: rowCap }),
this.prisma.savedSearch.findMany({ where: { userId: command.userId } }), this.prisma.savedSearch.findMany({ where: { userId: command.userId }, take: rowCap }),
this.prisma.transaction.findMany({ where: { buyerId: command.userId } }), this.prisma.transaction.findMany({ where: { buyerId: command.userId }, take: rowCap }),
]); ]);
this.logger.log(`User data exported for ${command.userId}`, 'ExportUserDataHandler'); const payload: UserDataExport = {
return {
user, user,
agent, agent,
listings, listings,
@@ -74,9 +100,34 @@ export class ExportUserDataHandler implements ICommandHandler<ExportUserDataComm
savedSearches, savedSearches,
transactions, transactions,
exportedAt: new Date().toISOString(), exportedAt: new Date().toISOString(),
rowCap,
sizeCap: this.sizeCapBytes,
}; };
const json = JSON.stringify(payload);
const byteLength = Buffer.byteLength(json, 'utf8');
if (byteLength > this.sizeCapBytes) {
this.logger.warn(
`Export for user ${command.userId} is ${byteLength} bytes, exceeds cap of ${this.sizeCapBytes} bytes`,
this.constructor.name,
);
throw new PayloadTooLargeException(
`Dữ liệu xuất (${Math.round(byteLength / 1024 / 1024)} MB) vượt giới hạn ` +
`${Math.round(this.sizeCapBytes / 1024 / 1024)} MB. ` +
`Vui lòng liên hệ hỗ trợ để xuất theo từng phần.`,
);
}
this.logger.log(
`User data exported for ${command.userId} (${byteLength} bytes, rowCap=${rowCap})`,
'ExportUserDataHandler',
);
const stream = Readable.from(Buffer.from(json, 'utf8'));
return { stream, truncated: false };
} catch (error) { } catch (error) {
if (error instanceof DomainException) throw error; if (error instanceof DomainException || error instanceof HttpException) throw error;
this.logger.error( this.logger.error(
`Failed to export user data: ${error instanceof Error ? error.message : error}`, `Failed to export user data: ${error instanceof Error ? error.message : error}`,
error instanceof Error ? error.stack : undefined, error instanceof Error ? error.stack : undefined,

View File

@@ -5,13 +5,16 @@ import {
Get, Get,
Param, Param,
Post, Post,
Res,
StreamableFile,
UseGuards, UseGuards,
} from '@nestjs/common'; } from '@nestjs/common';
import { CommandBus } from '@nestjs/cqrs'; import { CommandBus } from '@nestjs/cqrs';
import { ApiTags, ApiOperation, ApiResponse, ApiBearerAuth } from '@nestjs/swagger'; import { ApiTags, ApiOperation, ApiResponse, ApiBearerAuth, ApiProduces } from '@nestjs/swagger';
import { Response } from 'express';
import { CancelUserDeletionCommand } from '../../application/commands/cancel-user-deletion/cancel-user-deletion.command'; import { CancelUserDeletionCommand } from '../../application/commands/cancel-user-deletion/cancel-user-deletion.command';
import { ExportUserDataCommand } from '../../application/commands/export-user-data/export-user-data.command'; import { ExportUserDataCommand } from '../../application/commands/export-user-data/export-user-data.command';
import { type UserDataExport } from '../../application/commands/export-user-data/export-user-data.handler'; import { type ExportUserDataResult } from '../../application/commands/export-user-data/export-user-data.handler';
import { ForceDeleteUserCommand } from '../../application/commands/force-delete-user/force-delete-user.command'; import { ForceDeleteUserCommand } from '../../application/commands/force-delete-user/force-delete-user.command';
import { RequestUserDeletionCommand } from '../../application/commands/request-user-deletion/request-user-deletion.command'; import { RequestUserDeletionCommand } from '../../application/commands/request-user-deletion/request-user-deletion.command';
import { type JwtPayload } from '../../infrastructure/services/token.service'; import { type JwtPayload } from '../../infrastructure/services/token.service';
@@ -58,13 +61,33 @@ export class UserDataController {
@Get('me/export') @Get('me/export')
@UseGuards(JwtAuthGuard) @UseGuards(JwtAuthGuard)
@ApiBearerAuth('JWT') @ApiBearerAuth('JWT')
@ApiOperation({ summary: 'Export user data (GDPR Article 20)' }) @ApiProduces('application/json')
@ApiResponse({ status: 200, description: 'User data exported as JSON' }) @ApiOperation({
summary: 'Export user data (GDPR Article 20)',
description:
'Streams the full user data export as JSON. ' +
'Row cap (per collection) defaults to 10 000 rows; size cap defaults to 100 MB. ' +
'Both are configurable via EXPORT_ROW_CAP and EXPORT_SIZE_CAP_MB env vars.',
})
@ApiResponse({ status: 200, description: 'User data exported as streaming JSON' })
@ApiResponse({ status: 401, description: 'Unauthorized' }) @ApiResponse({ status: 401, description: 'Unauthorized' })
@ApiResponse({
status: 413,
description: 'Export exceeds size cap — contact support for chunked export',
})
async exportData( async exportData(
@CurrentUser() user: JwtPayload, @CurrentUser() user: JwtPayload,
): Promise<UserDataExport> { @Res({ passthrough: true }) res: Response,
return this.commandBus.execute(new ExportUserDataCommand(user.sub)); ): Promise<StreamableFile> {
const result: ExportUserDataResult = await this.commandBus.execute(
new ExportUserDataCommand(user.sub),
);
res.setHeader('Content-Type', 'application/json');
res.setHeader(
'Content-Disposition',
`attachment; filename="user-data-${user.sub}.json"`,
);
return new StreamableFile(result.stream);
} }
@Delete(':id/force') @Delete(':id/force')

View File

@@ -10,6 +10,9 @@ import {
HTTP_REQUESTS_TOTAL, HTTP_REQUESTS_TOTAL,
GOODGO_WS_CONNECTED_CLIENTS, GOODGO_WS_CONNECTED_CLIENTS,
GOODGO_WS_MESSAGES_TOTAL, GOODGO_WS_MESSAGES_TOTAL,
READ_MODEL_PROJECTOR_LAG_SECONDS,
READ_MODEL_REFRESH_DURATION_SECONDS,
READ_MODEL_RECONCILIATION_DRIFT_TOTAL,
WEB_VITALS_LCP, WEB_VITALS_LCP,
WEB_VITALS_FCP, WEB_VITALS_FCP,
WEB_VITALS_CLS, WEB_VITALS_CLS,
@@ -37,6 +40,12 @@ export class MetricsService {
private readonly wsConnectedClientsGauge: Gauge, private readonly wsConnectedClientsGauge: Gauge,
@InjectMetric(GOODGO_WS_MESSAGES_TOTAL) @InjectMetric(GOODGO_WS_MESSAGES_TOTAL)
private readonly wsMessagesCounter: Counter, private readonly wsMessagesCounter: Counter,
@InjectMetric(READ_MODEL_PROJECTOR_LAG_SECONDS)
private readonly projectorLagGauge: Gauge,
@InjectMetric(READ_MODEL_REFRESH_DURATION_SECONDS)
private readonly readModelRefreshHistogram: Histogram,
@InjectMetric(READ_MODEL_RECONCILIATION_DRIFT_TOTAL)
private readonly reconciliationDriftCounter: Counter,
@InjectMetric(WEB_VITALS_LCP) @InjectMetric(WEB_VITALS_LCP)
private readonly lcpHistogram: Histogram, private readonly lcpHistogram: Histogram,
@InjectMetric(WEB_VITALS_FCP) @InjectMetric(WEB_VITALS_FCP)
@@ -106,6 +115,21 @@ export class MetricsService {
this.wsMessagesCounter.inc({ namespace, event, direction }); this.wsMessagesCounter.inc({ namespace, event, direction });
} }
/** Set current projector lag (seconds behind source stream) for a handler. */
setProjectorLag(handler: string, lagSeconds: number): void {
this.projectorLagGauge.set({ handler }, lagSeconds);
}
/** Record the duration of a read-model view refresh. */
recordReadModelRefresh(view: string, durationSeconds: number): void {
this.readModelRefreshHistogram.observe({ view }, durationSeconds);
}
/** Increment the reconciliation drift counter for a read model. */
recordReconciliationDrift(model: string, count = 1): void {
this.reconciliationDriftCounter.inc({ model }, count);
}
/** Map metric name → the correct histogram. */ /** Map metric name → the correct histogram. */
private readonly vitalHistograms: Record<string, Histogram | undefined> = {}; private readonly vitalHistograms: Record<string, Histogram | undefined> = {};

View File

@@ -0,0 +1,98 @@
import {
EVENT_ENVELOPE_SCHEMA_VERSION,
assertValidEnvelope,
isKnownEventType,
isUuidV7,
uuidv7,
validateEnvelope,
type EventEnvelope,
} from '@goodgo/contracts-events';
import { describe, it, expect } from 'vitest';
describe('@goodgo/contracts-events', () => {
describe('uuidv7', () => {
it('produces a RFC 9562 v7 UUID', () => {
const id = uuidv7();
expect(isUuidV7(id)).toBe(true);
});
it('encodes the provided timestamp in the high bits', () => {
const now = 1_714_000_000_000; // stable, post-2024
const id = uuidv7(now);
// First 8 hex chars = high 32 bits of ms timestamp
const hex = id.replace(/-/g, '').slice(0, 12);
const ts = parseInt(hex, 16);
expect(ts).toBe(now);
});
it('generates monotonic-ish ids across rapid calls', () => {
const a = uuidv7();
const b = uuidv7();
// v7 starts with the timestamp, so same-ms pairs compare by random bits;
// at worst they're equal-prefix — both must still be valid.
expect(isUuidV7(a)).toBe(true);
expect(isUuidV7(b)).toBe(true);
});
});
describe('validateEnvelope', () => {
const base: EventEnvelope = {
schemaVersion: EVENT_ENVELOPE_SCHEMA_VERSION,
eventId: uuidv7(),
eventType: 'payment.completed',
occurredAt: '2026-04-23T14:00:00.000Z',
producer: 'api',
traceId: 'a'.repeat(32),
payload: {},
};
it('accepts a valid envelope', () => {
expect(validateEnvelope(base)).toEqual([]);
});
it('rejects a non-v7 eventId', () => {
const issues = validateEnvelope({ ...base, eventId: 'not-a-uuid' });
expect(issues.map((i) => i.path)).toContain('eventId');
});
it('rejects an invalid eventType', () => {
const issues = validateEnvelope({ ...base, eventType: 'PaymentCompleted' });
expect(issues.map((i) => i.path)).toContain('eventType');
});
it('rejects a trace id that is not 32 hex chars', () => {
const issues = validateEnvelope({ ...base, traceId: 'short' });
expect(issues.map((i) => i.path)).toContain('traceId');
});
it('rejects schemaVersion drift', () => {
const issues = validateEnvelope({ ...base, schemaVersion: 99 });
expect(issues.map((i) => i.path)).toContain('schemaVersion');
});
it('rejects missing payload', () => {
const { payload: _drop, ...rest } = base;
void _drop;
const issues = validateEnvelope(rest as unknown);
expect(issues.map((i) => i.path)).toContain('payload');
});
});
describe('assertValidEnvelope', () => {
it('throws with a flat message on invalid input', () => {
expect(() => assertValidEnvelope({ schemaVersion: 1 })).toThrow(/Invalid EventEnvelope/);
});
});
describe('isKnownEventType', () => {
it('recognises the first 3 schemas', () => {
expect(isKnownEventType('payment.completed')).toBe(true);
expect(isKnownEventType('listing.approved')).toBe(true);
expect(isKnownEventType('kyc.verified')).toBe(true);
});
it('rejects unknown event types', () => {
expect(isKnownEventType('payment.refunded')).toBe(false);
});
});
});

View File

@@ -0,0 +1,82 @@
import { uuidv7, type EventEnvelope } from '@goodgo/contracts-events';
import { describe, it, expect, beforeEach } from 'vitest';
import { buildEnvelope } from '../envelope-builder';
import { InMemoryEventBus } from '../in-memory.event-bus';
describe('InMemoryEventBus', () => {
let bus: InMemoryEventBus;
beforeEach(() => {
bus = new InMemoryEventBus();
});
function env(type: string, payload: unknown = {}): EventEnvelope {
return buildEnvelope({ producer: 'api' }, { eventType: type, payload });
}
it('records published envelopes and returns a transport id', async () => {
const result = await bus.publish(env('payment.completed', { paymentId: 'p1' }));
expect(result.eventId).toMatch(/^[0-9a-f-]{36}$/);
expect(result.stream).toBe('events:payment.completed');
expect(result.transportId).toMatch(/-\d+$/);
expect(bus.all()).toHaveLength(1);
});
it('filters by event type', async () => {
await bus.publishAll([
env('payment.completed'),
env('listing.approved'),
env('payment.completed'),
]);
expect(bus.byType('payment.completed')).toHaveLength(2);
expect(bus.byType('listing.approved')).toHaveLength(1);
expect(bus.byType('kyc.verified')).toHaveLength(0);
});
it('rejects malformed envelopes', async () => {
const bad = {
schemaVersion: 1,
eventId: 'not-a-v7',
eventType: 'payment.completed',
occurredAt: '2026-04-23T00:00:00Z',
producer: 'api',
traceId: 'a'.repeat(32),
payload: {},
};
await expect(bus.publish(bad as unknown as EventEnvelope)).rejects.toThrow(
/Invalid EventEnvelope/,
);
});
it('reset clears state', async () => {
await bus.publish(env('kyc.verified'));
expect(bus.all()).toHaveLength(1);
bus.reset();
expect(bus.all()).toHaveLength(0);
});
it('buildEnvelope defaults traceId to 32 zeros when tracing is off', () => {
const e = buildEnvelope({ producer: 'api' }, { eventType: 'kyc.verified', payload: {} });
expect(e.traceId).toBe('0'.repeat(32));
expect(e.producer).toBe('api');
expect(e.schemaVersion).toBe(1);
});
it('buildEnvelope honours explicit overrides (replay path)', () => {
const id = uuidv7();
const e = buildEnvelope(
{ producer: 'api' },
{
eventType: 'listing.approved',
payload: {},
eventId: id,
traceId: 'b'.repeat(32),
occurredAt: new Date('2026-01-01T00:00:00Z'),
producer: 'replay-cli',
},
);
expect(e.eventId).toBe(id);
expect(e.traceId).toBe('b'.repeat(32));
expect(e.producer).toBe('replay-cli');
expect(e.occurredAt).toBe('2026-01-01T00:00:00.000Z');
});
});

View File

@@ -0,0 +1,55 @@
import { uuidv7, type EventEnvelope } from '@goodgo/contracts-events';
const ZERO_TRACE_ID = '0'.repeat(32);
/**
* Returns the active trace id, or 32 zeros when none is propagated.
*
* The codebase does not yet depend on `@opentelemetry/api` (Sentry handles
* traces today). To honor the CTO condition that every envelope carries a
* `traceId` from Phase 0, we expose this hook as the integration point.
*/
export function currentTraceId(): string {
try {
const sentryGlobal = (globalThis as Record<string, unknown>)['__SENTRY__'];
if (sentryGlobal && typeof sentryGlobal === 'object') {
const hub = (sentryGlobal as { hub?: { getScope?: () => { getSpan?: () => { traceId?: string } | undefined } } }).hub;
const traceId = hub?.getScope?.()?.getSpan?.()?.traceId;
if (typeof traceId === 'string' && /^[0-9a-f]{32}$/i.test(traceId)) {
return traceId;
}
}
} catch {
// Defensive: never let trace lookup fail event publishing.
}
return ZERO_TRACE_ID;
}
export interface BuildEnvelopeInput<TPayload> {
eventType: string;
payload: TPayload;
producer?: string;
occurredAt?: Date;
traceId?: string;
eventId?: string;
}
/**
* Pure helper — builds an `EventEnvelope` with sensible defaults
* (UUIDv7 eventId, current trace id, ISO occurredAt). Kept outside
* the EventBus to make unit testing trivial.
*/
export function buildEnvelope<TPayload>(
defaults: { producer: string },
input: BuildEnvelopeInput<TPayload>,
): EventEnvelope<TPayload> {
return {
schemaVersion: 1,
eventId: input.eventId ?? uuidv7(),
eventType: input.eventType,
occurredAt: (input.occurredAt ?? new Date()).toISOString(),
producer: input.producer ?? defaults.producer,
traceId: input.traceId ?? currentTraceId(),
payload: input.payload,
};
}

View File

@@ -0,0 +1,14 @@
import type { EventEnvelope } from '@goodgo/contracts-events';
export interface EventBus {
publish<T>(envelope: EventEnvelope<T>): Promise<PublishResult>;
publishAll(envelopes: EventEnvelope[]): Promise<PublishResult[]>;
}
export interface PublishResult {
eventId: string;
transportId: string;
stream: string;
}
export const EVENT_BUS = Symbol('EventBus');

View File

@@ -0,0 +1,51 @@
import { type EventEnvelope, assertValidEnvelope } from '@goodgo/contracts-events';
import { Injectable } from '@nestjs/common';
import type { EventBus, PublishResult } from './event-bus.interface';
import { streamFor } from './redis-streams.event-bus';
/**
* Test/dev double for the EventBus. Records every published envelope
* and exposes lookup helpers. Used by:
* - unit tests in this module
* - Phase 1 dual-publish diff harness
*/
@Injectable()
export class InMemoryEventBus implements EventBus {
private readonly published: { stream: string; envelope: EventEnvelope }[] = [];
private sequence = 0;
async publish<T>(envelope: EventEnvelope<T>): Promise<PublishResult> {
assertValidEnvelope(envelope);
const stream = streamFor(envelope.eventType);
this.published.push({ stream, envelope: envelope as EventEnvelope });
this.sequence += 1;
return {
eventId: envelope.eventId,
transportId: `${Date.now()}-${this.sequence}`,
stream,
};
}
async publishAll(envelopes: EventEnvelope[]): Promise<PublishResult[]> {
const out: PublishResult[] = [];
for (const env of envelopes) {
out.push(await this.publish(env));
}
return out;
}
all(): readonly EventEnvelope[] {
return this.published.map((p) => p.envelope);
}
byType<T = unknown>(eventType: string): EventEnvelope<T>[] {
return this.published
.filter((p) => p.envelope.eventType === eventType)
.map((p) => p.envelope as EventEnvelope<T>);
}
reset(): void {
this.published.length = 0;
this.sequence = 0;
}
}

View File

@@ -0,0 +1,12 @@
export {
EVENT_BUS,
type EventBus,
type PublishResult,
} from './event-bus.interface';
export { RedisStreamsEventBus, streamFor } from './redis-streams.event-bus';
export { InMemoryEventBus } from './in-memory.event-bus';
export {
buildEnvelope,
currentTraceId,
type BuildEnvelopeInput,
} from './envelope-builder';

View File

@@ -0,0 +1,65 @@
import { type EventEnvelope, assertValidEnvelope } from '@goodgo/contracts-events';
import { Injectable, Logger } from '@nestjs/common';
// eslint-disable-next-line @typescript-eslint/consistent-type-imports -- NestJS DI requires value imports
import { RedisService } from '../redis.service';
import type { EventBus, PublishResult } from './event-bus.interface';
/**
* Stream naming: one stream per event-type. Per-event consumer groups
* are created lazily by individual consumer services (Phase 1+).
*
* events:payment.completed
* events:listing.approved
* events:kyc.verified
*
* `MAXLEN ~ 100000` per stream — RFC §5 mitigation. Nightly archive
* to S3 lands in Phase 3.
*/
const STREAM_PREFIX = 'events:';
const DEFAULT_MAXLEN = 100_000;
@Injectable()
export class RedisStreamsEventBus implements EventBus {
private readonly logger = new Logger(RedisStreamsEventBus.name);
private readonly maxlen: number;
constructor(private readonly redis: RedisService) {
const envMax = process.env['EVENT_BUS_STREAM_MAXLEN'];
this.maxlen = envMax ? Number(envMax) : DEFAULT_MAXLEN;
}
async publish<T>(envelope: EventEnvelope<T>): Promise<PublishResult> {
assertValidEnvelope(envelope);
const stream = streamFor(envelope.eventType);
const client = this.redis.getClient();
// XADD <stream> MAXLEN ~ <n> * envelope <json>
const transportId = await client.xadd(
stream,
'MAXLEN',
'~',
this.maxlen,
'*',
'envelope',
JSON.stringify(envelope),
);
if (transportId === null) {
throw new Error(`XADD returned NIL for stream ${stream}`);
}
this.logger.debug(
`Published ${envelope.eventType} eventId=${envelope.eventId} -> ${stream}@${transportId}`,
);
return { eventId: envelope.eventId, transportId, stream };
}
async publishAll(envelopes: EventEnvelope[]): Promise<PublishResult[]> {
const out: PublishResult[] = [];
for (const env of envelopes) {
out.push(await this.publish(env));
}
return out;
}
}
export function streamFor(eventType: string): string {
return `${STREAM_PREFIX}${eventType}`;
}

View File

@@ -42,3 +42,18 @@ export { FileValidationPipe } from './pipes/file-validation.pipe';
export type { FileValidationOptions, UploadedFile } from './pipes/file-validation.pipe'; export type { FileValidationOptions, UploadedFile } from './pipes/file-validation.pipe';
export { validateEnv, validateJwtSecret } from './env-validation'; export { validateEnv, validateJwtSecret } from './env-validation';
export { cacheMetaStorage, type CacheMeta, type CacheMetaStore } from './cache-meta.store'; export { cacheMetaStorage, type CacheMeta, type CacheMetaStore } from './cache-meta.store';
// RFC-001 Phase 1 — API versioning.
export {
API_VERSION_REGISTRY,
resolveMajorSpec,
type ApiMajorSpec,
type ApiVersionDeprecation,
type ApiVersionRegistry,
} from './versioning';
export {
VersionInterceptor,
DeprecationInterceptor,
API_MINOR_HEADER,
API_MINOR_RESOLVED_HEADER,
type ResolvedApiVersion,
} from './interceptors';

View File

@@ -0,0 +1,133 @@
import { describe, it, expect, beforeEach, vi } from 'vitest';
import { buildEnvelope } from '../../event-bus';
import { InMemoryEventBus } from '../../event-bus/in-memory.event-bus';
import { OutboxRelay } from '../outbox.relay';
type OutboxRow = {
id: string;
eventId: string;
eventType: string;
aggregateId: string | null;
envelope: unknown;
createdAt: Date;
publishedAt: Date | null;
attempts: number;
lastError: string | null;
};
/**
* Lightweight fake Prisma — just enough for OutboxRelay.tick():
* - pg_try_advisory_lock
* - eventOutbox.findMany / update
*/
function makeFakePrisma(rows: OutboxRow[], options: { acquireLock?: boolean } = {}) {
const acquireLock = options.acquireLock ?? true;
return {
$queryRawUnsafe: vi.fn(async (sql: string) => {
if (sql.includes('pg_try_advisory_lock')) return [{ locked: acquireLock }];
return [];
}),
eventOutbox: {
findMany: vi.fn(async (args: { where?: { publishedAt: null }; take?: number }) => {
const pending = rows.filter((r) => r.publishedAt === null);
return pending.slice(0, args.take ?? 100);
}),
update: vi.fn(async (args: { where: { id: string }; data: Record<string, unknown> }) => {
const row = rows.find((r) => r.id === args.where.id);
if (!row) throw new Error('row not found');
const data = args.data;
if ('publishedAt' in data) row.publishedAt = data['publishedAt'] as Date | null;
if ('lastError' in data) row.lastError = data['lastError'] as string | null;
if ('attempts' in data) {
const v = data['attempts'];
if (v && typeof v === 'object' && 'increment' in v) {
row.attempts += (v as { increment: number }).increment;
}
}
return row;
}),
},
};
}
function fakeRow(type: string): OutboxRow {
const envelope = buildEnvelope({ producer: 'api' }, { eventType: type, payload: { k: 'v' } });
return {
id: `row-${envelope.eventId}`,
eventId: envelope.eventId,
eventType: envelope.eventType,
aggregateId: null,
envelope,
createdAt: new Date(),
publishedAt: null,
attempts: 0,
lastError: null,
};
}
describe('OutboxRelay.tick', () => {
let bus: InMemoryEventBus;
beforeEach(() => {
bus = new InMemoryEventBus();
process.env['EVENT_OUTBOX_RELAY_ENABLED'] = 'false'; // don't auto-start timer
});
it('drains pending rows into the EventBus and marks them published', async () => {
const rows = [fakeRow('payment.completed'), fakeRow('listing.approved')];
const prisma = makeFakePrisma(rows);
const relay = new OutboxRelay(prisma as never, bus);
const result = await relay.tick();
expect(result.acquired).toBe(true);
expect(result.processed).toBe(2);
expect(result.failed).toBe(0);
expect(bus.all()).toHaveLength(2);
expect(rows.every((r) => r.publishedAt instanceof Date)).toBe(true);
});
it('does nothing when the advisory lock is held elsewhere', async () => {
const rows = [fakeRow('kyc.verified')];
const prisma = makeFakePrisma(rows, { acquireLock: false });
const relay = new OutboxRelay(prisma as never, bus);
const result = await relay.tick();
expect(result.acquired).toBe(false);
expect(result.processed).toBe(0);
expect(bus.all()).toHaveLength(0);
expect(rows[0]?.publishedAt).toBeNull();
});
it('records lastError and leaves publishedAt null on publish failure', async () => {
const rows = [fakeRow('payment.completed')];
const prisma = makeFakePrisma(rows);
const failing = {
publish: vi.fn(async () => {
throw new Error('XADD refused');
}),
publishAll: vi.fn(),
};
const relay = new OutboxRelay(prisma as never, failing as never);
const result = await relay.tick();
expect(result.processed).toBe(0);
expect(result.failed).toBe(1);
expect(rows[0]?.publishedAt).toBeNull();
expect(rows[0]?.lastError).toContain('XADD refused');
expect(rows[0]?.attempts).toBe(1);
});
it('skips rows that are already published', async () => {
const row = fakeRow('listing.approved');
row.publishedAt = new Date();
const prisma = makeFakePrisma([row]);
const relay = new OutboxRelay(prisma as never, bus);
const result = await relay.tick();
expect(result.processed).toBe(0);
expect(bus.all()).toHaveLength(0);
});
});

View File

@@ -0,0 +1,2 @@
export { OutboxService, type OutboxAppendOptions } from './outbox.service';
export { OutboxRelay } from './outbox.relay';

View File

@@ -0,0 +1,134 @@
import type { EventEnvelope } from '@goodgo/contracts-events';
import {
Inject,
Injectable,
Logger,
type OnModuleDestroy,
type OnModuleInit,
} from '@nestjs/common';
import { EVENT_BUS, type EventBus } from '../event-bus/event-bus.interface';
import { type PrismaService } from '../prisma.service';
/**
* Single-process relay that drains `event_outbox` into the EventBus.
*
* Concurrency: every node tries to acquire the same Postgres advisory
* lock (`pg_try_advisory_lock`); only the holder runs the poll loop.
* This is the single-process + advisory-lock design called out in
* RFC-004 §4 ("No leader-election library yet").
*/
const ADVISORY_LOCK_KEY = 0xe7b04204; // bespoke 32-bit key for the outbox relay
const DEFAULT_POLL_MS = 1_000;
const DEFAULT_BATCH_SIZE = 100;
@Injectable()
export class OutboxRelay implements OnModuleInit, OnModuleDestroy {
private readonly logger = new Logger(OutboxRelay.name);
private readonly pollIntervalMs: number;
private readonly batchSize: number;
private readonly enabled: boolean;
private timer: NodeJS.Timeout | null = null;
private running = false;
private stopped = false;
private holdsLock = false;
constructor(
private readonly prisma: PrismaService,
@Inject(EVENT_BUS) private readonly bus: EventBus,
) {
this.pollIntervalMs = Number(process.env['EVENT_OUTBOX_POLL_MS'] ?? DEFAULT_POLL_MS);
this.batchSize = Number(process.env['EVENT_OUTBOX_BATCH_SIZE'] ?? DEFAULT_BATCH_SIZE);
this.enabled = (process.env['EVENT_OUTBOX_RELAY_ENABLED'] ?? 'true').toLowerCase() !== 'false';
}
onModuleInit(): void {
if (!this.enabled) {
this.logger.log('OutboxRelay disabled via EVENT_OUTBOX_RELAY_ENABLED=false');
return;
}
this.scheduleNext();
}
async onModuleDestroy(): Promise<void> {
this.stopped = true;
if (this.timer) clearTimeout(this.timer);
if (this.holdsLock) {
try {
await this.prisma.$queryRawUnsafe(`SELECT pg_advisory_unlock(${ADVISORY_LOCK_KEY})`);
} catch (err) {
this.logger.warn(`Failed to release advisory lock: ${(err as Error).message}`);
}
}
}
private scheduleNext(): void {
if (this.stopped) return;
this.timer = setTimeout(() => {
void this.tick().finally(() => this.scheduleNext());
}, this.pollIntervalMs);
}
/** Public for tests — drains one batch synchronously. */
async tick(): Promise<{ acquired: boolean; processed: number; failed: number }> {
if (this.running) return { acquired: false, processed: 0, failed: 0 };
this.running = true;
try {
const acquired = await this.tryAcquireLock();
if (!acquired) return { acquired: false, processed: 0, failed: 0 };
const { processed, failed } = await this.drainBatch();
return { acquired: true, processed, failed };
} finally {
this.running = false;
}
}
private async tryAcquireLock(): Promise<boolean> {
if (this.holdsLock) return true;
const rows = await this.prisma.$queryRawUnsafe<{ locked: boolean }[]>(
`SELECT pg_try_advisory_lock(${ADVISORY_LOCK_KEY}) AS locked`,
);
const locked = rows[0]?.locked === true;
if (locked) {
this.holdsLock = true;
this.logger.log('Acquired event_outbox advisory lock — this node is now the relay leader');
}
return locked;
}
private async drainBatch(): Promise<{ processed: number; failed: number }> {
const pending = await this.prisma.eventOutbox.findMany({
where: { publishedAt: null },
orderBy: { createdAt: 'asc' },
take: this.batchSize,
});
if (pending.length === 0) return { processed: 0, failed: 0 };
let processed = 0;
let failed = 0;
for (const row of pending) {
try {
const envelope = row.envelope as unknown as EventEnvelope;
await this.bus.publish(envelope);
await this.prisma.eventOutbox.update({
where: { id: row.id },
data: { publishedAt: new Date(), attempts: { increment: 1 }, lastError: null },
});
processed += 1;
} catch (err) {
failed += 1;
const message = err instanceof Error ? err.message : String(err);
this.logger.error(
`Outbox publish failed for eventId=${row.eventId} type=${row.eventType}: ${message}`,
);
await this.prisma.eventOutbox.update({
where: { id: row.id },
data: { attempts: { increment: 1 }, lastError: message.slice(0, 1000) },
});
}
}
if (processed > 0 || failed > 0) {
this.logger.debug(`Outbox drained batch: processed=${processed} failed=${failed}`);
}
return { processed, failed };
}
}

View File

@@ -0,0 +1,49 @@
import { type EventEnvelope, assertValidEnvelope } from '@goodgo/contracts-events';
import { Injectable, Logger } from '@nestjs/common';
import type { Prisma } from '@prisma/client';
// eslint-disable-next-line @typescript-eslint/consistent-type-imports -- NestJS DI requires value imports
import { PrismaService } from '../prisma.service';
/**
* Transactional outbox writer. Call inside the same Prisma transaction
* as the domain change so the row commits atomically with the state
* mutation it describes. The Outbox **never** publishes directly; the
* relay (`OutboxRelay`) tails `event_outbox` and forwards to the EventBus.
*/
export interface OutboxAppendOptions {
aggregateId?: string;
}
type EventOutboxDelegate = PrismaService['eventOutbox'];
type PrismaTxLike = Pick<EventOutboxDelegate, 'create'> | { eventOutbox: Pick<EventOutboxDelegate, 'create'> };
@Injectable()
export class OutboxService {
private readonly logger = new Logger(OutboxService.name);
constructor(private readonly prisma: PrismaService) {}
async append(
tx: PrismaTxLike | PrismaService,
envelope: EventEnvelope,
options: OutboxAppendOptions = {},
): Promise<void> {
assertValidEnvelope(envelope);
const client = ('eventOutbox' in tx ? tx.eventOutbox : tx) as EventOutboxDelegate;
await client.create({
data: {
eventId: envelope.eventId,
eventType: envelope.eventType,
aggregateId: options.aggregateId ?? null,
envelope: envelope as unknown as Prisma.InputJsonValue,
},
});
}
async appendStandalone(envelope: EventEnvelope, options: OutboxAppendOptions = {}): Promise<void> {
await this.append(this.prisma, envelope, options);
this.logger.warn(
`appendStandalone used for ${envelope.eventType} eventId=${envelope.eventId} — prefer the transactional append()`,
);
}
}

View File

@@ -1,6 +1,6 @@
import { Global, type MiddlewareConsumer, Module, type NestModule, RequestMethod } from '@nestjs/common'; import { Global, type MiddlewareConsumer, Module, type NestModule, RequestMethod } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config'; import { ConfigModule } from '@nestjs/config';
import { APP_FILTER } from '@nestjs/core'; import { APP_FILTER, APP_INTERCEPTOR } from '@nestjs/core';
import { EventEmitterModule } from '@nestjs/event-emitter'; import { EventEmitterModule } from '@nestjs/event-emitter';
import { PrometheusModule, makeCounterProvider } from '@willsoto/nestjs-prometheus'; import { PrometheusModule, makeCounterProvider } from '@willsoto/nestjs-prometheus';
import { import {
@@ -10,8 +10,11 @@ import {
CACHE_DEGRADATION_TOTAL, CACHE_DEGRADATION_TOTAL,
} from './infrastructure/cache.service'; } from './infrastructure/cache.service';
import { EventBusService } from './infrastructure/event-bus.service'; import { EventBusService } from './infrastructure/event-bus.service';
import { EVENT_BUS, RedisStreamsEventBus } from './infrastructure/event-bus';
import { OutboxRelay, OutboxService } from './infrastructure/outbox';
import { FieldEncryptionService } from './infrastructure/field-encryption.service'; import { FieldEncryptionService } from './infrastructure/field-encryption.service';
import { GlobalExceptionFilter } from './infrastructure/filters/global-exception.filter'; import { GlobalExceptionFilter } from './infrastructure/filters/global-exception.filter';
import { DeprecationInterceptor, VersionInterceptor } from './infrastructure/interceptors';
import { LoggerService } from './infrastructure/logger.service'; import { LoggerService } from './infrastructure/logger.service';
import { CorrelationIdMiddleware } from './infrastructure/middleware/correlation-id.middleware'; import { CorrelationIdMiddleware } from './infrastructure/middleware/correlation-id.middleware';
import { CsrfMiddleware } from './infrastructure/middleware/csrf.middleware'; import { CsrfMiddleware } from './infrastructure/middleware/csrf.middleware';
@@ -35,6 +38,10 @@ import { TypesenseClientService } from './infrastructure/typesense-client.servic
RedisService, RedisService,
CacheService, CacheService,
EventBusService, EventBusService,
// RFC-004 Phase 0 (GOO-172) — durable async messaging backbone.
{ provide: EVENT_BUS, useClass: RedisStreamsEventBus },
OutboxService,
OutboxRelay,
TypesenseClientService, TypesenseClientService,
makeCounterProvider({ makeCounterProvider({
name: CACHE_HIT_TOTAL, name: CACHE_HIT_TOTAL,
@@ -55,8 +62,18 @@ import { TypesenseClientService } from './infrastructure/typesense-client.servic
provide: APP_FILTER, provide: APP_FILTER,
useClass: GlobalExceptionFilter, useClass: GlobalExceptionFilter,
}, },
// RFC-001 Phase 1 (GOO-170) — order matters: VersionInterceptor first
// populates req.apiVersion; DeprecationInterceptor reads from it.
{
provide: APP_INTERCEPTOR,
useClass: VersionInterceptor,
},
{
provide: APP_INTERCEPTOR,
useClass: DeprecationInterceptor,
},
], ],
exports: [PrismaService, RedisService, CacheService, LoggerService, EventBusService, FieldEncryptionService, TypesenseClientService, PrometheusModule], exports: [PrismaService, RedisService, CacheService, LoggerService, EventBusService, EVENT_BUS, OutboxService, FieldEncryptionService, TypesenseClientService, PrometheusModule],
}) })
export class SharedModule implements NestModule { export class SharedModule implements NestModule {
configure(consumer: MiddlewareConsumer): void { configure(consumer: MiddlewareConsumer): void {

View File

@@ -0,0 +1,53 @@
# @goodgo/contracts-events
Cross-runtime (Node + Python) event contracts for RFC-004's async messaging
backbone. See [GOO-95](/GOO/issues/GOO-95) for the RFC and
[GOO-172](/GOO/issues/GOO-172) for Phase 0.
## What lives here
- `src/envelope.ts``EventEnvelope<T>` TypeScript type + `EVENT_ENVELOPE_SCHEMA_VERSION`.
- `src/uuid-v7.ts` — pure-Node UUIDv7 generator (no runtime deps).
- `src/event-types.ts` — string-literal union of all known event types.
- `schemas/envelope.schema.json` — JSON Schema for the envelope itself.
- `schemas/<event-type>.schema.json` — JSON Schema for each event payload.
The `schemas/` directory is consumed by the Python AI services
(`libs/ai-services`) via `redis-py` consumers — JSON Schema is the single
source of truth across runtimes.
## Envelope shape
```ts
interface EventEnvelope<TPayload = unknown> {
schemaVersion: number; // bump when envelope itself changes
eventId: string; // UUIDv7 — time-ordered, idempotency key
eventType: string; // dotted: "payment.completed"
occurredAt: string; // ISO-8601 UTC
producer: string; // service name, e.g. "api"
traceId: string; // OpenTelemetry-compatible (32 hex chars or "00…")
payload: TPayload; // event-specific, validated by per-type schema
}
```
`schemaVersion` starts at `1`. Bump only when the **envelope** changes;
payload changes are versioned per event-type schema independently.
## First 3 schemas (Phase 0 deliverable)
| Event type | Trigger |
|----------------------|--------------------------------------------------|
| `payment.completed` | Payment moves to `succeeded` after gateway IPN |
| `listing.approved` | Moderation approves a listing |
| `kyc.verified` | KYC review marks a user verified |
Phase 1 (notifications cutover, [GOO-173](/GOO/issues/GOO-173)) will add
the rest of the production event surface.
## Adding a new event type
1. Pick a stable dotted name (`<aggregate>.<past-tense-verb>`).
2. Add a `schemas/<name>.schema.json` JSON Schema describing the payload.
3. Add the literal to `src/event-types.ts`.
4. (Optional) Re-export a typed payload alias from `src/index.ts`.
5. Land + dual-publish for at least one sprint before any consumer hard-fails on it.

View File

@@ -0,0 +1,19 @@
{
"name": "@goodgo/contracts-events",
"version": "0.1.0",
"private": true,
"main": "./src/index.ts",
"types": "./src/index.ts",
"exports": {
".": "./src/index.ts",
"./schemas/*": "./schemas/*"
},
"scripts": {
"lint": "echo \"(no lint configured)\"",
"typecheck": "tsc --noEmit",
"test": "echo \"(no tests yet)\""
},
"devDependencies": {
"typescript": "^5.5.0"
}
}

View File

@@ -0,0 +1,52 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://goodgo.vn/schemas/events/envelope.schema.json",
"title": "EventEnvelope",
"description": "Cross-runtime event envelope for RFC-004 (GOO-95). Source of truth — Node and Python consumers validate against this file.",
"type": "object",
"additionalProperties": false,
"required": [
"schemaVersion",
"eventId",
"eventType",
"occurredAt",
"producer",
"traceId",
"payload"
],
"properties": {
"schemaVersion": {
"type": "integer",
"const": 1,
"description": "Envelope wire-format version. Currently 1."
},
"eventId": {
"type": "string",
"pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$",
"description": "UUIDv7 — time-ordered. Used as idempotency key (30-day TTL in idempotency table)."
},
"eventType": {
"type": "string",
"pattern": "^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$",
"description": "Dotted event type, e.g. payment.completed."
},
"occurredAt": {
"type": "string",
"format": "date-time",
"description": "ISO-8601 UTC timestamp of the domain event (not publish time)."
},
"producer": {
"type": "string",
"minLength": 1,
"description": "Producing service identifier, e.g. api, ai-services."
},
"traceId": {
"type": "string",
"pattern": "^[0-9a-f]{32}$",
"description": "OpenTelemetry-compatible 32-hex trace id. Use 32 zeros when no active span."
},
"payload": {
"description": "Event-specific payload; validated separately against schemas/<eventType>.schema.json."
}
}
}

View File

@@ -0,0 +1,20 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://goodgo.vn/schemas/events/kyc.verified.schema.json",
"title": "kyc.verified",
"description": "Emitted when a user's KYC review transitions to VERIFIED.",
"type": "object",
"additionalProperties": false,
"required": ["userId", "verifiedByUserId", "level", "verifiedAt", "documentRefs"],
"properties": {
"userId": { "type": "string", "minLength": 1 },
"verifiedByUserId": { "type": "string", "minLength": 1 },
"level": { "type": "string", "enum": ["basic", "enhanced"] },
"verifiedAt": { "type": "string", "format": "date-time" },
"documentRefs": {
"type": "array",
"items": { "type": "string", "minLength": 1 },
"description": "Opaque references (e.g. S3 keys) to the documents used for verification."
}
}
}

View File

@@ -0,0 +1,28 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://goodgo.vn/schemas/events/listing.approved.schema.json",
"title": "listing.approved",
"description": "Emitted when a Listing is approved by moderation and goes live.",
"type": "object",
"additionalProperties": false,
"required": [
"listingId",
"propertyId",
"agentId",
"approvedByUserId",
"approvedAt",
"expiresAt"
],
"properties": {
"listingId": { "type": "string", "minLength": 1 },
"propertyId": { "type": "string", "minLength": 1 },
"agentId": { "type": "string", "minLength": 1 },
"approvedByUserId": { "type": "string", "minLength": 1 },
"approvedAt": { "type": "string", "format": "date-time" },
"expiresAt": {
"type": ["string", "null"],
"format": "date-time",
"description": "Null when the listing has no scheduled expiry."
}
}
}

View File

@@ -0,0 +1,32 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://goodgo.vn/schemas/events/payment.completed.schema.json",
"title": "payment.completed",
"description": "Emitted when a Payment aggregate transitions to SUCCEEDED after a verified gateway IPN (VNPay / MoMo / ZaloPay).",
"type": "object",
"additionalProperties": false,
"required": [
"paymentId",
"orderId",
"userId",
"amount",
"currency",
"gateway",
"gatewayTransactionId",
"paidAt"
],
"properties": {
"paymentId": { "type": "string", "minLength": 1 },
"orderId": { "type": "string", "minLength": 1 },
"userId": { "type": "string", "minLength": 1 },
"amount": {
"type": "string",
"pattern": "^-?\\d+(\\.\\d+)?$",
"description": "Decimal as string to preserve VND precision (no float rounding)."
},
"currency": { "type": "string", "enum": ["VND", "USD"] },
"gateway": { "type": "string", "enum": ["vnpay", "momo", "zalopay"] },
"gatewayTransactionId": { "type": "string", "minLength": 1 },
"paidAt": { "type": "string", "format": "date-time" }
}
}

View File

@@ -0,0 +1,69 @@
import { isUuidV7 } from './uuid-v7';
export const EVENT_ENVELOPE_SCHEMA_VERSION = 1;
export interface EventEnvelope<TPayload = unknown> {
schemaVersion: number;
eventId: string;
eventType: string;
occurredAt: string;
producer: string;
traceId: string;
payload: TPayload;
}
const TRACE_ID_RE = /^[0-9a-f]{32}$/i;
const ISO_8601_RE =
/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,9})?(?:Z|[+-]\d{2}:\d{2})$/;
export interface EnvelopeValidationIssue {
path: string;
message: string;
}
export function validateEnvelope(envelope: unknown): EnvelopeValidationIssue[] {
const issues: EnvelopeValidationIssue[] = [];
if (envelope === null || typeof envelope !== 'object') {
return [{ path: '$', message: 'envelope must be an object' }];
}
const e = envelope as Record<string, unknown>;
if (e['schemaVersion'] !== EVENT_ENVELOPE_SCHEMA_VERSION) {
issues.push({
path: 'schemaVersion',
message: `expected ${EVENT_ENVELOPE_SCHEMA_VERSION}, got ${String(e['schemaVersion'])}`,
});
}
if (typeof e['eventId'] !== 'string' || !isUuidV7(e['eventId'])) {
issues.push({ path: 'eventId', message: 'must be a UUIDv7 string' });
}
if (
typeof e['eventType'] !== 'string' ||
!/^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)+$/.test(e['eventType'])
) {
issues.push({
path: 'eventType',
message: 'must match /^[a-z][a-z0-9_]*(\\.[a-z][a-z0-9_]*)+$/',
});
}
if (typeof e['occurredAt'] !== 'string' || !ISO_8601_RE.test(e['occurredAt'])) {
issues.push({ path: 'occurredAt', message: 'must be an ISO-8601 timestamp' });
}
if (typeof e['producer'] !== 'string' || e['producer'].length === 0) {
issues.push({ path: 'producer', message: 'must be a non-empty string' });
}
if (typeof e['traceId'] !== 'string' || !TRACE_ID_RE.test(e['traceId'])) {
issues.push({ path: 'traceId', message: 'must be 32 hex characters' });
}
if (!('payload' in e)) {
issues.push({ path: 'payload', message: 'is required (use {} for empty)' });
}
return issues;
}
export function assertValidEnvelope(envelope: unknown): asserts envelope is EventEnvelope {
const issues = validateEnvelope(envelope);
if (issues.length > 0) {
const flat = issues.map((i) => `${i.path}: ${i.message}`).join('; ');
throw new Error(`Invalid EventEnvelope — ${flat}`);
}
}

View File

@@ -0,0 +1,11 @@
export const KNOWN_EVENT_TYPES = [
'kyc.verified',
'listing.approved',
'payment.completed',
] as const;
export type KnownEventType = (typeof KNOWN_EVENT_TYPES)[number];
export function isKnownEventType(value: string): value is KnownEventType {
return (KNOWN_EVENT_TYPES as readonly string[]).includes(value);
}

View File

@@ -0,0 +1,37 @@
export {
EVENT_ENVELOPE_SCHEMA_VERSION,
type EventEnvelope,
type EnvelopeValidationIssue,
validateEnvelope,
assertValidEnvelope,
} from './envelope';
export { uuidv7, isUuidV7 } from './uuid-v7';
export { KNOWN_EVENT_TYPES, type KnownEventType, isKnownEventType } from './event-types';
export interface PaymentCompletedPayload {
paymentId: string;
orderId: string;
userId: string;
amount: string;
currency: 'VND' | 'USD';
gateway: 'vnpay' | 'momo' | 'zalopay';
gatewayTransactionId: string;
paidAt: string;
}
export interface ListingApprovedPayload {
listingId: string;
propertyId: string;
agentId: string;
approvedByUserId: string;
approvedAt: string;
expiresAt: string | null;
}
export interface KycVerifiedPayload {
userId: string;
verifiedByUserId: string;
level: 'basic' | 'enhanced';
verifiedAt: string;
documentRefs: string[];
}

View File

@@ -0,0 +1,44 @@
import { randomBytes } from 'node:crypto';
/**
* UUIDv7 — 48-bit Unix-ms timestamp in the high bits, 74 random bits below.
*
* Time-ordered, monotonic enough for our needs (idempotency keys + Stream IDs).
* No dependency on the `uuid` package — Phase 0 keeps the foundation
* tree-shakeable for the Python side (which uses its own implementation).
*
* Reference: RFC 9562 §5.7.
*/
export function uuidv7(now: number = Date.now()): string {
const ts = BigInt(now); // milliseconds since epoch
const bytes = randomBytes(16);
// 48-bit timestamp (big-endian) in bytes 0..5
bytes[0] = Number((ts >> 40n) & 0xffn);
bytes[1] = Number((ts >> 32n) & 0xffn);
bytes[2] = Number((ts >> 24n) & 0xffn);
bytes[3] = Number((ts >> 16n) & 0xffn);
bytes[4] = Number((ts >> 8n) & 0xffn);
bytes[5] = Number(ts & 0xffn);
// Version 7 in the high nibble of byte 6
bytes[6] = (bytes[6]! & 0x0f) | 0x70;
// RFC 4122 variant (10xx) in the high bits of byte 8
bytes[8] = (bytes[8]! & 0x3f) | 0x80;
const hex = Buffer.from(bytes).toString('hex');
return [
hex.slice(0, 8),
hex.slice(8, 12),
hex.slice(12, 16),
hex.slice(16, 20),
hex.slice(20, 32),
].join('-');
}
const UUID_V7_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
export function isUuidV7(value: string): boolean {
return UUID_V7_RE.test(value);
}

View File

@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"strict": true,
"esModuleInterop": true,
"resolveJsonModule": true,
"skipLibCheck": true,
"noEmit": true
},
"include": ["src/**/*.ts", "schemas/**/*.json"]
}

View File

@@ -2,3 +2,4 @@ packages:
- 'apps/*' - 'apps/*'
- 'packages/*' - 'packages/*'
- 'libs/*' - 'libs/*'
- 'libs/contracts/*'

View File

@@ -0,0 +1,26 @@
-- RFC-004 Phase 0 (GOO-172): transactional outbox for async messaging backbone.
-- Producers insert into event_outbox in the same tx as the domain change.
-- A single relay process (Postgres advisory-lock leader) tails pending rows
-- and publishes them to Redis Streams, then flips published_at.
CREATE TABLE "event_outbox" (
"id" TEXT NOT NULL,
"eventId" TEXT NOT NULL,
"eventType" TEXT NOT NULL,
"aggregateId" TEXT,
"envelope" JSONB NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"publishedAt" TIMESTAMP(3),
"attempts" INTEGER NOT NULL DEFAULT 0,
"lastError" TEXT,
CONSTRAINT "event_outbox_pkey" PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX "event_outbox_eventId_key" ON "event_outbox"("eventId");
-- Hot path: relay scans `WHERE publishedAt IS NULL ORDER BY createdAt`.
CREATE INDEX "event_outbox_publishedAt_createdAt_idx" ON "event_outbox"("publishedAt", "createdAt");
-- Diagnostics: per-type backlog inspection.
CREATE INDEX "event_outbox_eventType_createdAt_idx" ON "event_outbox"("eventType", "createdAt");

View File

@@ -1567,3 +1567,32 @@ model VnAdministrativeAlias {
@@index([newWardCode]) @@index([newWardCode])
@@map("vn_administrative_aliases") @@map("vn_administrative_aliases")
} }
/// Transactional outbox for RFC-004 async messaging backbone (GOO-95).
/// Producers write one row per domain event in the same Postgres
/// transaction as the domain state change. A single relay process
/// (Postgres advisory-lock leader) tails pending rows and publishes
/// them to Redis Streams, flipping `publishedAt` on success.
model EventOutbox {
id String @id @default(cuid())
/// UUIDv7 from the envelope — idempotency key + stable cross-runtime id.
eventId String @unique
/// Dotted event type (`payment.completed`). Used by the relay to route.
eventType String
/// Aggregate identifier (e.g. paymentId, listingId) — for partitioning / debugging.
aggregateId String?
/// Fully-formed `EventEnvelope` JSON ready to XADD. Never mutated after insert.
envelope Json
/// When the row was inserted (inside the domain tx).
createdAt DateTime @default(now())
/// When the relay confirmed XADD acceptance. Null = still pending.
publishedAt DateTime?
/// Monotonic retry count for the relay (reset on success).
attempts Int @default(0)
/// Last error message on failure — surfaced in admin dashboards / Sentry.
lastError String?
@@index([publishedAt, createdAt])
@@index([eventType, createdAt])
@@map("event_outbox")
}