- Move 8 stray .md (+5 .txt) from ~/Desktop into docs/explorations/from-desktop/ - Reorganize 27 .md/.txt at workspace root: - audit reports -> docs/audits/ - exploration reports -> docs/explorations/ - design system -> docs/design-system/ - Keep only README/CHANGELOG/CONTRIBUTING/CLAUDE at repo root - Refresh docs/README.md as canonical index with links to all groups - Note: pre-existing docs/audits/AUDIT_INDEX.md and AUDIT_SUMMARY.md were overwritten by the newer root-level versions during the move Co-Authored-By: Paperclip <noreply@paperclip.ing>
313 lines
11 KiB
Markdown
313 lines
11 KiB
Markdown
# 🎯 Analytics Module Architecture Exploration — Complete
|
||
|
||
## Documents Created
|
||
|
||
I've prepared **3 comprehensive guides** for you:
|
||
|
||
### 1️⃣ **analytics_architecture_guide.md** (36 KB)
|
||
**The Comprehensive Reference** — Read this first for deep understanding
|
||
- ✅ DDD layer breakdown with code examples
|
||
- ✅ All 24 endpoints documented
|
||
- ✅ Query handler patterns (2 styles: @Cacheable vs manual)
|
||
- ✅ Complete caching system (Redis patterns, TTLs, invalidation)
|
||
- ✅ Real code examples from `GetMarketSnapshotHandler`, `GetDistrictStatsHandler`
|
||
- ✅ Full Prisma schema for Property, Listing, MarketIndex, Valuation
|
||
- ✅ Shared module utilities (CacheService, Cacheable decorator, interceptor)
|
||
- ✅ **7-step guide: How to add GET /analytics/trending-areas endpoint**
|
||
- ✅ Testing patterns
|
||
- ✅ Error handling conventions
|
||
|
||
### 2️⃣ **quick_reference.md** (8 KB)
|
||
**The Visual Quick Start** — Use this to navigate fast
|
||
- ✅ Layer stack diagram (Presentation → Application → Domain → Infrastructure)
|
||
- ✅ Request flow example (HTTP → Controller → QueryHandler → Cache → DB)
|
||
- ✅ Caching strategy matrix (when to cache, TTLs, prefixes)
|
||
- ✅ Decorators & guards cheat sheet
|
||
- ✅ Prisma schema snapshot
|
||
- ✅ Response structure with/without cache metadata
|
||
- ✅ 7-step endpoint addition checklist
|
||
|
||
### 3️⃣ **file_paths_reference.md** (8 KB)
|
||
**The Navigation Map** — Find files & understand structure
|
||
- ✅ Core module files (analytics.module.ts, index.ts)
|
||
- ✅ All 24 endpoints mapped to file paths
|
||
- ✅ DTO files organized by type (request vs response)
|
||
- ✅ All 15+ query types with descriptions
|
||
- ✅ Domain, Infrastructure, and Shared layer breakdowns
|
||
- ✅ Database schema models with fields & indexes
|
||
- ✅ Directory tree with line counts
|
||
- ✅ Import patterns reference
|
||
- ✅ Key metrics & numbers
|
||
|
||
---
|
||
|
||
## ✨ Key Findings
|
||
|
||
### Architecture Pattern
|
||
```
|
||
Domain-Driven Design (DDD) + CQRS (Command Query Responsibility Segregation)
|
||
4-layer structure: Presentation → Application → Domain → Infrastructure
|
||
```
|
||
|
||
### Controllers (Entry Points)
|
||
- **AnalyticsController**: 19 endpoints (`/analytics/...`)
|
||
- **AvmController**: 5 endpoints (`/avm/...`)
|
||
- **Total**: 24 endpoints, all with guards (JWT, Quota, Rate Limit)
|
||
|
||
### Query Handlers: 2 Caching Patterns
|
||
|
||
**Pattern 1: @Cacheable Decorator** (Simpler)
|
||
```ts
|
||
@QueryHandler(GetDistrictStatsQuery)
|
||
export class GetDistrictStatsHandler {
|
||
@Cacheable({
|
||
prefix: CachePrefix.MARKET_DISTRICT,
|
||
ttl: CacheTTL.DISTRICT_STATS,
|
||
resource: 'district_stats',
|
||
keyFrom: (query) => [query.city, query.period],
|
||
})
|
||
async execute(query): Promise<DistrictStatsDto> {
|
||
return this.marketIndexRepo.getDistrictStats(query.city, query.period);
|
||
}
|
||
}
|
||
```
|
||
|
||
**Pattern 2: cache.getOrSet()** (For complex computation)
|
||
```ts
|
||
async execute(query): Promise<MarketSnapshotDto> {
|
||
const cacheKey = CacheService.buildKey(CachePrefix.MARKET_SNAPSHOT, query.city);
|
||
return await this.cache.getOrSet(
|
||
cacheKey,
|
||
() => this.computeSnapshot(query.city), // Heavy computation
|
||
CacheTTL.MARKET_SNAPSHOT, // TTL in seconds
|
||
'market_snapshot', // Prometheus label
|
||
);
|
||
}
|
||
```
|
||
|
||
### Redis Caching Strategy
|
||
- **Cache-aside pattern**: Try Redis → if miss, call loader, store result
|
||
- **Envelope format**: `{ __v: data, cachedAt: ISO, ttlSeconds: 300 }`
|
||
- **Graceful degradation**: If Redis down, calls loader directly (no error)
|
||
- **Metrics**: `cache_hit_total`, `cache_miss_total`, `cache_degradation_total`
|
||
- **TTLs**: Dashboard=300s, Reports=900s, Trends=1800s, Predictions=NO_CACHE
|
||
|
||
### Prisma Schema
|
||
- **Property**: id, type, address, district, city, location (PostGIS Point), area, rooms, etc.
|
||
- **Listing**: id, propertyId, sellerId, status, priceVND (BigInt), aiPriceEstimate, publishedAt
|
||
- **MarketIndex**: district, city, propertyType, period; medianPrice (BigInt), avgPriceM2, stats
|
||
- **Valuation**: id, propertyId, estimatedPrice, confidence, method, features (Json)
|
||
|
||
### DDD Layers
|
||
1. **Presentation**: Controllers, DTOs, Interceptors
|
||
2. **Application**: Query/Command Handlers (@QueryHandler, @CommandHandler)
|
||
3. **Domain**: Entities, Repository Interfaces, Service Interfaces
|
||
4. **Infrastructure**: Prisma Repositories, HTTP Services, External clients
|
||
|
||
### Shared Module Utilities
|
||
- **CacheService**: Core cache-aside with Redis
|
||
- **@Cacheable**: Method decorator for handlers
|
||
- **CacheMetaInterceptor**: Wraps responses with `{ data, cacheMeta }`
|
||
- **LoggerService**: Winston-based logging
|
||
- **PrismaService**: ORM wrapper
|
||
- **RedisService**: Redis client wrapper
|
||
- **Guards**: JWT, Quota, Rate Limit
|
||
|
||
---
|
||
|
||
## 🚀 Quick Start: Adding New Endpoint
|
||
|
||
Example: **GET /analytics/trending-areas**
|
||
|
||
### 7-Step Process
|
||
|
||
1. **Request DTO** (`presentation/dto/get-trending-areas.dto.ts`)
|
||
```ts
|
||
export class GetTrendingAreasDto {
|
||
@IsOptional() city?: string = 'Hồ Chí Minh';
|
||
@IsOptional() propertyType?: PropertyType;
|
||
@IsOptional() @Min(1) limit?: number = 10;
|
||
}
|
||
```
|
||
|
||
2. **Query Class** (`application/queries/get-trending-areas/get-trending-areas.query.ts`)
|
||
```ts
|
||
export class GetTrendingAreasQuery {
|
||
constructor(
|
||
public readonly city: string,
|
||
public readonly propertyType: PropertyType | undefined,
|
||
public readonly limit: number,
|
||
) {}
|
||
}
|
||
```
|
||
|
||
3. **Handler** (`application/queries/get-trending-areas/get-trending-areas.handler.ts`)
|
||
```ts
|
||
@QueryHandler(GetTrendingAreasQuery)
|
||
export class GetTrendingAreasHandler implements IQueryHandler {
|
||
@Cacheable({
|
||
prefix: CachePrefix.TRENDING_AREAS,
|
||
ttl: CacheTTL.TRENDING_AREAS,
|
||
resource: 'trending_areas',
|
||
keyFrom: (query) => [query.city, query.propertyType, query.limit],
|
||
})
|
||
async execute(query: GetTrendingAreasQuery): Promise<GetTrendingAreasDto> {
|
||
// Your logic here
|
||
return { city: query.city, areas: [...], cachedAt: null, nextRefreshAt: null };
|
||
}
|
||
}
|
||
|
||
export interface GetTrendingAreasDto {
|
||
city: string;
|
||
areas: TrendingAreaDto[];
|
||
cachedAt: string | null;
|
||
nextRefreshAt: string | null;
|
||
}
|
||
```
|
||
|
||
4. **Register Handler** (in `analytics.module.ts`)
|
||
```ts
|
||
const QueryHandlers = [
|
||
// ... existing
|
||
GetTrendingAreasHandler, // Add here
|
||
];
|
||
```
|
||
|
||
5. **Controller Method** (in `analytics.controller.ts`)
|
||
```ts
|
||
@ApiBearerAuth('JWT')
|
||
@UseGuards(JwtAuthGuard, QuotaGuard)
|
||
@RequireQuota('analytics_queries')
|
||
@Get('trending-areas')
|
||
@ApiOperation({ summary: 'Get trending districts' })
|
||
@ApiResponse({ status: 200, description: 'Trending areas retrieved' })
|
||
async getTrendingAreas(@Query() dto: GetTrendingAreasDto): Promise<GetTrendingAreasDto> {
|
||
return this.queryBus.execute(
|
||
new GetTrendingAreasQuery(dto.city || 'Hồ Chí Minh', dto.propertyType, dto.limit || 10),
|
||
);
|
||
}
|
||
```
|
||
|
||
6. **Export DTOs** (in `presentation/dto/index.ts`)
|
||
```ts
|
||
export * from './get-trending-areas.dto';
|
||
```
|
||
|
||
7. **Test** (in `__tests__/get-trending-areas.handler.spec.ts`)
|
||
```ts
|
||
it('should return trending areas', async () => {
|
||
const query = new GetTrendingAreasQuery('Hồ Chí Minh', undefined, 10);
|
||
const result = await handler.execute(query);
|
||
expect(result.areas).toBeDefined();
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 Architecture Decision Points
|
||
|
||
| Decision | Current Approach | Why |
|
||
|----------|------------------|-----|
|
||
| **Caching** | Redis + cache-aside | TTL-based expiry is simple & performant |
|
||
| **Cache Invalidation** | Prefix-based SCAN | Non-blocking, doesn't require key enumeration |
|
||
| **Cache Metadata** | AsyncLocalStorage + Interceptor | Per-request context without global state |
|
||
| **Query Patterns** | CQRS with QueryBus | Separates reads from writes, enables caching layer |
|
||
| **Rate Limiting** | EndpointRateLimitGuard | Per-endpoint control, different rates for different ops |
|
||
| **Quota Metering** | @RequireQuota decorator | Subscription-aware, tracks usage |
|
||
| **Response Format** | DTO with cache metadata | Frontend knows freshness of data |
|
||
| **Error Handling** | DomainException + InternalServerError | Differentiates logic errors from system errors |
|
||
| **Graceful Degradation** | Cache bypass if Redis down | Service stays up during Redis maintenance |
|
||
|
||
---
|
||
|
||
## 🎯 Core Conventions to Remember
|
||
|
||
✅ **Always cache with TTL**
|
||
- Dashboard tiles: 300s
|
||
- Aggregations: 300s
|
||
- Reports: 900s
|
||
- Trends: 1800s
|
||
- Predictions: NO CACHE (always fresh)
|
||
|
||
✅ **Always use CacheService.buildKey()**
|
||
- Ensures deterministic, lowercase keys
|
||
- Replaces spaces with underscores
|
||
- Format: `prefix:param1:param2:param3`
|
||
|
||
✅ **Always wrap handlers in try-catch**
|
||
```ts
|
||
try {
|
||
// logic
|
||
} catch (error) {
|
||
if (error instanceof DomainException) throw error;
|
||
this.logger.error(...);
|
||
throw new InternalServerErrorException('...');
|
||
}
|
||
```
|
||
|
||
✅ **Always return DTO with null metadata**
|
||
```ts
|
||
return {
|
||
// ...data fields
|
||
cachedAt: null, // Filled by CacheMetaInterceptor
|
||
nextRefreshAt: null, // Filled by CacheMetaInterceptor
|
||
};
|
||
```
|
||
|
||
✅ **Always use @UseInterceptors(CacheMetaInterceptor)**
|
||
- On controllers to wrap response
|
||
- Adds: `{ data: T, cacheMeta: { cachedAt, nextRefreshAt, source } }`
|
||
|
||
✅ **Always add guards to endpoints**
|
||
```ts
|
||
@UseGuards(JwtAuthGuard, QuotaGuard) // Auth + quota check
|
||
@RequireQuota('analytics_queries') // Meter usage
|
||
@EndpointRateLimit({ limit: 10, windowSeconds: 60 }) // Rate limit if needed
|
||
```
|
||
|
||
---
|
||
|
||
## 📁 File Summary
|
||
|
||
| File Path | Purpose | Lines |
|
||
|-----------|---------|-------|
|
||
| `presentation/controllers/analytics.controller.ts` | Main endpoints | 331 |
|
||
| `presentation/controllers/avm.controller.ts` | Valuation endpoints | 171 |
|
||
| `application/queries/*/get-*.handler.ts` | Query execution + caching | 50-100 ea |
|
||
| `domain/repositories/market-index.repository.ts` | Repository interface | 58 |
|
||
| `infrastructure/repositories/prisma-market-index.repository.ts` | Prisma implementation | 150+ |
|
||
| `presentation/interceptors/cache-meta.interceptor.ts` | Response wrapper | 61 |
|
||
| `../shared/infrastructure/cache.service.ts` | Redis cache layer | 191 |
|
||
| `../shared/infrastructure/decorators/cacheable.decorator.ts` | @Cacheable | 57 |
|
||
| `analytics.module.ts` | NestJS module definition | 102 |
|
||
|
||
---
|
||
|
||
## 🎓 Next Steps
|
||
|
||
1. **Read** → Start with `quick_reference.md` for visual understanding
|
||
2. **Reference** → Use `file_paths_reference.md` to find specific files
|
||
3. **Deep Dive** → Study `analytics_architecture_guide.md` for patterns & code
|
||
4. **Build** → Follow the 7-step checklist to add your first endpoint
|
||
5. **Test** → Create query handler spec following existing patterns
|
||
|
||
---
|
||
|
||
## Questions to Validate Understanding
|
||
|
||
After reading the guides, you should be able to answer:
|
||
|
||
1. What are the 4 DDD layers and what goes in each?
|
||
2. How does the cache-aside pattern work when Redis is down?
|
||
3. What's the difference between @Cacheable and cache.getOrSet()?
|
||
4. Why do response DTOs have `cachedAt: null` and `nextRefreshAt: null`?
|
||
5. How is the cache key built deterministically?
|
||
6. What TTLs are used for different endpoint types?
|
||
7. What guards are required on all analytics endpoints?
|
||
8. How do you add a new GET endpoint in 7 steps?
|
||
9. What's the purpose of CacheMetaInterceptor?
|
||
10. How does graceful degradation work if Redis is unavailable?
|
||
|
||
---
|
||
|