- Move 8 stray .md (+5 .txt) from ~/Desktop into docs/explorations/from-desktop/ - Reorganize 27 .md/.txt at workspace root: - audit reports -> docs/audits/ - exploration reports -> docs/explorations/ - design system -> docs/design-system/ - Keep only README/CHANGELOG/CONTRIBUTING/CLAUDE at repo root - Refresh docs/README.md as canonical index with links to all groups - Note: pre-existing docs/audits/AUDIT_INDEX.md and AUDIT_SUMMARY.md were overwritten by the newer root-level versions during the move Co-Authored-By: Paperclip <noreply@paperclip.ing>
25 KiB
25 KiB
GoodGo Analytics Module - Architecture Exploration Summary
1. MODULE STRUCTURE (DDD Layers)
File Organization
apps/api/src/modules/analytics/
├── analytics.module.ts # NestJS module (CQRS setup, DI)
├── index.ts # Exports
├── application/ # Application layer
│ ├── commands/ # Write operations
│ │ ├── generate-report/
│ │ ├── track-event/
│ │ └── update-market-index/
│ ├── queries/ # Read operations
│ │ ├── batch-valuation/
│ │ ├── get-district-stats/
│ │ ├── get-heatmap/
│ │ ├── get-listing-ai-advice/
│ │ ├── get-market-report/
│ │ ├── get-nearby-pois/
│ │ ├── get-neighborhood-score/
│ │ ├── get-price-trend/
│ │ ├── get-project-ai-advice/
│ │ ├── get-valuation/
│ │ ├── industrial-valuation/
│ │ ├── predict-valuation/
│ │ ├── valuation-comparison/
│ │ ├── valuation-explanation/
│ │ ├── valuation-history/
│ │ ├── _shared/ # Shared utilities
│ │ │ └── ai-json-client.ts
│ ├── event-handlers/
│ │ └── listing-created-moderation.handler.ts
│ └── __tests__/
├── domain/ # Domain layer
│ ├── repositories/
│ │ ├── market-index.repository.ts
│ │ ├── valuation.repository.ts
│ ├── services/
│ │ ├── avm-service.ts
│ │ └── neighborhood-score.service.ts
│ ├── entities/
│ └── aggregates/
├── infrastructure/ # Infrastructure layer
│ ├── repositories/
│ │ ├── prisma-market-index.repository.ts
│ │ └── prisma-valuation.repository.ts
│ ├── services/
│ │ ├── ai-service.client.ts
│ │ ├── http-avm.service.ts
│ │ ├── market-index-cron.service.ts
│ │ ├── neighborhood-score.service.ts
│ │ └── prisma-avm.service.ts
│ └── __tests__/
└── presentation/ # Presentation layer
├── controllers/
│ ├── analytics.controller.ts
│ ├── avm.controller.ts
│ └── index.ts
├── dto/ # Request/Response DTOs
│ ├── avm-compare-query.dto.ts
│ ├── avm-explain-query.dto.ts
│ ├── batch-valuation.dto.ts
│ ├── get-district-stats.dto.ts
│ ├── get-heatmap.dto.ts
│ ├── get-market-report.dto.ts
│ ├── get-nearby-pois.dto.ts
│ ├── get-price-trend.dto.ts
│ ├── get-valuation.dto.ts
│ ├── industrial-valuation.dto.ts
│ ├── predict-valuation.dto.ts
│ ├── valuation-comparison.dto.ts
│ ├── valuation-history.dto.ts
│ └── index.ts
├── __tests__/
└── index.ts
Architecture Pattern: DDD + CQRS
- Commands: State mutations (GenerateReportHandler, TrackEventHandler, UpdateMarketIndexHandler)
- Queries: Read operations with cache-aside pattern
- Event Handlers: Listen to domain events (listing-created-moderation)
- Repositories: Abstract data access (Market Index, Valuation)
- Services: Business logic (AVM, Neighborhood Scoring)
2. CONTROLLER & ENDPOINT STRUCTURE
Analytics Controller (analytics.controller.ts)
Routes: GET/POST /analytics/* and public endpoints
Key Endpoints:
GET /analytics/market-report → GetMarketReportQuery
GET /analytics/price-trend → GetPriceTrendQuery
GET /analytics/heatmap → GetHeatmapQuery
GET /analytics/district-stats → GetDistrictStatsQuery
GET /analytics/valuation → GetValuationQuery (by propertyId or coords)
POST /analytics/valuation → PredictValuationQuery (manual input form)
POST /analytics/valuation/batch → BatchValuationQuery (1-50 properties)
GET /analytics/valuation/history/:id → ValuationHistoryQuery
POST /analytics/valuation/compare → ValuationComparisonQuery (2-5 properties)
GET /analytics/neighborhoods/:district/score → GetNeighborhoodScoreQuery (no auth)
GET /analytics/pois/nearby → GetNearbyPOIsQuery (no auth, public)
POST /analytics/listings/:id/ai-advice → GetListingAiAdviceQuery (Claude)
POST /analytics/projects/:id/ai-advice → GetProjectAiAdviceQuery (Claude)
AVM Controller (avm.controller.ts)
Routes: GET/POST /avm/*
Key Endpoints:
POST /avm/batch → BatchValuationQuery
GET /avm/history/:propertyId → ValuationHistoryQuery
GET /avm/compare → ValuationComparisonQuery (query string: ids)
GET /avm/explain → ValuationExplanationQuery (valuationId)
POST /avm/industrial → IndustrialValuationQuery
Guard & Decorator Stack
@ApiBearerAuth('JWT') // Swagger doc
@UseGuards(
EndpointRateLimitGuard, // Redis sliding-window rate limit
JwtAuthGuard, // Verify JWT token
QuotaGuard // Check user subscription quota
)
@RequireQuota('analytics_queries') // Decorator: specify quota resource
@EndpointRateLimit({
limit: 10,
windowSeconds: 60,
keyStrategy: 'user' | 'ip' // Rate limit by authenticated user or IP
})
3. QUERY/HANDLER PATTERN (CQRS Implementation)
Query Definition (Example: GetPriceTrendQuery)
export class GetPriceTrendQuery {
constructor(
public readonly district: string,
public readonly city: string,
public readonly propertyType: PropertyType, // From @prisma/client
public readonly periods: string[],
) {}
}
Handler Pattern (Example: GetPriceTrendHandler)
@QueryHandler(GetPriceTrendQuery)
export class GetPriceTrendHandler implements IQueryHandler<GetPriceTrendQuery> {
constructor(
@Inject(MARKET_INDEX_REPOSITORY)
private readonly marketIndexRepo: IMarketIndexRepository,
private readonly cache: CacheService, // Cache-aside pattern
private readonly logger: LoggerService,
) {}
async execute(query: GetPriceTrendQuery): Promise<PriceTrendDto> {
try {
const cacheKey = CacheService.buildKey(
CachePrefix.MARKET_TREND,
query.district,
query.city,
query.propertyType,
query.periods?.join(','),
);
return this.cache.getOrSet(
cacheKey,
async () => {
const trend = await this.marketIndexRepo.getPriceTrend(...);
return { district, city, propertyType, trend };
},
CacheTTL.MARKET_DATA, // 30 minutes
'price_trend', // Metric label
);
} catch (error) {
if (error instanceof DomainException) throw error;
this.logger.error(...);
throw new InternalServerErrorException('...');
}
}
}
Key Pattern:
- Inject repository (abstraction) and cache service
- Build cache key with CacheService.buildKey()
- Use cache.getOrSet() for cache-aside pattern
- Specify TTL from CacheTTL constant and metric name
- Catch DomainException separately, rethrow or wrap
4. REDIS CACHING PATTERNS
Cache Configuration (CacheService)
Cache TTLs:
CacheTTL = {
LISTING_DETAIL: 300, // 5 min
SEARCH_RESULTS: 120, // 2 min
DISTRICT_STATS: 300, // 5 min
MARKET_REPORT: 900, // 15 min
HEATMAP: 300, // 5 min
MARKET_DATA: 1800, // 30 min (price trends)
USER_PROFILE: 600, // 10 min
USER_QUOTA: 60, // 1 min (frequently invalidated)
PLAN_LIST: 3600, // 1 hour
REFERENCE_DATA: 86400, // 24 hours (static districts/wards)
}
Cache Key Prefixes:
enum CachePrefix {
LISTING = 'cache:listing',
SEARCH = 'cache:search',
GEO_SEARCH = 'cache:geo_search',
MARKET_REPORT = 'cache:market:report',
MARKET_TREND = 'cache:market:trend', // For price trends
MARKET_HEATMAP = 'cache:market:heatmap',
MARKET_DISTRICT = 'cache:market:district',
USER_PROFILE = 'cache:user:profile',
USER_QUOTA = 'cache:user:quota',
VALUATION = 'cache:valuation',
PLAN_LIST = 'cache:plan:list',
REFERENCE = 'cache:reference',
AGENT_LISTINGS = 'cache:agent:listings',
}
Cache-Aside Implementation:
async getOrSet<T>(
key: string,
loader: () => Promise<T>,
ttlSeconds: number,
resource: string, // Metric label
): Promise<T> {
// 1. Fast-path: if Redis unavailable, call loader directly (graceful degradation)
if (!this.redis.isAvailable()) {
// Metric: cache_degradation_total[resource]++
return await loader();
}
// 2. Try to get from cache
// 3. If miss: call loader, store in Redis, return
// 4. Metrics: cache_hit_total or cache_miss_total[resource]++
}
Metrics Tracked:
cache_hit_total[resource] // Counter: hits by resource
cache_miss_total[resource] // Counter: misses by resource
cache_degradation_total[resource] // Counter: fallbacks when Redis down
Rate Limiting with Redis (Sliding Window)
Lua Script in EndpointRateLimitGuard:
-- Sliding-window algorithm using Redis sorted set
-- KEYS[1] = rate limit key (e.g., "rate:user:123:POST:/analytics/valuation")
-- ARGV[1] = now (ms timestamp)
-- ARGV[2] = windowMs (duration)
-- ARGV[3] = limit (max requests)
-- ARGV[4] = requestId (unique)
-- ARGV[5] = windowSec (TTL)
-- Remove entries older than window
redis.call('ZREMRANGEBYSCORE', key, 0, now - windowMs)
local current = redis.call('ZCARD', key)
if current < limit then
redis.call('ZADD', key, now, requestId) -- Add current request
redis.call('EXPIRE', key, windowSec + 1) -- Set expiry
return {current + 1, 0}
else
-- Compute Retry-After header
local oldest = redis.call('ZRANGE', key, 0, 0, 'WITHSCORES')
return {current, retryAfterMs}
end
Rate Limit Headers Returned:
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 5
X-RateLimit-Reset: 1640000000
Retry-After: 12
5. PRISMA SCHEMA - PROPERTY & LISTING MODELS
Property Model
model Property {
id String @id @default(cuid())
addressNormalized String?
propertyType PropertyType // APARTMENT, HOUSE, LAND, COMMERCIAL, etc.
status PropertyStatus // ACTIVE, SOLD, RENTED, REMOVED
areaM2 Float?
usableAreaM2 Float?
bedrooms Int?
bathrooms Int?
floors Int?
floor Int?
totalFloors Int?
direction Direction?
yearBuilt Int?
legalStatus String?
amenities Json?
nearbyPOIs Json?
metroDistanceM Float?
projectName String?
projectDevelopmentId String?
projectDevelopment ProjectDevelopment? @relation(...)
furnishing Furnishing?
propertyCondition PropertyCondition?
balconyDirection Direction?
maintenanceFeeVND BigInt?
parkingSlots Int?
viewType String[] @default([])
petFriendly Boolean?
suitableFor String[] @default([])
whyThisLocation String? @db.Text
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
listings Listing[]
valuations Valuation[]
media PropertyMedia[]
// Indexes for analytics queries
@@index([propertyType])
@@index([district, city])
@@index([location], type: Gist) // PostGIS spatial index
@@index([district, propertyType])
@@index([district, city, propertyType])
}
Listing Model (with Analytics Fields)
model Listing {
id String @id @default(cuid())
propertyId String
property Property @relation(...)
agentId String?
agent Agent? @relation(...)
sellerId String
seller User @relation(...)
transactionType TransactionType // BUY_SELL, RENT
status ListingStatus @default(DRAFT)
priceVND BigInt // CHECK: > 0
pricePerM2 Float? // Derived for analytics
rentPriceMonthly BigInt?
commissionPct Float? @default(2.0)
// AI Valuation fields
aiPriceEstimate BigInt? // AVM estimate
aiConfidence Float?
moderationScore Float?
moderationNotes String?
// Analytics tracking
viewCount Int @default(0)
saveCount Int @default(0)
inquiryCount Int @default(0)
// Lifecycle
featuredUntil DateTime?
expiresAt DateTime?
publishedAt DateTime?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
transactions Transaction[]
inquiries Inquiry[]
orders Order[]
priceHistories PriceHistory[]
savedByUsers SavedListing[]
@@index([status])
@@index([transactionType])
@@index([priceVND])
@@index([sellerId])
@@index([propertyId])
@@index([publishedAt])
@@index([createdAt])
@@index([status, createdAt(sort: Desc)]) // For market analytics
@@index([status, transactionType, priceVND]) // For price analysis
}
Price History Model
model PriceHistory {
id String @id @default(cuid())
listingId String
listing Listing @relation(...)
oldPrice BigInt // CHECK: > 0
newPrice BigInt // CHECK: > 0
source String @default("manual_update")
changedAt DateTime @default(now())
@@index([listingId, changedAt(sort: Desc)]) // For trend queries
}
Market Index Model (Pre-calculated Analytics)
model MarketIndex {
id String @id @default(cuid())
district String
city String
propertyType PropertyType
period String // "2024-Q1" or "2024-04" format
medianPrice BigInt
avgPriceM2 Float
totalListings Int
daysOnMarket Int
inventoryLevel Int
absorptionRate Float?
yoyChange Float? // Year-over-year % change
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@unique([district, city, propertyType, period])
@@index([city, period])
@@index([period, propertyType])
}
Valuation Model
model Valuation {
id String @id @default(cuid())
propertyId String
property Property @relation(...)
estimatedPrice BigInt
confidence Float // 0-1 confidence score
drivers Json? // Key price drivers
comparables Json? // Similar properties used
explanation String?
model String // "v1" or "v2"
createdAt DateTime @default(now())
@@index([propertyId, createdAt(sort: Desc)])
@@index([createdAt])
}
6. SHARED GUARDS & DECORATORS
File Locations
apps/api/src/modules/shared/
├── infrastructure/
│ ├── decorators/
│ │ ├── endpoint-rate-limit.decorator.ts ← Rate limit config
│ │ ├── user-rate-limit.decorator.ts
│ │ └── cacheable.decorator.ts
│ ├── guards/
│ │ ├── endpoint-rate-limit.guard.ts ← Sliding-window enforcement
│ │ ├── user-rate-limit.guard.ts
│ │ └── feature-listing-throttler.guard.ts
│ ├── middleware/
│ │ ├── correlation-id.middleware.ts ← Trace ID injection
│ │ ├── csrf.middleware.ts
│ │ ├── request-logging.middleware.ts ← Audit logging
│ │ └── sanitize-input.middleware.ts
│ ├── cache.service.ts ← Cache-aside + Redis
│ ├── redis.service.ts ← Redis connection pool
│ ├── logger.service.ts ← Structured logging
│ ├── prisma.service.ts
│ ├── field-encryption.service.ts
│ └── filters/
│ └── global-exception.filter.ts ← Error response standardization
├── domain/
│ ├── domain-exception.ts ← Base error class
│ ├── error-codes.ts
│ ├── base-entity.ts
│ ├── domain-event.ts
│ ├── aggregate-root.ts
│ └── value-object.ts
└── utils/
└── ...
Key Decorators & Guards
@EndpointRateLimit Decorator:
interface EndpointRateLimitOptions {
limit: number; // Max requests
windowSeconds?: number; // Default 60
keyStrategy?: 'ip' | 'user'; // Default 'ip'
adminBypass?: boolean; // Admins skip limit (default true)
}
// Usage:
@EndpointRateLimit({ limit: 10, windowSeconds: 60, keyStrategy: 'user' })
@UseGuards(EndpointRateLimitGuard, JwtAuthGuard)
async method() {}
Response Standardization (via GlobalExceptionFilter):
interface ErrorResponseBody {
statusCode: number;
errorCode: ErrorCode; // Enum: NOT_FOUND, VALIDATION_FAILED, etc.
message: string;
details?: Record<string, unknown>;
correlationId?: string; // From CorrelationIdMiddleware
timestamp: string;
}
Exception Hierarchy
class DomainException extends HttpException {
constructor(
errorCode: ErrorCode,
message: string,
statusCode?: HttpStatus,
details?: Record
)
}
// Specialized exceptions:
class NotFoundException extends DomainException
class ValidationException extends DomainException
class ConflictException extends DomainException
class UnauthorizedException extends DomainException
class ForbiddenException extends DomainException
7. DTO PATTERNS IN ANALYTICS
Request DTOs (Query Parameters)
// GET /analytics/market-report?city=...&period=...&propertyType=...
export class GetMarketReportDto {
@IsString() city: string;
@IsString() period: string;
@IsEnum(PropertyType) propertyType?: PropertyType;
}
// POST /analytics/valuation with body
export class PredictValuationDto {
@IsEnum(PropertyType) propertyType: PropertyType;
@IsNumber() area: number;
@IsString() district: string;
@IsString() city: string;
@IsNumber() latitude?: number;
@IsNumber() longitude?: number;
@IsNumber() bedrooms?: number;
@IsNumber() bathrooms?: number;
@IsBoolean() hasElevator?: boolean;
@IsBoolean() hasParking?: boolean;
@IsBoolean() hasPool?: boolean;
@IsNumber() distanceToHospitalKm?: number;
@IsNumber() distanceToParkKm?: number;
@IsNumber() distanceToMallKm?: number;
@IsString() floodZoneRisk?: string;
}
Response DTOs (Handler Return Types)
// From handler
export interface PriceTrendDto {
district: string;
city: string;
propertyType: string;
trend: PriceTrendPoint[];
}
export interface PriceTrendPoint {
period: string;
medianPrice: string; // Stringified BigInt
avgPriceM2: number;
totalListings: number;
}
// From repository
export interface MarketReportResult {
district: string;
city: string;
propertyType: PropertyType;
period: string;
medianPrice: string; // Stringified for JSON safety
avgPriceM2: number;
totalListings: number;
daysOnMarket: number;
inventoryLevel: number;
absorptionRate: number | null;
yoyChange: number | null;
}
8. DEPENDENCY INJECTION & MODULE EXPORTS
Analytics Module Registration
@Module({
imports: [CqrsModule, ListingsModule, AdminModule, ProjectsModule],
controllers: [AnalyticsController, AvmController],
providers: [
{ provide: AI_SERVICE_CLIENT, useClass: AiServiceClient },
{ provide: MARKET_INDEX_REPOSITORY, useClass: PrismaMarketIndexRepository },
{ provide: VALUATION_REPOSITORY, useClass: PrismaValuationRepository },
PrismaAVMService,
{ provide: AVM_SERVICE, useClass: HttpAVMService }, // HTTP → Python AI
PrismaNeighborhoodScoreService,
{ provide: NEIGHBORHOOD_SCORE_SERVICE, useClass: HttpNeighborhoodScoreService },
MarketIndexCronService,
...CommandHandlers,
...QueryHandlers,
...EventHandlers,
],
exports: [
MARKET_INDEX_REPOSITORY,
VALUATION_REPOSITORY,
AVM_SERVICE,
AI_SERVICE_CLIENT,
],
})
export class AnalyticsModule {}
Shared Module (Global)
@Global()
@Module({
imports: [ConfigModule.forRoot(...), EventEmitterModule.forRoot(), PrometheusModule.register(...)],
providers: [
LoggerService,
PrismaService,
RedisService, // Redis connection pool
CacheService, // Cache-aside pattern + metrics
EventBusService,
FieldEncryptionService,
// Prometheus metrics for cache
makeCounterProvider({ name: CACHE_HIT_TOTAL, labelNames: ['resource'] }),
makeCounterProvider({ name: CACHE_MISS_TOTAL, labelNames: ['resource'] }),
makeCounterProvider({ name: CACHE_DEGRADATION_TOTAL, labelNames: ['resource', 'operation'] }),
{ provide: APP_FILTER, useClass: GlobalExceptionFilter },
],
exports: [PrismaService, RedisService, CacheService, LoggerService, ...],
})
export class SharedModule implements NestModule {
// Middleware registration:
configure(consumer: MiddlewareConsumer) {
consumer
.apply(CorrelationIdMiddleware, SanitizeInputMiddleware, RequestLoggingMiddleware)
.forRoutes('*');
consumer
.apply(CsrfMiddleware)
.exclude([{ path: 'auth/login', method: RequestMethod.POST }, ...])
.forRoutes('*');
}
}
9. KEY PATTERNS & CONVENTIONS
Cache Key Building
CacheService.buildKey(CachePrefix.MARKET_TREND, district, city, propertyType, periods)
// → "cache:market:trend:{district}:{city}:{propertyType}:{periods}"
Error Handling Pattern
async execute(query: Query): Promise<Result> {
try {
const cacheKey = CacheService.buildKey(...);
return this.cache.getOrSet(cacheKey, async () => {
// Business logic
}, CacheTTL.MARKET_DATA, 'metric_label');
} catch (error) {
if (error instanceof DomainException) throw error; // Re-throw domain errors
this.logger.error(`Failed to ...`, error.stack, this.constructor.name);
throw new InternalServerErrorException('User-facing message');
}
}
Repository Pattern (Dependency Inversion)
// domain/repositories/market-index.repository.ts
export const MARKET_INDEX_REPOSITORY = Symbol('MARKET_INDEX_REPOSITORY');
export interface IMarketIndexRepository { ... }
// infrastructure/repositories/prisma-market-index.repository.ts
@Injectable()
export class PrismaMarketIndexRepository implements IMarketIndexRepository { ... }
// In handlers:
@Inject(MARKET_INDEX_REPOSITORY) private readonly repo: IMarketIndexRepository
Quota & Rate Limit Stack
@ApiBearerAuth('JWT')
@EndpointRateLimit({ limit: 10, windowSeconds: 60, keyStrategy: 'user' })
@UseGuards(EndpointRateLimitGuard, JwtAuthGuard, QuotaGuard) // Order matters!
@RequireQuota('analytics_queries') // Quota resource name
Query Handler Naming
- Query class:
GetPriceTrendQuery - Handler:
GetPriceTrendHandler(decorated with@QueryHandler(GetPriceTrendQuery)) - DTO:
GetPriceTrendDto(response type) - Query class file:
get-price-trend.query.ts - Handler file:
get-price-trend.handler.ts
10. QUICK REFERENCE: PATHS & KEYS
Module Root:
apps/api/src/modules/analytics/
Controllers:
analytics.controller.ts → /analytics/*
avm.controller.ts → /avm/*
Cache Prefixes (for market analytics):
cache:market:report → Market report data
cache:market:trend → Price trends
cache:market:heatmap → Heatmap data
cache:market:district → District statistics
cache:valuation → Property valuations
Shared Module Exports:
PrismaService → Database
RedisService → Redis connection
CacheService → Cache-aside + metrics
LoggerService → Structured logging
EventBusService → Event emission
Key Decorators:
@EndpointRateLimit({...}) → Per-endpoint sliding-window rate limit
@RequireQuota('...') → Subscription quota check
@UseGuards(...) → Auth, rate limit, quota guards
TTLs:
MARKET_DATA: 1800 → 30 minutes (price trends, historical)
MARKET_REPORT: 900 → 15 minutes (report summaries)
HEATMAP: 300 → 5 minutes (heatmap tiles)
DISTRICT_STATS: 300 → 5 minutes (statistics)