docs: consolidate exploration & audit reports under docs/ (TEC-3094)
- Move 8 stray .md (+5 .txt) from ~/Desktop into docs/explorations/from-desktop/ - Reorganize 27 .md/.txt at workspace root: - audit reports -> docs/audits/ - exploration reports -> docs/explorations/ - design system -> docs/design-system/ - Keep only README/CHANGELOG/CONTRIBUTING/CLAUDE at repo root - Refresh docs/README.md as canonical index with links to all groups - Note: pre-existing docs/audits/AUDIT_INDEX.md and AUDIT_SUMMARY.md were overwritten by the newer root-level versions during the move Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Ho Ngoc Hai
387
docs/explorations/EXPLORATION_SUMMARY.md
Normal file
387
docs/explorations/EXPLORATION_SUMMARY.md
Normal file
@@ -0,0 +1,387 @@
|
||||
# Analytics Module Exploration - Summary Report
|
||||
|
||||
## ✅ Exploration Complete
|
||||
|
||||
I've thoroughly explored the GoodGo Platform analytics module and created comprehensive documentation. Here's what was analyzed:
|
||||
|
||||
---
|
||||
|
||||
## 📄 Documentation Created
|
||||
|
||||
1. **ANALYTICS_ARCHITECTURE.md** (Comprehensive, 10 sections)
|
||||
- Full module structure with DDD/CQRS layers
|
||||
- Controller & endpoint mapping
|
||||
- Query/handler CQRS pattern deep-dive
|
||||
- Redis caching patterns with Lua scripts
|
||||
- Prisma schema for Property, Listing, MarketIndex, Valuation models
|
||||
- Shared guards, decorators, and exception patterns
|
||||
- DTO patterns and module dependency injection
|
||||
- Key patterns and quick reference paths
|
||||
|
||||
2. **ANALYTICS_QUICK_REFERENCE.md** (Developer-friendly reference)
|
||||
- Quick architecture overview
|
||||
- File paths and module organization
|
||||
- Guard & decorator stack with usage examples
|
||||
- Cache patterns (cache-aside, TTLs, prefixes)
|
||||
- Prisma models summary
|
||||
- CQRS handler pattern code example
|
||||
- Error handling pattern
|
||||
- Common endpoints list
|
||||
- Dependency injection patterns
|
||||
- Key conventions table
|
||||
|
||||
3. **ANALYTICS_ARCHITECTURE_DIAGRAM.txt** (Visual reference)
|
||||
- Complete system architecture diagram
|
||||
- Data flow walkthrough for example request
|
||||
- All layers from HTTP → Database
|
||||
- External service integrations
|
||||
- Shared utilities & exports
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Key Findings
|
||||
|
||||
### 1. Architecture Pattern: **DDD + CQRS**
|
||||
- **Presentation**: Controllers (analytics, avm) + DTOs
|
||||
- **Application**: Query/Command handlers with Prometheus metrics
|
||||
- **Domain**: Repository interfaces, service abstractions, entities
|
||||
- **Infrastructure**: Prisma repositories, external service clients
|
||||
|
||||
### 2. Controllers (2 total)
|
||||
- `AnalyticsController` → `/analytics/*` (13+ endpoints)
|
||||
- `AvmController` → `/avm/*` (5 endpoints)
|
||||
|
||||
### 3. Query Handlers (14+)
|
||||
All follow cache-aside pattern:
|
||||
- Price trends, heatmaps, market reports, district stats
|
||||
- Valuations (single, batch, history, comparison, explanation)
|
||||
- Industrial valuation
|
||||
- Neighborhood scores, nearby POIs
|
||||
- AI advice (Claude integration for listings/projects)
|
||||
|
||||
### 4. Redis Caching Strategy
|
||||
**Pattern**: Cache-aside with graceful degradation
|
||||
```
|
||||
cache.getOrSet(key, loader, TTL, metricLabel)
|
||||
```
|
||||
|
||||
**TTLs for analytics**:
|
||||
- MARKET_DATA: 1800s (30 min) - price trends
|
||||
- MARKET_REPORT: 900s (15 min) - summaries
|
||||
- HEATMAP: 300s (5 min) - tiles
|
||||
- DISTRICT_STATS: 300s (5 min) - statistics
|
||||
|
||||
**Cache Prefixes**:
|
||||
```
|
||||
cache:market:report
|
||||
cache:market:trend
|
||||
cache:market:heatmap
|
||||
cache:market:district
|
||||
cache:valuation
|
||||
```
|
||||
|
||||
### 5. Rate Limiting (Redis Sliding-Window)
|
||||
**Guard**: `EndpointRateLimitGuard`
|
||||
**Decorator**: `@EndpointRateLimit({ limit, windowSeconds, keyStrategy })`
|
||||
**Implementation**: Lua script with sorted set (ZSET) in Redis
|
||||
**Strategy**: `'user'` (by user ID) or `'ip'` (by client IP)
|
||||
|
||||
Example:
|
||||
```typescript
|
||||
@EndpointRateLimit({ limit: 10, windowSeconds: 60, keyStrategy: 'user' })
|
||||
@UseGuards(EndpointRateLimitGuard, JwtAuthGuard, QuotaGuard)
|
||||
```
|
||||
|
||||
### 6. Guards Stack (Order Matters)
|
||||
1. `EndpointRateLimitGuard` → Redis rate limit
|
||||
2. `JwtAuthGuard` → JWT verification
|
||||
3. `QuotaGuard` → Subscription quota check
|
||||
|
||||
### 7. Prisma Schema - Key Models
|
||||
|
||||
**Property**:
|
||||
- propertyType (APARTMENT, HOUSE, LAND, COMMERCIAL)
|
||||
- status (ACTIVE, SOLD, RENTED, REMOVED)
|
||||
- district, city, location (PostGIS geometry)
|
||||
- areaM2, bedrooms, bathrooms, yearBuilt, etc.
|
||||
|
||||
**Listing** (analytics-aware):
|
||||
- priceVND (BigInt, checked > 0)
|
||||
- pricePerM2 (float, derived for analytics)
|
||||
- aiPriceEstimate, aiConfidence (AVM fields)
|
||||
- viewCount, saveCount, inquiryCount (tracking)
|
||||
- publishedAt, createdAt, status
|
||||
|
||||
**MarketIndex** (pre-calculated):
|
||||
- medianPrice, avgPriceM2, totalListings
|
||||
- daysOnMarket, inventoryLevel, absorptionRate
|
||||
- yoyChange (year-over-year)
|
||||
- Unique index: (district, city, propertyType, period)
|
||||
|
||||
**Valuation** (AVM storage):
|
||||
- estimatedPrice, confidence
|
||||
- drivers (JSON), comparables (JSON)
|
||||
- explanation, model version
|
||||
|
||||
### 8. Repository Pattern
|
||||
**Interface** (domain):
|
||||
```typescript
|
||||
export const MARKET_INDEX_REPOSITORY = Symbol('MARKET_INDEX_REPOSITORY');
|
||||
export interface IMarketIndexRepository {
|
||||
findById(id: string): Promise<MarketIndexEntity | null>;
|
||||
getMarketReport(...): Promise<MarketReportResult[]>;
|
||||
getPriceTrend(...): Promise<PriceTrendPoint[]>;
|
||||
getHeatmap(...): Promise<HeatmapDataPoint[]>;
|
||||
getDistrictStats(...): Promise<DistrictStatsResult[]>;
|
||||
}
|
||||
```
|
||||
|
||||
**Implementation** (infrastructure):
|
||||
```typescript
|
||||
@Injectable()
|
||||
export class PrismaMarketIndexRepository implements IMarketIndexRepository {
|
||||
constructor(private readonly prisma: PrismaService) {}
|
||||
// Converts Prisma → Domain entities
|
||||
}
|
||||
```
|
||||
|
||||
**Injection** (handler):
|
||||
```typescript
|
||||
constructor(
|
||||
@Inject(MARKET_INDEX_REPOSITORY)
|
||||
private readonly repo: IMarketIndexRepository
|
||||
) {}
|
||||
```
|
||||
|
||||
### 9. Error Handling Pattern
|
||||
```typescript
|
||||
try {
|
||||
return this.cache.getOrSet(...);
|
||||
} catch (error) {
|
||||
if (error instanceof DomainException) throw error; // Re-throw
|
||||
this.logger.error(...); // Log with context
|
||||
throw new InternalServerErrorException('...'); // Wrap & return user message
|
||||
}
|
||||
```
|
||||
|
||||
### 10. Shared Module Exports
|
||||
From `@modules/shared`:
|
||||
- **CacheService** with `getOrSet()` method
|
||||
- **RedisService** connection pool
|
||||
- **LoggerService** structured logging
|
||||
- **PrismaService** database access
|
||||
- **DomainException** & subclasses
|
||||
- **EndpointRateLimit** decorator
|
||||
- **EndpointRateLimitGuard** guard
|
||||
- **JwtAuthGuard**, **QuotaGuard**
|
||||
- Error response standardization via GlobalExceptionFilter
|
||||
|
||||
### 11. DTO Conventions
|
||||
- **Request DTOs**: Query parameters, body validation
|
||||
- **Response DTOs**: Defined as handler return type interfaces
|
||||
- **BigInt handling**: Always stringified for JSON safety (`.toString()`)
|
||||
|
||||
---
|
||||
|
||||
## 🔗 File Path Quick Map
|
||||
|
||||
```
|
||||
ANALYTICS ROOT
|
||||
└── apps/api/src/modules/analytics/
|
||||
|
||||
CONTROLLERS
|
||||
├── presentation/controllers/analytics.controller.ts (13+ endpoints)
|
||||
└── presentation/controllers/avm.controller.ts (5 endpoints)
|
||||
|
||||
QUERY HANDLERS (14+ total)
|
||||
├── application/queries/get-price-trend/ ← Cache-aside pattern
|
||||
├── application/queries/get-heatmap/
|
||||
├── application/queries/get-market-report/
|
||||
├── application/queries/get-valuation/
|
||||
├── application/queries/predict-valuation/
|
||||
├── application/queries/batch-valuation/
|
||||
├── application/queries/valuation-history/
|
||||
├── application/queries/valuation-comparison/
|
||||
├── application/queries/valuation-explanation/
|
||||
├── application/queries/get-neighborhood-score/
|
||||
├── application/queries/get-nearby-pois/
|
||||
├── application/queries/get-listing-ai-advice/ (Claude)
|
||||
└── application/queries/get-project-ai-advice/ (Claude)
|
||||
|
||||
COMMAND HANDLERS (3)
|
||||
├── application/commands/generate-report/
|
||||
├── application/commands/track-event/
|
||||
└── application/commands/update-market-index/
|
||||
|
||||
EVENT HANDLERS (1)
|
||||
└── application/event-handlers/listing-created-moderation.handler.ts
|
||||
|
||||
REPOSITORIES
|
||||
├── domain/repositories/market-index.repository.ts (interface)
|
||||
├── domain/repositories/valuation.repository.ts (interface)
|
||||
├── infrastructure/repositories/prisma-market-index.repository.ts
|
||||
└── infrastructure/repositories/prisma-valuation.repository.ts
|
||||
|
||||
SERVICES
|
||||
├── infrastructure/services/http-avm.service.ts (→ Python AI)
|
||||
├── infrastructure/services/prisma-avm.service.ts (fallback)
|
||||
├── infrastructure/services/http-neighborhood-score.service.ts
|
||||
├── infrastructure/services/prisma-neighborhood-score.service.ts
|
||||
├── infrastructure/services/ai-service.client.ts (Claude)
|
||||
└── infrastructure/services/market-index-cron.service.ts
|
||||
|
||||
SHARED MODULE (Global)
|
||||
└── apps/api/src/modules/shared/
|
||||
├── infrastructure/
|
||||
│ ├── cache.service.ts ← Cache-aside
|
||||
│ ├── redis.service.ts ← Connection pool
|
||||
│ ├── logger.service.ts ← Structured logging
|
||||
│ ├── prisma.service.ts ← Database
|
||||
│ ├── guards/endpoint-rate-limit.guard.ts ← Lua sliding-window
|
||||
│ ├── decorators/endpoint-rate-limit.decorator.ts
|
||||
│ ├── middleware/correlation-id.middleware.ts ← Trace ID
|
||||
│ └── filters/global-exception.filter.ts ← Error standardization
|
||||
└── domain/
|
||||
├── domain-exception.ts ← Exception base
|
||||
└── error-codes.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💡 Implementation Patterns to Follow
|
||||
|
||||
### 1. New Query Handler Template
|
||||
```typescript
|
||||
@QueryHandler(YourQuery)
|
||||
export class YourQueryHandler implements IQueryHandler<YourQuery> {
|
||||
constructor(
|
||||
@Inject(YOUR_REPOSITORY)
|
||||
private readonly repo: IYourRepository,
|
||||
private readonly cache: CacheService,
|
||||
private readonly logger: LoggerService,
|
||||
) {}
|
||||
|
||||
async execute(query: YourQuery): Promise<YourDto> {
|
||||
try {
|
||||
const cacheKey = CacheService.buildKey(
|
||||
CachePrefix.YOUR_PREFIX,
|
||||
query.param1,
|
||||
query.param2,
|
||||
);
|
||||
|
||||
return this.cache.getOrSet(
|
||||
cacheKey,
|
||||
async () => {
|
||||
const data = await this.repo.yourMethod(...);
|
||||
return { ...data };
|
||||
},
|
||||
CacheTTL.YOUR_TTL,
|
||||
'your_metric_label',
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof DomainException) throw error;
|
||||
this.logger.error(`Failed to ...`, error?.stack, this.constructor.name);
|
||||
throw new InternalServerErrorException('User-friendly message');
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. New Controller Endpoint Template
|
||||
```typescript
|
||||
@ApiBearerAuth('JWT')
|
||||
@EndpointRateLimit({ limit: 10, windowSeconds: 60, keyStrategy: 'user' })
|
||||
@UseGuards(EndpointRateLimitGuard, JwtAuthGuard, QuotaGuard)
|
||||
@RequireQuota('analytics_queries')
|
||||
@Get('your-endpoint')
|
||||
@ApiOperation({ summary: 'Description' })
|
||||
async yourMethod(@Query() dto: YourDto): Promise<YourResultDto> {
|
||||
return this.queryBus.execute(
|
||||
new YourQuery(dto.param1, dto.param2, ...)
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Register Handler in Module
|
||||
```typescript
|
||||
const QueryHandlers = [
|
||||
// existing handlers...
|
||||
YourQueryHandler,
|
||||
];
|
||||
|
||||
@Module({
|
||||
providers: [
|
||||
...QueryHandlers,
|
||||
],
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Next Steps for Implementation
|
||||
|
||||
When building new analytics features:
|
||||
|
||||
1. **Define Cache Strategy**
|
||||
- Choose TTL from `CacheTTL.*` or create new one
|
||||
- Use appropriate `CachePrefix.*`
|
||||
|
||||
2. **Create Query & Handler**
|
||||
- Query class: simple data holder
|
||||
- Handler: cache-aside + repository call
|
||||
|
||||
3. **Define DTO**
|
||||
- Request DTO for controller parameters
|
||||
- Response DTO as handler return type
|
||||
|
||||
4. **Add Controller Endpoint**
|
||||
- Use guard stack: `EndpointRateLimitGuard` → `JwtAuthGuard` → `QuotaGuard`
|
||||
- Call `queryBus.execute()`
|
||||
|
||||
5. **Register in Module**
|
||||
- Add handler to `QueryHandlers` array
|
||||
- Add to module `providers`
|
||||
|
||||
6. **Error Handling**
|
||||
- Catch and rethrow `DomainException`
|
||||
- Log unexpected errors with context
|
||||
- Return user-friendly message
|
||||
|
||||
---
|
||||
|
||||
## 📊 Statistics
|
||||
|
||||
- **Controllers**: 2
|
||||
- **Query Handlers**: 14+
|
||||
- **Command Handlers**: 3
|
||||
- **Event Handlers**: 1
|
||||
- **DTOs**: 15+
|
||||
- **Repositories**: 2 interfaces + 2 implementations
|
||||
- **Cache Prefixes**: 12
|
||||
- **TTLs Configured**: 10+
|
||||
- **Endpoints**: 18+
|
||||
- **Rate Limit Configurations**: Multiple per endpoint
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Architecture Highlights
|
||||
|
||||
✅ **DDD Layers**: Clear separation of concerns
|
||||
✅ **CQRS Pattern**: Query/Command handlers with event sourcing capability
|
||||
✅ **Cache-Aside Pattern**: Redis caching with graceful degradation
|
||||
✅ **Sliding-Window Rate Limiting**: Accurate per-endpoint limiting with Redis
|
||||
✅ **Dependency Injection**: Repository pattern with interface abstraction
|
||||
✅ **Error Standardization**: Global exception filter with error codes
|
||||
✅ **Prometheus Metrics**: Cache hit/miss/degradation tracking
|
||||
✅ **Middleware Stack**: Correlation ID, audit logging, CSRF, input sanitization
|
||||
✅ **External Service Fallback**: HTTP → Python AI with Prisma fallback
|
||||
✅ **Quota Management**: Subscription-based quota enforcement per resource
|
||||
|
||||
---
|
||||
|
||||
**All documentation files saved to project root:**
|
||||
- `ANALYTICS_ARCHITECTURE.md` - Comprehensive reference
|
||||
- `ANALYTICS_QUICK_REFERENCE.md` - Developer quick guide
|
||||
- `ANALYTICS_ARCHITECTURE_DIAGRAM.txt` - Visual overview
|
||||
- `EXPLORATION_SUMMARY.md` - This file
|
||||
|
||||
Reference in New Issue
Block a user