Commit Graph

27 Commits

Author SHA1 Message Date
Ho Ngoc Hai
1332c759f5 Merge feat/goo-175-phase3-ws3b-bull-board into master
Some checks failed
CI / E2E Tests (push) Has been skipped
CI / AI Services (Python) — Smoke (push) Failing after 7s
Deploy / Build API Image (push) Failing after 17s
Deploy / Build Web Image (push) Failing after 7s
CodeQL Analysis / CodeQL (javascript-typescript) (push) Failing after 1m16s
Deploy / Build AI Services Image (push) Failing after 6s
E2E Tests / Playwright E2E (push) Failing after 10s
Deploy / Deploy to Staging (push) Has been skipped
Deploy / Smoke Test Staging (push) Has been skipped
Deploy / Deploy to Production (push) Failing after 10m47s
Security Scanning / Dependency Audit (pnpm) (push) Failing after 6s
Deploy / Rollback Staging (push) Has been skipped
Security Scanning / Trivy Scan — API Image (push) Failing after 40s
Security Scanning / Trivy Scan — Web Image (push) Failing after 38s
Security Scanning / Trivy Scan — AI Services Image (push) Failing after 42s
Security Scanning / Trivy Filesystem Scan (push) Failing after 34s
Security Scanning / Security Gate (push) Failing after 3s
CI / Lint → Typecheck → Test → Build (22) (push) Failing after 12s
Deploy / Smoke Test Production (push) Has been cancelled
Deploy / Rollback Production (push) Has been cancelled
6 commits covering:
- BullMQ Redis split + Prometheus queue metrics + Bull Board admin UI
  (RFC-004 Phase 3 WS1 / WS3a / WS3b)
- Dual-key JWT verification for WebSocket auth
- Test infrastructure stubs + AVM spec fix (GOO-131)
- Complete MFA grace period feature for required roles + SLO monitoring

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 12:19:17 +07:00
Ho Ngoc Hai
abeb8fd322 feat(auth): complete MFA grace period for required roles + ops monitoring
Finishes the half-implemented MFA enforcement work and ships the SLO
monitoring rules at the same time.

MFA grace period (auth):
- New `mfa-policy.ts` central source of truth: `MFA_REQUIRED_ROLES = [ADMIN]`,
  `MFA_GRACE_PERIOD_DAYS = 14`, `MFA_REAUTH_WINDOW_MINUTES = 15`.
- New columns `User.mfaGraceStartedAt` + `User.mfaLastVerifiedAt`
  (migration `20260429000000_add_mfa_grace_columns`).
- `JwtPayload.mfa: 'none' | 'grace' | 'enrollment_required'` claim now
  carried in every access token so the FE + admin guards can react.
- `LoginUserHandler.resolveMfaGraceClaim()`:
  * If role requires MFA and user has not enrolled, lazy-stamp
    `mfaGraceStartedAt` on first login (returns `mfa: 'grace'`,
    `remainingDays: 14`).
  * After window expires → `mfa: 'enrollment_required'`, `remainingDays: 0`
    (callers must force enrolment on sensitive routes).
  * Otherwise → `mfa: 'none'`.
- `LocalStrategy` now passes `totpEnabled` + `mfaGraceStartedAt` through
  to the command so the handler can branch without an extra query.
- `IUserRepository` + `PrismaUserRepository` get
  `updateMfaGraceStartedAt` / `updateMfaLastVerifiedAt`.
- `UserEntity` carries the two new fields end-to-end (props, getters,
  `createNew` + `createPasswordless` factories). Fixed an orphan-property
  syntax bug in `createPasswordless` that was breaking typecheck.
- `oauth.service.ts` `UserEntity` construction now includes `deletedAt`
  + the two MFA fields (was missing required props).
- Add missing `jsonwebtoken` + `@types/jsonwebtoken` to `apps/api`
  (transitively pulled in via `jwt-rotation.ts` from commit 3705193 but
  never declared, so `tsc --noEmit` was failing).
- Update `login-user.handler.spec.ts` + `local.strategy.spec.ts` to cover
  grace-window + enrolment-required branches. 338/338 auth tests pass.

Ops monitoring:
- New `monitoring/prometheus/slo-rules.yml` with recording + alerting
  rules for the agreed SLOs.
- Wire it into `prometheus.yml` + alertmanager routing.
- Capture the SLO soak-test results in
  `docs/audits/slo-soak-test-log.md`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-29 12:00:23 +07:00
Ho Ngoc Hai
89826858ac feat(api): mount Bull Board behind admin auth (RFC-004 Phase 3 WS3b)
Adds the Bull Board BullMQ dashboard at /api/v1/admin/queues,
guarded by a JWT + ADMIN-role middleware that mirrors the existing
JwtAuthGuard + RolesGuard contract. The dashboard registers all
queues in QUEUE_METRICS_QUEUE_NAMES automatically.

- New QueuesModule with BullBoardModule.forRoot/forFeature wiring
- BullBoardAuthMiddleware (cookie-first JWT extraction, ADMIN-only)
- CSRF exclusion for dashboard routes in AppModule
- 8 unit tests covering auth contract
- Dependencies: @bull-board/api, @bull-board/express, @bull-board/nestjs

Refs: GOO-175

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-26 08:23:00 +07:00
Ho Ngoc Hai
fa3ba88f40 feat(auth): add row/size caps + streaming to export-user-data
- Add per-collection row cap (default 10k, env EXPORT_ROW_CAP) via Prisma
  take on all findMany calls
- Add total size cap (default 100MB, env EXPORT_SIZE_CAP_MB); throws
  PayloadTooLargeException (413) when exceeded
- Convert response to Node.js Readable stream piped via NestJS StreamableFile
  to avoid large in-memory buffers
- Export ExportUserDataResult interface (stream + truncated flag) from handler
- Update controller to set Content-Type/Content-Disposition headers and
  return StreamableFile
- Document EXPORT_ROW_CAP and EXPORT_SIZE_CAP_MB env vars in Swagger
- Extend tests: row-cap assertion (take arg), size-cap 413 path, stream assertions

Fixes GOO-223 (M-1 from GOO-200 audit).

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-24 12:10:54 +07:00
Ho Ngoc Hai
3be66f72df feat(listings): rate limit feature-listing via @nestjs/throttler (TEC-2930)
- Wire ThrottlerModule to a Redis-backed storage (shared across API
  instances) using @nest-lab/throttler-storage-redis.
- Add FeatureListingThrottlerGuard that tracks per-user when JWT is
  present, falling back to the real client IP behind the reverse proxy —
  keeps per-user and per-IP buckets independent.
- Apply @Throttle({ default: { limit: 10, ttl: 60_000 } }) + the guard
  to POST /listings/:id/feature and document 429 in Swagger.
- Integration test (feature-listing-throttle.integration.spec.ts)
  verifies: 10 reqs pass / 11th returns 429 with Retry-After, separate
  IPs keep their own quotas, and the tracker key logic.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-20 08:31:26 +07:00
Ho Ngoc Hai
329a821b4a feat(notifications): production-ready WebSocket gateway (TEC-2766)
- Add RedisIoAdapter (shared/infra) for multi-instance Socket.IO fan-out
  with graceful fallback to the in-memory IoAdapter when Redis is
  unreachable.
- Pin Socket.IO heartbeat (pingInterval/pingTimeout/connectTimeout)
  via env-tunable gateway options for reconnect stability.
- Expose Prometheus metrics on /notifications: goodgo_ws_connected_clients
  (Gauge) and goodgo_ws_messages_total (Counter) with namespace/event/
  direction labels. Wired through MetricsService and tracked across
  connect/disconnect + emits.
- Unit tests: RedisIoAdapter connect/fallback/close, new MetricsService
  WS helpers, and gateway metric increments/decrements on auth paths.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-18 15:06:25 +07:00
Ho Ngoc Hai
deb04989de feat(api): add industrial, transfer, and reports backend modules
Add three new NestJS modules following DDD/CQRS architecture:
- Industrial: KCN (industrial park) management with PostGIS geo queries, Typesense search, and market statistics
- Transfer: Furniture/premises transfer listings with AI-powered price estimation and depreciation modeling
- Reports: Async AI report generation via BullMQ with Claude narrative service, PDF generation, and macro data integration

Includes Prisma schema models, migrations, seed scripts, and app.module wiring with BullMQ Redis config.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-16 09:11:16 +07:00
Ho Ngoc Hai
d4e100a00c feat(api): add price history, Stringee SMS, Zalo OA, WebSocket notifications, and feature-listing command
- Add PriceHistory model + migration, price-changed domain event, and event handler
- Add GetPriceHistory query handler and controller endpoint
- Implement StringeeSmsService and ZaloOaService with unit tests
- Add Zalo ZNS templates for Vietnamese notification messages
- Add WebSocket notification gateway for real-time push
- Add FeatureListingCommand for promoted listings
- Apply remaining consistent-type-imports lint fixes across API modules

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-16 05:15:04 +07:00
Ho Ngoc Hai
bcd591d625 fix: Move @nestjs/config from devDependencies to dependencies
Some checks failed
CI / Lint → Typecheck → Test → Build (22) (push) Failing after 9s
Deploy / Build AI Services Image (push) Failing after 11s
Deploy / Smoke Test Staging (push) Has been skipped
CI / E2E Tests (push) Has been skipped
Deploy / Build API Image (push) Failing after 11s
Deploy / Build Web Image (push) Failing after 11s
E2E Tests / Playwright E2E (push) Failing after 15s
Deploy / Deploy to Staging (push) Has been skipped
Deploy / Deploy to Production (push) Has been skipped
Deploy / Smoke Test Production (push) Has been skipped
Deploy / Rollback Staging (push) Has been skipped
Deploy / Rollback Production (push) Has been skipped
@nestjs/config is used at runtime (ConfigModule in shared.module)
but was incorrectly in devDependencies, causing MODULE_NOT_FOUND
when running with --prod install.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 14:28:09 +07:00
Ho Ngoc Hai
1fbe2f4e73 feat: add MFA/TOTP auth, PII encryption, agents/leads/inquiries modules, and comprehensive tests
- Add TOTP-based MFA with setup, verify, disable, backup codes, and challenge flow
- Add PII field encryption middleware with AES-256-GCM and deterministic search hashes
- Add agents, inquiries, and leads domain modules with entities, events, value objects
- Add web dashboard pages for inquiries and leads with detail dialogs
- Add 30+ component tests (valuation, charts, listings, search, providers, UI)
- Add Prisma migrations for encryption hash columns and MFA TOTP support
- Fix all ESLint errors (unused imports, duplicate imports, lint auto-fixes)
- Update dependencies and lock file
- Clean up obsolete exploration/QA docs, add audit documentation

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-11 23:43:20 +07:00
Ho Ngoc Hai
4372a9ee12 chore: update package dependencies and Playwright config
Update root, API, and web package.json files with latest dependencies.
Refresh pnpm-lock.yaml and update Playwright configuration.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-11 01:40:59 +07:00
Ho Ngoc Hai
862078df37 feat(web): add auth+search i18n translations and filter-bar accessibility
Add missing auth and search translation namespaces to vi.json and en.json
that are required by login/register pages and search filter-bar component.
Update filter-bar with useTranslations('search'), aria-labels, and
role="search" for WCAG 2.1 AA compliance.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 10:22:59 +07:00
Ho Ngoc Hai
35feccb529 feat(analytics): integrate AI/ML services — AVM endpoint, moderation pipeline, market index cron
- Add AiServiceClient HTTP client for Python FastAPI AI service with timeout and fallback
- Add HttpAVMService that calls Python AVM endpoint, falls back to PrismaAVMService on failure
- Add ListingCreatedModerationHandler: auto-flags suspicious listings via AI moderation on create
- Add MarketIndexCronService: daily cron job aggregating market stats per district/city/type
- Wire ScheduleModule and new providers into AnalyticsModule and AppModule
- Add unit tests for AiServiceClient, HttpAVMService, and moderation handler (all passing)

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-09 10:21:05 +07:00
Ho Ngoc Hai
801e29e65c feat(api): add health check endpoints with @nestjs/terminus
Add HealthModule with /health (liveness) and /ready (readiness) probes.
Readiness checks DB (Prisma) and Redis connectivity.
Replaces the basic /health endpoint in AppController.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-09 00:33:44 +07:00
Ho Ngoc Hai
03231271ca fix(security): remove MinIO hardcoded credentials & add presigned URL support
- Remove hardcoded minioadmin/minioadmin_secret fallback from docker-compose.yml,
  require MINIO_ACCESS_KEY/MINIO_SECRET_KEY env vars (fail-fast with :? syntax)
- Align docker-compose.yml env var names with .env.example (MINIO_ACCESS_KEY/SECRET_KEY)
- Update CI e2e workflow to use GitHub vars with non-default fallbacks
- Update .env.test to use non-default test credentials
- Add @aws-sdk/s3-request-presigner and getPresignedUploadUrl() method to
  MinioMediaStorageService for properly signed client-side uploads
- Remove hardcoded credentials from dev-environment docs

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 22:44:50 +07:00
Ho Ngoc Hai
23bb380d34 feat(auth): implement Google and Zalo OAuth backend strategies
Add complete OAuth2 authentication flow for Google and Zalo providers:
- OAuthService: handles account linking (by email/phone), new user
  creation for OAuth-only accounts, and JWT token generation
- GoogleOAuthStrategy: passport-google-oauth20 integration
- ZaloOAuthStrategy: custom OAuth2 implementation using Zalo's API
  (authorization URL generation, code exchange, user info fetch)
- OAuthController: redirect and callback endpoints for both providers
  with httpOnly cookie-based token management
- Unit tests for OAuthService (7 tests), GoogleOAuthStrategy (4 tests),
  and ZaloOAuthStrategy (7 tests)
- OAuth env vars added to .env.example and env-validation warnings

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 14:14:02 +07:00
Ho Ngoc Hai
400a75845c feat(observability): integrate Sentry error tracking for API and Web apps
- API: add @sentry/nestjs with instrument.ts, SentryModule, and SentryGlobalFilter
- Web: add @sentry/nextjs with client/server/edge configs, instrumentation hook
- Update next.config.js with withSentryConfig wrapper
- Replace TODO in error.tsx with Sentry.captureException
- Add SENTRY_DSN, SENTRY_AUTH_TOKEN, SENTRY_ORG, SENTRY_PROJECT to .env.example

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 13:44:57 +07:00
Ho Ngoc Hai
af71270a2e feat: upgrade major dependencies to latest versions
- Prisma 6.19 → 7.7 (driver adapter pattern, prisma.config.ts)
- TypeScript 5.9 → 6.0 (ignoreDeprecations, CSS type declarations)
- Vitest 3.2 → 4.1
- Pino 9.14 → 10.3
- @types/node 22.x → 25.x

All 307 tests pass, typecheck clean, build succeeds.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 13:15:36 +07:00
Ho Ngoc Hai
e5f370ced1 feat(security): add CSRF double-submit cookie protection
Add CSRF middleware with double-submit cookie pattern for all
state-changing requests. Integrate cookie-parser, update CORS
headers, and add client-side CSRF token handling.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 05:03:24 +07:00
Ho Ngoc Hai
8e7672694b feat(api): add OpenAPI/Swagger documentation for all API endpoints
Install @nestjs/swagger, configure Swagger UI at /api/docs with JWT bearer
auth, and add ApiTags/ApiOperation/ApiResponse/ApiProperty decorators to
all 8 controllers (50+ endpoints) and 31 DTOs across auth, listings,
search, payments, subscriptions, admin, notifications, and analytics modules.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 04:08:11 +07:00
Ho Ngoc Hai
fcdb3cac9c fix(media): replace hardcoded MinIO creds and raw fetch with S3 SDK
- Remove `minioadmin` fallback credentials — app now throws on missing
  MINIO_ACCESS_KEY / MINIO_SECRET_KEY env vars
- Replace raw fetch() PUT/DELETE with @aws-sdk/client-s3 (PutObject,
  DeleteObject) using AWS Signature V4 auth
- Add OnModuleInit bucket existence check + auto-creation
- Use forcePathStyle for MinIO S3 compatibility

Closes TEC-1452

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 04:02:45 +07:00
Ho Ngoc Hai
cb00b12d7b feat(mcp): add MCP Server Integration — Property Search, Analytics, Valuation
Implement 3 MCP servers in libs/mcp-servers/ using @modelcontextprotocol/sdk:

- Property Search: NL search via Typesense, property comparison, detail lookup
- Market Analytics: market reports, price trends, district comparison
- Valuation: AVM integration with Python AI service, feature extraction, batch valuation

Includes NestJS integration module with SSE transport for in-process hosting.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 03:22:27 +07:00
Ho Ngoc Hai
d99dfbafbc feat(monitoring): add Prometheus metrics endpoint and Grafana dashboards
Add observability stack with @willsoto/nestjs-prometheus for /metrics endpoint,
Prometheus scraping config, and 4 auto-provisioned Grafana dashboards
(API overview, database, search, business metrics).

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 03:08:54 +07:00
Ho Ngoc Hai
0b29fac35e feat(notifications): add multi-channel notification module with Email, FCM, templates, and event listeners
- Domain: NotificationLog/NotificationPreference entities, repositories, channel value object
- Infrastructure: EmailService (nodemailer/SMTP), FcmService (firebase-admin), TemplateService (Handlebars)
- Application: SendNotification CQRS command, UserRegistered + AgentVerified event listeners
- Presentation: NotificationsController with history, preferences, and templates endpoints
- Prisma: NotificationLog and NotificationPreference models with proper indexes
- Templates: Vietnamese notification templates for user.registered, agent.verified, listing.approved, inquiry.received, password.reset

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 01:42:17 +07:00
Ho Ngoc Hai
391c040100 feat(auth): implement Auth module with register, login, JWT, guards, and CQRS
- Add RefreshToken and OAuthAccount models to Prisma schema
- Implement clean architecture: domain (entities, VOs, events, repo interfaces),
  infrastructure (Prisma repos, Passport strategies, token service),
  application (CQRS command/query handlers), presentation (controller, guards, DTOs)
- Endpoints: POST /auth/register, /auth/login, /auth/refresh, GET /auth/profile,
  GET /auth/profile/agent, PATCH /auth/kyc
- JWT access + refresh token rotation with family-based revocation
- Role-based guards (BUYER, SELLER, AGENT, ADMIN)
- 16 unit tests (value objects, entity) + integration test suite
- All 80 tests passing, clean TypeScript build

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-08 00:24:42 +07:00
Ho Ngoc Hai
83d55de65b feat: add ESLint flat config, Prettier, dependency-cruiser, and Husky
Setup code quality tooling for the monorepo:
- ESLint 9 flat config with TypeScript, import ordering, and NestJS rules
- Prettier with consistent formatting across all files
- dependency-cruiser enforcing module boundary rules (no cross-module internals, no circular deps)
- Husky + lint-staged for pre-commit hooks
- Auto-fixed existing files for type imports and import ordering

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-07 23:57:28 +07:00
Ho Ngoc Hai
e1e5fa6252 feat: scaffold monorepo with Turborepo + NestJS + Next.js
- Turborepo monorepo with pnpm workspaces
- apps/api: NestJS 11.x with CQRS module
- apps/web: Next.js 14 App Router + TailwindCSS
- src/modules/shared: base entities, Result pattern, value objects
- TypeScript 5.7+ strict mode, shared tsconfig base
- Build pipeline: dev, build, lint, test, typecheck

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-04-07 23:52:33 +07:00