From 18b5980f296c13d33e8415079564fa794ddab70e Mon Sep 17 00:00:00 2001 From: Ho Ngoc Hai Date: Fri, 10 Apr 2026 23:32:00 +0700 Subject: [PATCH] docs: consolidate and update project documentation - Fix port numbers across all docs (API :3001, Web :3000) - Add 6 missing modules to README, CLAUDE.md, and architecture doc (agents, health, inquiries, leads, reviews, metrics/web-vitals) - Add Swagger UI reference and /api/v1 prefix notes - Create docs/api-endpoints.md with complete REST API reference - Create docs/README.md as documentation index - Update deployment guide with Loki, Promtail, pg-backup services - Update health check table with all current endpoints Co-Authored-By: Paperclip --- CLAUDE.md | 17 ++- README.md | 52 ++++++--- docs/README.md | 26 +++++ docs/api-endpoints.md | 252 ++++++++++++++++++++++++++++++++++++++++ docs/architecture.md | 10 +- docs/deployment.md | 16 ++- docs/dev-environment.md | 4 +- 7 files changed, 345 insertions(+), 32 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/api-endpoints.md diff --git a/CLAUDE.md b/CLAUDE.md index dba324f..b583abb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -45,13 +45,18 @@ pnpm dev # Start all apps (API :3001, Web :3000) apps/api/src/modules/ auth/ — Authentication (JWT, OAuth, refresh tokens, CSRF) listings/ — Property listings CRUD - payments/ — VNPay payment integration + payments/ — VNPay, MoMo, ZaloPay payment integration subscriptions/ — Plans, quotas, usage tracking - admin/ — Moderation, KYC, user management - analytics/ — Market data, heatmaps, price trends - search/ — Geo search, full-text search (Typesense) - notifications/ — Email, push (FCM), in-app notifications - metrics/ — Prometheus metrics + admin/ — Moderation, KYC, user management, audit logs + analytics/ — Market data, heatmaps, price trends, AVM + search/ — Geo search, full-text search (Typesense), saved searches + notifications/ — Email, in-app notifications + agents/ — Agent profiles, quality scores + inquiries/ — Property inquiry management + leads/ — Lead tracking and conversion + reviews/ — Property reviews and ratings + health/ — Liveness and readiness probes + metrics/ — Prometheus metrics, web vitals mcp/ — MCP tool server endpoints shared/ — Domain primitives, guards, pipes, logging ``` diff --git a/README.md b/README.md index d8a72c3..70fb0b1 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ Vietnam's intelligent real estate platform — property search, AI-powered valua | **AI/ML** | FastAPI, XGBoost, Claude API, Underthesea | | **MCP** | Model Context Protocol servers (property search, valuation, analytics) | | **Storage** | MinIO (S3-compatible) | -| **Monitoring** | Prometheus + Grafana | +| **Monitoring** | Prometheus, Grafana, Loki + Promtail | | **Payments** | VNPay, MoMo, ZaloPay | ## Architecture Overview @@ -52,14 +52,15 @@ Vietnam's intelligent real estate platform — property search, AI-powered valua ``` goodgo-platform-ai/ ├── apps/ -│ ├── api/ # NestJS backend (port 3000) -│ └── web/ # Next.js frontend (port 3001) +│ ├── api/ # NestJS backend (port 3001) +│ └── web/ # Next.js frontend (port 3000) ├── libs/ │ ├── ai-services/ # Python FastAPI — AVM + content moderation │ └── mcp-servers/ # MCP server implementations ├── prisma/ # Database schema & migrations -├── e2e/ # Playwright E2E tests -├── monitoring/ # Prometheus & Grafana configs +├── e2e/ # Playwright E2E tests (API + Web) +├── monitoring/ # Prometheus, Grafana, Loki & Promtail configs +├── scripts/ # Backup, restore & utility scripts └── docs/ # Developer documentation ``` @@ -102,7 +103,9 @@ pnpm db:seed pnpm dev ``` -The API will be available at `http://localhost:3000` and the web app at `http://localhost:3001`. +The API will be available at `http://localhost:3001/api/v1` and the web app at `http://localhost:3000`. + +> **Swagger UI**: Open `http://localhost:3001/api/v1/docs` for interactive API documentation. ### Infrastructure Services @@ -113,6 +116,7 @@ The API will be available at `http://localhost:3000` and the web app at `http:// | Typesense | 8108 | `http://localhost:8108/health` | | MinIO | 9000 / 9001 | `http://localhost:9001` (console) | | AI Services (FastAPI) | 8000 | `http://localhost:8000/health` | +| Loki (log aggregation) | 3100 | `http://localhost:3100/ready` | | Prometheus | 9090 | `http://localhost:9090` | | Grafana | 3002 | `http://localhost:3002` | @@ -151,26 +155,38 @@ pnpm test:e2e:report # Open HTML test report ## API Modules +All API routes are prefixed with `/api/v1/`. Each module follows Domain-Driven Design with `presentation/`, `application/`, `domain/`, and `infrastructure/` layers. + | Module | Description | |--------|-------------| -| **auth** | Registration, login, JWT + refresh token rotation, OAuth (Google/Zalo), KYC | +| **auth** | Registration, login, JWT + refresh token rotation, OAuth (Google/Zalo), KYC, user data export/deletion | | **listings** | Property listing CRUD, status workflow, media management | -| **search** | Typesense full-text search with geo-spatial filters | -| **payments** | VNPay, MoMo, ZaloPay integration | -| **subscriptions** | Plan management, usage tracking | -| **notifications** | Email and in-app notifications | -| **admin** | Listing moderation, user management | +| **search** | Typesense full-text search with geo-spatial filters, saved searches | +| **payments** | VNPay, MoMo, ZaloPay integration with callback verification | +| **subscriptions** | Plan management, usage tracking, quota enforcement | +| **notifications** | Email and in-app notification history & preferences | +| **admin** | Listing moderation, user management, audit logs | | **analytics** | Market reports, price indices, AVM integration | +| **agents** | Real estate agent profiles and verification | +| **inquiries** | Property inquiry management | +| **leads** | Lead tracking and conversion | +| **reviews** | Property reviews and ratings | +| **health** | Liveness and readiness health checks | | **mcp** | MCP server bridge (property search, valuation, analytics) | -| **metrics** | Prometheus metrics endpoint | - -Each module follows Domain-Driven Design with `presentation/`, `application/`, `domain/`, and `infrastructure/` layers. +| **metrics** | Prometheus metrics and web vitals collection | +| **shared** | Cross-cutting concerns: guards, pipes, filters, Prisma/Redis services | ## Documentation -- [Development Environment](docs/dev-environment.md) — Docker setup and local services -- [Architecture](docs/architecture.md) — System design, data flow, module structure -- [Deployment](docs/deployment.md) — Production deployment guide +| Document | Description | +|----------|-------------| +| [Development Environment](docs/dev-environment.md) | Docker setup and local services | +| [Architecture](docs/architecture.md) | System design, data flow, module structure | +| [API Endpoints](docs/api-endpoints.md) | REST API endpoint reference | +| [API Error Codes](docs/api-error-codes.md) | Error response format and all error codes | +| [Deployment](docs/deployment.md) | Production deployment guide | +| [Backup & Restore](docs/backup-restore.md) | Backup procedures and disaster recovery | +| [Contributing](CONTRIBUTING.md) | Error handling conventions and coding patterns | ## License diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..cf867d0 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,26 @@ +# GoodGo Platform Documentation + +## Getting Started + +| Document | Description | +|----------|-------------| +| [Development Environment](dev-environment.md) | Docker setup, local services, troubleshooting | +| [Architecture](architecture.md) | System design, data flow, module structure | + +## API Reference + +| Document | Description | +|----------|-------------| +| [API Endpoints](api-endpoints.md) | Complete REST API endpoint reference | +| [API Error Codes](api-error-codes.md) | Error response format and all error codes | + +## Operations + +| Document | Description | +|----------|-------------| +| [Deployment](deployment.md) | Production deployment guide and checklists | +| [Backup & Restore](backup-restore.md) | Backup procedures and disaster recovery runbook | + +## Audits + +See [audits/README.md](audits/README.md) for code quality, accessibility, and test coverage audit reports. diff --git a/docs/api-endpoints.md b/docs/api-endpoints.md new file mode 100644 index 0000000..25f81cf --- /dev/null +++ b/docs/api-endpoints.md @@ -0,0 +1,252 @@ +# API Endpoints Reference + +All routes are prefixed with `/api/v1/` except health checks. Authentication uses Bearer JWT tokens. + +> **Interactive docs**: Swagger UI is available at `http://localhost:3001/api/v1/docs` when running the API locally. + +## Auth (`/auth`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/auth/register` | Public | Register a new user | +| POST | `/auth/login` | Public | Login with phone and password | +| POST | `/auth/refresh` | Cookie | Refresh access token using httpOnly refresh cookie | +| POST | `/auth/logout` | JWT | Logout and clear auth cookies | +| POST | `/auth/exchange-token` | Public | Exchange OAuth token pair for httpOnly cookies | +| GET | `/auth/profile` | JWT | Get current user profile | +| GET | `/auth/profile/agent` | JWT | Get agent profile for current user | +| PATCH | `/auth/kyc` | Admin | Verify user KYC status | + +### OAuth + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/auth/google` | Public | Initiate Google OAuth2 login | +| GET | `/auth/google/callback` | Public | Google OAuth2 callback | +| GET | `/auth/zalo` | Public | Initiate Zalo OAuth2 login | +| GET | `/auth/zalo/callback` | Public | Zalo OAuth2 callback | + +### User Data (GDPR) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| DELETE | `/users/me` | JWT | Request account deletion (30-day grace period) | +| POST | `/users/me/cancel-deletion` | JWT | Cancel pending account deletion | +| GET | `/users/me/export` | JWT | Export user data (GDPR Article 20) | +| DELETE | `/users/:id/force` | Admin | Force-delete user immediately | + +--- + +## Listings (`/listings`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/listings` | JWT + Quota | Create a new property listing | +| GET | `/listings` | Public | Search and filter property listings | +| GET | `/listings/pending` | Admin | Get listings pending moderation | +| GET | `/listings/:id` | Public | Get listing details by ID | +| PATCH | `/listings/:id/status` | JWT | Update listing status | +| POST | `/listings/:id/media` | JWT | Upload media (multipart file upload) | +| PATCH | `/listings/:id/moderate` | Admin | Moderate a listing | + +--- + +## Search (`/search`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/search` | Public | Full-text and faceted property search | +| GET | `/search/geo` | Public | Geographic radius search | +| POST | `/search/reindex` | Admin | Reindex all properties in Typesense | + +### Saved Searches (`/saved-searches`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/saved-searches` | JWT | Save search filters | +| GET | `/saved-searches` | JWT | List saved searches | +| GET | `/saved-searches/:id` | JWT | Get saved search details | +| PATCH | `/saved-searches/:id` | JWT | Update saved search | +| DELETE | `/saved-searches/:id` | JWT | Delete saved search | + +--- + +## Payments (`/payments`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/payments` | JWT | Create a new payment | +| GET | `/payments` | JWT | List transactions for authenticated user | +| GET | `/payments/:id` | JWT | Get payment status by ID | +| POST | `/payments/:id/refund` | Admin | Refund a payment | +| POST | `/payments/callback/:provider` | Public | Payment provider webhook (VNPay, MoMo, ZaloPay) | + +--- + +## Subscriptions (`/subscriptions`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/subscriptions/plans` | Public | List all subscription plans | +| GET | `/subscriptions/plans/:tier` | Public | Get subscription plan by tier | +| POST | `/subscriptions` | JWT | Create a new subscription | +| PUT | `/subscriptions/upgrade` | JWT | Upgrade existing subscription | +| DELETE | `/subscriptions` | JWT | Cancel active subscription | +| POST | `/subscriptions/usage` | JWT | Record metered usage | +| GET | `/subscriptions/quota/:metric` | JWT | Check remaining quota for a metric | +| GET | `/subscriptions/billing` | JWT | Get billing history | + +--- + +## Admin (`/admin`) + +### User Management + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/admin/users` | Admin | List users with optional filters | +| GET | `/admin/users/:id` | Admin | Get user details by ID | +| PATCH | `/admin/users/status` | Admin | Update user active status | +| POST | `/admin/users/ban` | Admin | Ban or unban a user | +| POST | `/admin/subscriptions/adjust` | Admin | Adjust user subscription plan | +| GET | `/admin/dashboard` | Admin | Get admin dashboard statistics | +| GET | `/admin/revenue` | Admin | Get revenue statistics by date range | +| GET | `/admin/audit-logs` | Admin | Get admin audit logs | + +### Moderation + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/admin/moderation` | Admin | Get listing moderation queue | +| POST | `/admin/moderation/approve` | Admin | Approve a listing | +| POST | `/admin/moderation/reject` | Admin | Reject a listing | +| POST | `/admin/moderation/bulk` | Admin | Bulk approve or reject listings | +| GET | `/admin/kyc` | Admin | Get KYC verification queue | +| POST | `/admin/kyc/approve` | Admin | Approve KYC verification | +| POST | `/admin/kyc/reject` | Admin | Reject KYC verification | + +--- + +## Notifications (`/notifications`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/notifications/history` | JWT | Get notification history | +| GET | `/notifications/preferences` | JWT | Get notification preferences | +| PUT | `/notifications/preferences` | JWT | Update notification preferences | +| GET | `/notifications/unread` | JWT | Get unread notifications | +| PATCH | `/notifications/:id/read` | JWT | Mark notification as read | +| PATCH | `/notifications/read-all` | JWT | Mark all notifications as read | +| GET | `/notifications/templates` | JWT | Get available notification templates | + +--- + +## Analytics (`/analytics`) + +All analytics endpoints require JWT and are quota-guarded. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/analytics/market-report` | JWT + Quota | Get market report for a city | +| GET | `/analytics/price-trend` | JWT + Quota | Get price trend for a district | +| GET | `/analytics/heatmap` | JWT + Quota | Get price heatmap for a city | +| GET | `/analytics/district-stats` | JWT + Quota | Get statistics by district | +| GET | `/analytics/valuation` | JWT + Quota | Get automated property valuation (AVM) | + +--- + +## Agents (`/agents`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/agents/me/dashboard` | Agent | Get agent dashboard stats | +| POST | `/agents/:agentId/recalculate-score` | Admin | Recalculate agent quality score | + +--- + +## Inquiries (`/inquiries`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/inquiries` | JWT | Create inquiry for a listing | +| GET | `/inquiries/listing/:listingId` | JWT | List inquiries by listing | +| GET | `/inquiries/agent/me` | Agent | List inquiries for current agent | +| PATCH | `/inquiries/:id/read` | Agent | Mark inquiry as read | + +--- + +## Leads (`/leads`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/leads` | Agent | Create lead | +| GET | `/leads` | Agent | List leads for agent | +| GET | `/leads/stats` | Agent | Get lead statistics | +| PATCH | `/leads/:id/status` | Agent | Update lead status | +| DELETE | `/leads/:id` | Agent | Delete lead | + +--- + +## Reviews (`/reviews`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/reviews` | JWT | Create a review | +| GET | `/reviews` | Public | List reviews by target | +| GET | `/reviews/stats` | Public | Get aggregate rating stats for a target | +| GET | `/reviews/me` | JWT | Get reviews by authenticated user | +| DELETE | `/reviews/:id` | JWT | Delete own review | + +--- + +## Health (`/health`) + +Health endpoints are **not** prefixed with `/api/v1/`. + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/health` | Public | Liveness probe | +| GET | `/health/ready` | Public | Readiness probe (DB + Redis) | +| GET | `/health/db` | Public | Database connectivity check | +| GET | `/health/redis` | Public | Redis connectivity check | + +--- + +## MCP (`/mcp`) + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| GET | `/mcp/servers` | JWT | List available MCP servers | +| GET | `/mcp/:serverName/sse` | JWT | Open SSE connection to MCP server | +| POST | `/mcp/:serverName/messages` | JWT | Send message to MCP server session | + +--- + +## Metrics + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/web-vitals` | Public | Ingest Core Web Vitals metrics | + +--- + +## Authentication Summary + +| Auth Type | Description | +|-----------|-------------| +| **Public** | No authentication required | +| **JWT** | Requires `Authorization: Bearer ` header | +| **Admin** | JWT + `ADMIN` role required | +| **Agent** | JWT + `AGENT` role required | +| **Quota** | JWT + subscription quota enforcement | +| **Cookie** | Uses httpOnly refresh token cookie | + +## Rate Limiting + +The following endpoints have stricter rate limits: + +- Auth endpoints (register, login, refresh): `10 req/60s` +- OAuth callbacks: `10 req/60s` +- Payment callbacks: `60 req/60s` +- MCP endpoints: `30 req/60s` +- Default: `60 req/60s` diff --git a/docs/architecture.md b/docs/architecture.md index 3e8d160..6bf1074 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -27,6 +27,10 @@ GoodGo Platform AI is a monorepo containing a NestJS backend, Next.js frontend, │ │ Search │ │ Admin │ │Subscriptions│ │ │ ├────────┤ ├────────┤ ├─────────────┤ │ │ │Analytics│ │ MCP │ │Notifications│ │ + │ ├────────┤ ├────────┤ ├─────────────┤ │ + │ │ Agents │ │Inquires│ │ Leads │ │ + │ ├────────┤ ├────────┤ ├─────────────┤ │ + │ │Reviews │ │ Health │ │ Metrics │ │ │ └────────┘ └────────┘ └─────────────┘ │ └───┬─────┬──────┬─────────┬──────────────┘ │ │ │ │ @@ -224,10 +228,12 @@ Python FastAPI microservice (`libs/ai-services/`) provides: ## Monitoring -- **Prometheus** scrapes `/metrics` endpoint every 15 seconds +- **Prometheus** scrapes `/api/v1/metrics` endpoint every 15 seconds - **Grafana** dashboards auto-provisioned from `monitoring/grafana/dashboards/` +- **Loki + Promtail** aggregate container logs; viewable in Grafana - **Pino** structured JSON logging with configurable log levels -- Metrics include HTTP request duration, error rates, and custom business metrics +- Metrics include HTTP request duration, error rates, web vitals, and custom business metrics +- Log retention: 15 days (configured in `monitoring/loki/loki-config.yml`) ## Event System diff --git a/docs/deployment.md b/docs/deployment.md index f439d83..75d841a 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -6,8 +6,8 @@ GoodGo Platform AI consists of four deployable services: | Service | Technology | Default Port | |---------|-----------|-------------| -| **API** | NestJS (Node.js) | 3000 | -| **Web** | Next.js | 3001 | +| **API** | NestJS (Node.js) | 3001 | +| **Web** | Next.js | 3000 | | **AI Services** | FastAPI (Python) | 8000 | | **Infrastructure** | Docker Compose | Various | @@ -63,8 +63,11 @@ This starts: - **Typesense 27** (port 8108) - **MinIO** (API: 9000, Console: 9001) - **AI Services** (port 8000) +- **pg-backup** — automated daily PostgreSQL backups at 02:00 UTC with verification at 04:00 UTC +- **Loki** (port 3100) — log aggregation +- **Promtail** — log collection agent (ships container logs to Loki) - **Prometheus** (port 9090) -- **Grafana** (port 3002) +- **Grafana** (port 3002) — dashboards for metrics and logs Verify all services are healthy: @@ -101,7 +104,7 @@ Output: `apps/api/dist/` Run in production: ```bash -NODE_ENV=production node apps/api/dist/main.js +NODE_ENV=production PORT=3001 node apps/api/dist/main.js ``` ### Web (Next.js) @@ -167,9 +170,12 @@ docker run -p 8000:8000 --env-file ../../.env goodgo-ai-services | Service | Endpoint | Expected Response | |---------|----------|-------------------| -| API | `GET /metrics` | Prometheus metrics | +| API | `GET /health` | `{"status": "ok"}` | +| API (Swagger) | `GET /api/v1/docs` | Swagger UI page | +| API (Metrics) | `GET /api/v1/metrics` | Prometheus metrics | | AI Services | `GET /health` | `{"status": "ok"}` | | Typesense | `GET /health` | `{"ok": true}` | +| Loki | `GET /ready` | 200 OK | | Redis | `redis-cli ping` | `PONG` | | PostgreSQL | `pg_isready -h host -p 5432` | Exit code 0 | diff --git a/docs/dev-environment.md b/docs/dev-environment.md index d53f1d0..aa65bfa 100644 --- a/docs/dev-environment.md +++ b/docs/dev-environment.md @@ -34,7 +34,9 @@ pnpm db:seed pnpm dev ``` -API runs at `http://localhost:3000`, Web at `http://localhost:3001`. +API runs at `http://localhost:3001/api/v1`, Web at `http://localhost:3000`. + +> **Swagger UI**: `http://localhost:3001/api/v1/docs` — interactive API documentation. ## Infrastructure Services