diff --git a/.gitignore b/.gitignore index 6cc5a29..451cbf2 100644 --- a/.gitignore +++ b/.gitignore @@ -29,6 +29,7 @@ coverage/ playwright-report/ test-results/ blob-report/ +load-tests/results/*.json # misc *.log diff --git a/load-tests/results/BASELINE-REPORT.md b/load-tests/results/BASELINE-REPORT.md new file mode 100644 index 0000000..78cab2a --- /dev/null +++ b/load-tests/results/BASELINE-REPORT.md @@ -0,0 +1,137 @@ +# K6 Baseline Load Test Report + +**Date:** 2026-04-09 +**Environment:** Local dev (macOS, localhost:3001) +**K6 Version:** v1.7.1 +**API Status:** Running (NestJS dev mode) + +## Executive Summary + +All four K6 test suites executed successfully. Latency metrics are excellent across all endpoints — well within SLA thresholds. However, functional correctness is impacted by dev environment limitations (auth endpoints return 500 due to database/dependency issues, Typesense unavailable for search). + +**Key finding:** The API server handles high concurrency well at the network/framework level. Actual business logic performance needs re-validation once backend dependencies (PostgreSQL + Prisma, Typesense, VNPay) are fully operational. + +## SLA Thresholds (from config) + +| Metric | Target | Status | +|--------|--------|--------| +| p50 latency | < 200ms | **PASS** (all suites < 3ms) | +| p95 latency | < 500ms | **PASS** (all suites < 6ms) | +| p99 latency | < 1000ms | **PASS** (all suites < 19ms) | +| Error rate | < 1% | **FAIL** (auth/listings/payments return 500) | + +## Detailed Results + +### 1. Auth Suite (auth.js) + +| Metric | Value | +|--------|-------| +| Peak VUs | 100 | +| Duration | 2m (ramp 30s → sustain 1m → ramp-down 30s) | +| Total Requests | 4,572 | +| Throughput | 37.6 req/s | +| p50 latency | 2.12ms | +| p95 latency | 5.24ms | +| p99 latency | 9.12ms | +| Error Rate | 100% (HTTP 500 — internal server error) | +| Data Received | 7.6 MB (62 kB/s) | + +**Notes:** +- Register endpoint (`POST /auth/register`) returns 500 — likely Prisma/DB connection issue in dev +- Login tests could not run (no users in pool due to failed registration) +- Latency baseline is solid — the framework + routing layer responds fast + +### 2. Listings Suite (listings.js) + +| Metric | Value | +|--------|-------| +| Peak VUs | 500 | +| Duration | 3m (ramp 30s → sustain 2m → ramp-down 30s) | +| Total Requests | 56,347 | +| Throughput | 311.7 req/s | +| p50 latency | 0.55ms | +| p95 latency | 2.37ms | +| p99 latency | 4.24ms | +| Error Rate | 100% (HTTP 500) | +| Data Received | 68 MB (377 kB/s) | + +**Notes:** +- Highest throughput of all suites (311 req/s at 500 VUs) +- Sub-millisecond p50 indicates very fast error responses +- Detail endpoint not tested (no listings created in setup) +- Need to re-test with seeded DB data for realistic benchmarks + +### 3. Search Suite (search.js) + +| Metric | Value | +|--------|-------| +| Peak VUs | 200 | +| Duration | 3m (ramp 30s → sustain 2m → ramp-down 30s) | +| Total Requests | 18,743 | +| Throughput | 103.7 req/s | +| p50 latency | 1.27ms | +| p95 latency | 2.59ms | +| p99 latency | 4.57ms | +| Error Rate | 0% (503 responses treated as acceptable) | +| Data Received | 155 MB (856 kB/s) | + +**Notes:** +- Text search: p95 = 2.65ms, Geo search: p95 = 2.55ms +- HTTP error rate 0% (503 from Typesense is expected in dev without Typesense running) +- Response validation fails (no `data` field in 503 responses) — expected +- Fixed `URLSearchParams` K6 compatibility issue during this test run + +### 4. Payments Suite (payments.js) + +| Metric | Value | +|--------|-------| +| Peak VUs | 50 | +| Duration | 2m (ramp 30s → sustain 1m → ramp-down 30s) | +| Total Requests | 20 | +| Throughput | 0.17 req/s | +| p50 latency | 1.31ms | +| p95 latency | 5.76ms | +| p99 latency | 18.77ms | +| Error Rate | 100% (setup registration failed) | +| Data Received | 33 kB | + +**Notes:** +- Very low request count (20) because setup failed to register users +- Without auth tokens, payment VUs exit immediately +- p99 at 18.77ms is the highest across suites — suggests payment endpoint has more processing overhead +- Need auth working before meaningful payment benchmarks + +## Bottlenecks Identified + +1. **Auth/Registration (Critical):** `POST /auth/register` returns 500 — blocks all authenticated test flows. Root cause: likely Prisma client not connecting to PostgreSQL or missing migration state. +2. **Typesense Unavailable:** Search returns 503 — expected for local dev without Typesense service. Tests correctly handle this with fallback checks. +3. **K6 Compatibility:** `search.js` used `URLSearchParams` (not available in K6 runtime) — **fixed** by replacing with string interpolation. +4. **Payment Gateway:** VNPay integration returns errors in dev — expected, tests accept 502/503 as valid. + +## Baseline Benchmarks (Framework-Level) + +These numbers represent the NestJS framework + routing layer performance (error responses), not full business logic: + +| Endpoint Category | p50 | p95 | p99 | Max RPS | +|-------------------|-----|-----|-----|---------| +| Auth (register/login) | 2.12ms | 5.24ms | 9.12ms | ~38/s | +| Listings (search/detail) | 0.55ms | 2.37ms | 4.24ms | ~312/s | +| Search (text/geo) | 1.27ms | 2.59ms | 4.57ms | ~104/s | +| Payments (create/list) | 1.31ms | 5.76ms | 18.77ms | ~0.2/s* | + +*Payments throughput is artificially low due to setup failure. + +## Recommendations + +1. **Re-run with full backend:** Set up PostgreSQL + seed data, start Redis, and optionally Typesense before re-running to get business-logic-level benchmarks. +2. **CI integration:** Wire `load-test.yml` GitHub Actions workflow to run against a staging environment with real data. +3. **Performance regression alerts:** Set thresholds in CI — fail the pipeline if p95 > 500ms or error rate > 5%. +4. **Grafana dashboards:** Export K6 results to Grafana Cloud or InfluxDB for historical tracking. +5. **Increase payment VUs:** Once auth works, test payments at 100+ VUs to find gateway bottlenecks. + +## Files Generated + +- `load-tests/results/auth.json` — Raw K6 JSON output +- `load-tests/results/listings.json` — Raw K6 JSON output +- `load-tests/results/search.json` — Raw K6 JSON output +- `load-tests/results/payments.json` — Raw K6 JSON output diff --git a/load-tests/scripts/search.js b/load-tests/scripts/search.js index 02ad95b..270c225 100644 --- a/load-tests/scripts/search.js +++ b/load-tests/scripts/search.js @@ -50,13 +50,9 @@ export default function () { if (iter % 2 === 0) { // Full-text search const query = TEXT_QUERIES[iter % TEXT_QUERIES.length]; - const params = new URLSearchParams({ - q: query, - limit: '20', - offset: '0', - }); + const url = `${BASE_URL}/search?q=${encodeURIComponent(query)}&limit=20&offset=0`; - const res = http.get(`${BASE_URL}/search?${params.toString()}`, { + const res = http.get(url, { tags: { name: 'GET /search (text)' }, }); @@ -71,14 +67,9 @@ export default function () { } else { // Geo search const geo = GEO_SEARCHES[iter % GEO_SEARCHES.length]; - const params = new URLSearchParams({ - lat: String(geo.lat), - lng: String(geo.lng), - radius: String(geo.radius), - limit: '20', - }); + const url = `${BASE_URL}/search/geo?lat=${geo.lat}&lng=${geo.lng}&radius=${geo.radius}&limit=20`; - const res = http.get(`${BASE_URL}/search/geo?${params.toString()}`, { + const res = http.get(url, { tags: { name: 'GET /search/geo' }, });