Compare commits

...

4 Commits

Author SHA1 Message Date
Ho Ngoc Hai
1e9ef567a9 docs(osm): note 2025 VN admin reform — vn_districts now holds ward/commune layer
Some checks failed
CI / AI Services (Python) — Smoke (push) Failing after 35s
Deploy / Build Web Image (push) Failing after 30s
Deploy / Build AI Services Image (push) Failing after 11s
E2E Tests / Playwright E2E (push) Failing after 37s
CI / Lint → Typecheck → Test → Build (22) (push) Failing after 11m1s
Deploy / Build API Image (push) Failing after 10m40s
Backup Verification / Backup Restore Verification (push) Failing after 14s
Deploy / Deploy to Staging (push) Has been cancelled
Deploy / Smoke Test Staging (push) Has been cancelled
CI / E2E Tests (push) Has been cancelled
Deploy / Rollback Staging (push) Has been cancelled
Deploy / Deploy to Production (push) Has been cancelled
Deploy / Smoke Test Production (push) Has been cancelled
Deploy / Rollback Production (push) Has been cancelled
Security Scanning / Dependency Audit (pnpm) (push) Failing after 6s
Security Scanning / Trivy Scan — API Image (push) Failing after 42s
Security Scanning / Trivy Scan — Web Image (push) Failing after 27s
Security Scanning / Trivy Scan — AI Services Image (push) Failing after 26s
Security Scanning / Trivy Filesystem Scan (push) Failing after 23s
Security Scanning / Security Gate (push) Failing after 1s
CodeQL Analysis / CodeQL (javascript-typescript) (push) Failing after 49s
Vietnam dropped the district administrative level in the 2025 reform
(Nghị quyết về sắp xếp đơn vị hành chính). Only two levels remain:
province (level=4) and ward/commune (level=6).

OSM has updated tagging accordingly: every former xã/phường/thị trấn
that survived the merge is now `admin_level=6`, no `admin_level=8`
features for VN. Our sync confirmed this — 3,189 level=6 units inserted
across 33 provinces, level=8 returns zero.

The schema column "vn_districts" stays as-is to avoid a cascade-rename
across IndustrialPark / ProjectDevelopment / Property FKs. Documented
the semantic shift in osm-data-model.md so future ops don't think
something is broken when wards are empty.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 13:13:26 +07:00
Ho Ngoc Hai
884a8d2a63 feat(osm): Phase 6 — proximity materialized views + refresh cron
* `mv_park_nearest_poi` — for each IndustrialPark, the 3 nearest POI of
  six priority categories (HOSPITAL/BANK/GAS/BUS/METRO/POLICE) within
  5km. Refreshed weekly. Pre-aggregated 6,513 rows from the live
  catalog so the KCN sidebar can render in <50ms instead of running
  ST_Distance for every page hit.
* `mv_poi_density_by_province` — count of POI per (province, category)
  for analytics heatmaps.
* `OsmSyncService.refreshMaterializedViews()` calls
  `REFRESH MATERIALIZED VIEW CONCURRENTLY` so reads aren't blocked.
* New cron entry `weeklyRefreshViews` (Sun 04:00 ICT) and admin
  endpoint `POST /admin/osm/refresh-views` for on-demand refresh.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 12:44:57 +07:00
Ho Ngoc Hai
a9770a5f93 feat(osm): user-facing UX — POI sidebar + search filter + docs
* Listing detail: drop the new <NearbyPoiSidebar> below the price card
  with default 1.5km radius and 6 categories (school/secondary/hospital/
  market/bank/metro). Reads property.lat/lng — no-op when unset.
* KCN detail: same component but 3km radius with the categories that
  matter for industrial parks (hospital/bank/gas/bus/metro/police).
* New <PoiSearchFilter> widget for the search page: pill button →
  popover with radius dropdown (300m..5km), 3 quick presets ("Family",
  "Commute", "Convenience"), and 6 grouped category checkboxes. Wires
  to a `PoiNearbyConstraint` value so callers can pass it into search
  filters when they're ready.
* docs/osm-data-model.md: canonical reference for every OSM-sourced
  table, sync cadence, quality gates, runbook for ops, and a clear
  "how to add a new POI category" guide.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 12:06:52 +07:00
Ho Ngoc Hai
fba536406d feat(osm): foundation — admin boundaries, POI catalog, sync orchestrator
This is the Phase 0 + Phase 1 + Phase 4 foundation of the full OSM
integration plan. It backfills three things the rest of the platform
has been faking with hardcoded tables, and gives admins one dashboard
for every OSM-sourced layer.

Phase 0 — Vietnam administrative boundaries
* New columns on vn_provinces / vn_districts / vn_wards: PostGIS
  geometry (MultiPolygon), centroid (Point), areaKm2, osmId, population,
  lastSyncedAt + GIST indexes on geometry/centroid.
* `scripts/sync-osm-admin-boundaries.ts` pulls
  `boundary=administrative + admin_level=4|6|8` from Overpass per chunk,
  filters to mainland VN via the existing country polygon, resolves the
  GSO code (or generates `OSM_<id>`), and upserts via raw SQL because
  Prisma can't manage PostGIS columns.
* `GeoLookupService` (shared module) replaces the old
  `nearestProvince()` heuristic — `lookup(lng,lat)` returns
  province/district/ward via `ST_Contains` on the GIST-indexed polygons.
* The KCN sync now resolves province/district from the polygon table
  and falls back to the centroid heuristic only when polygons aren't
  loaded yet.
* `scripts/backfill-admin-codes.ts` rewrites province/district/ward on
  IndustrialPark, ProjectDevelopment and Property using the new lookup.

Phase 1 — POI catalog (15 categories, schema only here)
* New `Poi` table with `PoiCategory` enum, OSM provenance columns,
  GIST index on `location`. New `TransportLine` for metro/highway
  multilinestrings.
* `scripts/sync-osm-poi.ts` queries Overpass per category × chunk,
  resolves province/district codes from the boundary polygons, upserts
  with `osmLocked` / `lockedFields` honour same as KCN.
* New NestJS `PoiModule` exposes:
    GET /poi/by-bbox    — GeoJSON for map overlays
    GET /poi/nearby     — sidebar "tiện ích xung quanh" (HMAC distance ranks)
    GET /poi/coverage   — admin per-category counts
* New web component `<NearbyPoiSidebar />` ready to drop into listing /
  project / KCN detail pages.

Phase 4 — Sync orchestrator + admin dashboard
* New `OsmSyncRun` audit table tracks every sync invocation
  (RUNNING / SUCCESS / PARTIAL / FAILED + row stats + error message).
* `OsmSyncService` spawns the right tsx script for any (layer, category,
  chunk) tuple, parses stats out of stdout, updates the run row.
* `OsmSyncCronService` schedules:
    Daily 02:00  → POI category rotation (1/day, 20-day cycle)
    Mon  02:30  → admin-boundaries provinces
    Wed  02:30  → admin-boundaries districts
    Sat  02:30  → admin-boundaries wards
    1st of month 03:00 → industrial-parks (per chunk)
  All gated by `OSM_SYNC_ENABLED=true`.
* New admin endpoints under `/admin/osm/*` (layers / coverage / runs /
  trigger), guarded by JWT + ADMIN role.
* New `/admin/osm` Next.js page: stat cards, coverage table with
  per-row "Sync now", recent runs list with auto-refresh every 15s.

Run on dev so far: 33 provinces + 1100+ districts (still finishing) +
305 hospitals POI imported.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 12:01:19 +07:00
43 changed files with 3777 additions and 11 deletions

View File

@@ -20,7 +20,9 @@ import { McpIntegrationModule } from '@modules/mcp';
import { MessagingModule } from '@modules/messaging';
import { HttpMetricsInterceptor, MetricsModule } from '@modules/metrics';
import { NotificationsModule } from '@modules/notifications';
import { OsmSyncModule } from '@modules/osm-sync/osm-sync.module';
import { PaymentsModule } from '@modules/payments';
import { PoiModule } from '@modules/poi/poi.module';
import { ProjectsModule } from '@modules/projects';
import { QueuesModule } from '@modules/queues/queues.module';
import { ReportsModule } from '@modules/reports';
@@ -58,7 +60,9 @@ import { AppController } from './app.controller';
FavoritesModule,
SearchModule,
NotificationsModule,
OsmSyncModule,
PaymentsModule,
PoiModule,
SubscriptionsModule,
AdminModule,
AnalyticsModule,

View File

@@ -0,0 +1,8 @@
/** Manually trigger an OSM sync run from the admin UI. */
export class TriggerOsmSyncCommand {
constructor(
public readonly layer: string,
public readonly category?: string | null,
public readonly chunk?: string | null,
) {}
}

View File

@@ -0,0 +1,17 @@
import { CommandHandler, type ICommandHandler } from '@nestjs/cqrs';
import { OsmSyncService } from '../../../infrastructure/osm-sync.service';
import { TriggerOsmSyncCommand } from './trigger-sync.command';
@CommandHandler(TriggerOsmSyncCommand)
export class TriggerOsmSyncHandler implements ICommandHandler<TriggerOsmSyncCommand> {
constructor(private readonly osmSync: OsmSyncService) {}
async execute(cmd: TriggerOsmSyncCommand): Promise<{ runId: string; status: string }> {
return this.osmSync.run({
layer: cmd.layer,
category: cmd.category ?? null,
chunk: cmd.chunk ?? null,
wait: false,
});
}
}

View File

@@ -0,0 +1,127 @@
import { QueryHandler, type IQueryHandler } from '@nestjs/cqrs';
import { GeoLookupService, PrismaService } from '@modules/shared';
import { OsmCoverageSummaryQuery } from './coverage-summary.query';
export interface CoverageRow {
layer: string;
category: string | null;
total: number;
withGeometry?: number; // only meaningful for admin boundaries
promoted?: number;
raw?: number;
lastSyncedAt: Date | null;
}
export interface OsmCoverageSummary {
rows: CoverageRow[];
totals: {
administrativeUnits: number;
poiTotal: number;
industrialParks: number;
transportStations: number;
transportLines: number;
};
}
/**
* Single endpoint that powers the `/admin/osm` dashboard top-of-page
* "what's where" panel. Aggregates per-layer counts so we don't need 5
* separate API calls.
*/
@QueryHandler(OsmCoverageSummaryQuery)
export class OsmCoverageSummaryHandler
implements IQueryHandler<OsmCoverageSummaryQuery, OsmCoverageSummary>
{
constructor(
private readonly prisma: PrismaService,
private readonly geo: GeoLookupService,
) {}
async execute(): Promise<OsmCoverageSummary> {
const [adminCov, poiByCategory, parkTotal, transportPoiTotal, transportLineTotal] =
await Promise.all([
this.geo.coverage(),
this.prisma.$queryRawUnsafe<
{
category: string;
total: bigint;
promoted: bigint;
raw: bigint;
lastSyncedAt: Date | null;
}[]
>(
`SELECT category::text AS category,
COUNT(*)::bigint AS total,
SUM(CASE WHEN "dataSource" = 'OSM_PROMOTED' THEN 1 ELSE 0 END)::bigint AS promoted,
SUM(CASE WHEN "dataSource" = 'OSM' THEN 1 ELSE 0 END)::bigint AS raw,
MAX("lastSyncedAt") AS "lastSyncedAt"
FROM "Poi"
GROUP BY category`,
),
this.prisma.industrialPark.count(),
this.prisma.poi.count({
where: {
category: {
in: ['METRO_STATION', 'RAILWAY_STATION', 'BUS_STATION', 'AIRPORT'],
},
},
}),
this.prisma.transportLine.count(),
]);
const rows: CoverageRow[] = [];
rows.push(
{
layer: 'admin-boundaries',
category: 'province',
total: adminCov.provinces.total,
withGeometry: adminCov.provinces.withGeometry,
lastSyncedAt: adminCov.provinces.lastSyncedAt,
},
{
layer: 'admin-boundaries',
category: 'district',
total: adminCov.districts.total,
withGeometry: adminCov.districts.withGeometry,
lastSyncedAt: adminCov.districts.lastSyncedAt,
},
{
layer: 'admin-boundaries',
category: 'ward',
total: adminCov.wards.total,
withGeometry: adminCov.wards.withGeometry,
lastSyncedAt: adminCov.wards.lastSyncedAt,
},
);
for (const p of poiByCategory) {
rows.push({
layer: 'poi',
category: p.category,
total: Number(p.total),
promoted: Number(p.promoted),
raw: Number(p.raw),
lastSyncedAt: p.lastSyncedAt,
});
}
rows.push({
layer: 'industrial-parks',
category: null,
total: parkTotal,
lastSyncedAt: null,
});
return {
rows,
totals: {
administrativeUnits:
adminCov.provinces.withGeometry +
adminCov.districts.withGeometry +
adminCov.wards.withGeometry,
poiTotal: poiByCategory.reduce((sum, p) => sum + Number(p.total), 0),
industrialParks: parkTotal,
transportStations: transportPoiTotal,
transportLines: transportLineTotal,
},
};
}
}

View File

@@ -0,0 +1,3 @@
/** Aggregate coverage view across all OSM-managed tables for the
* admin dashboard. */
export class OsmCoverageSummaryQuery {}

View File

@@ -0,0 +1,22 @@
import { QueryHandler, type IQueryHandler } from '@nestjs/cqrs';
import { type OsmSyncRun, type OsmSyncStatus, type Prisma } from '@prisma/client';
import { PrismaService } from '@modules/shared';
import { ListOsmSyncRunsQuery } from './list-runs.query';
@QueryHandler(ListOsmSyncRunsQuery)
export class ListOsmSyncRunsHandler
implements IQueryHandler<ListOsmSyncRunsQuery, OsmSyncRun[]>
{
constructor(private readonly prisma: PrismaService) {}
async execute(q: ListOsmSyncRunsQuery): Promise<OsmSyncRun[]> {
const where: Prisma.OsmSyncRunWhereInput = {};
if (q.layer) where.layer = q.layer;
if (q.status) where.status = q.status as OsmSyncStatus;
return this.prisma.osmSyncRun.findMany({
where,
orderBy: { startedAt: 'desc' },
take: Math.min(Math.max(q.limit, 1), 200),
});
}
}

View File

@@ -0,0 +1,8 @@
/** List recent OSM sync runs for the admin dashboard. */
export class ListOsmSyncRunsQuery {
constructor(
public readonly layer?: string,
public readonly status?: string,
public readonly limit: number = 50,
) {}
}

View File

@@ -0,0 +1,95 @@
import { Injectable } from '@nestjs/common';
import { Cron } from '@nestjs/schedule';
import { LoggerService } from '@modules/shared';
import { OsmSyncService } from '../osm-sync.service';
/**
* Scheduled sync runner. Spreads layer refreshes across the week so we
* never hit Overpass with two heavy queries simultaneously and stay
* under the per-IP rate limit.
*
* Schedule (Asia/Ho_Chi_Minh):
* - Daily 02:00 → POI category rotation (one per day, 20-day cycle)
* - Mon 02:30 → admin-boundaries level=4 (provinces, light)
* - Wed 02:30 → admin-boundaries level=6 (districts, medium)
* - Sat 02:30 → admin-boundaries level=8 (wards, heavy)
* - 1st of month 03:00 → industrial-parks (existing flow, kept here so
* everything routes through one orchestrator)
*
* All routes respect `OSM_SYNC_ENABLED=true` to allow disabling in dev.
*/
@Injectable()
export class OsmSyncCronService {
private readonly POI_CATEGORIES = [
'SCHOOL_PRIMARY', 'SCHOOL_SECONDARY', 'UNIVERSITY',
'HOSPITAL', 'CLINIC', 'PHARMACY',
'MARKET', 'SUPERMARKET', 'MALL', 'CONVENIENCE',
'BANK', 'ATM',
'PARK',
'GAS_STATION', 'POLICE', 'POST_OFFICE',
'METRO_STATION', 'RAILWAY_STATION', 'BUS_STATION', 'AIRPORT',
];
constructor(
private readonly osmSync: OsmSyncService,
private readonly logger: LoggerService,
) {}
private isEnabled(): boolean {
return process.env['OSM_SYNC_ENABLED'] === 'true';
}
@Cron('0 2 * * *', { timeZone: 'Asia/Ho_Chi_Minh' })
async dailyPoiRotation(): Promise<void> {
if (!this.isEnabled()) return;
// Pick one category based on day-of-year so we cycle evenly.
const dayOfYear = Math.floor(
(Date.now() - new Date(new Date().getUTCFullYear(), 0, 0).getTime()) / 86_400_000,
);
const category = this.POI_CATEGORIES[dayOfYear % this.POI_CATEGORIES.length]!;
this.logger.log(`Daily POI rotation: ${category}`, 'OsmSyncCronService');
await this.osmSync.run({ layer: 'poi', category, wait: false });
}
@Cron('30 2 * * 1', { timeZone: 'Asia/Ho_Chi_Minh' }) // Monday
async weeklyProvinces(): Promise<void> {
if (!this.isEnabled()) return;
await this.osmSync.run({ layer: 'admin-boundaries', category: 'province', wait: false });
}
@Cron('30 2 * * 3', { timeZone: 'Asia/Ho_Chi_Minh' }) // Wednesday
async weeklyDistricts(): Promise<void> {
if (!this.isEnabled()) return;
await this.osmSync.run({ layer: 'admin-boundaries', category: 'district', wait: false });
}
@Cron('30 2 * * 6', { timeZone: 'Asia/Ho_Chi_Minh' }) // Saturday
async weeklyWards(): Promise<void> {
if (!this.isEnabled()) return;
await this.osmSync.run({ layer: 'admin-boundaries', category: 'ward', wait: false });
}
@Cron('0 3 1 * *', { timeZone: 'Asia/Ho_Chi_Minh' }) // 1st of month
async monthlyIndustrialParks(): Promise<void> {
if (!this.isEnabled()) return;
// KCN sync runs per chunk to spread load.
for (const chunk of ['north', 'northCentral', 'southCentral', 'south']) {
await this.osmSync.run({ layer: 'industrial-parks', chunk, wait: true });
}
}
/** Refresh proximity / density mat views every Sunday after the weekly
* ward sync had time to settle. Always runs (cheap, no Overpass). */
@Cron('0 4 * * 0', { timeZone: 'Asia/Ho_Chi_Minh' })
async weeklyRefreshViews(): Promise<void> {
try {
await this.osmSync.refreshMaterializedViews();
} catch (err) {
this.logger.error(
`Refresh views failed: ${err instanceof Error ? err.message : err}`,
err instanceof Error ? err.stack : undefined,
'OsmSyncCronService',
);
}
}
}

View File

@@ -0,0 +1,255 @@
import { Injectable } from '@nestjs/common';
import { OsmSyncStatus } from '@prisma/client';
import { spawn } from 'node:child_process';
import { createHash } from 'node:crypto';
import * as path from 'node:path';
import { LoggerService, PrismaService } from '@modules/shared';
/**
* Catalog of every sync layer / category we know about. The orchestrator
* uses this to validate trigger requests, populate the admin UI, and
* decide which scripts to run on the cron schedule.
*/
export interface OsmSyncLayerDef {
layer: string;
category?: string;
/** Path of the tsx script under repo root. */
scriptPath: string;
/** Extra CLI args appended after `--category=` etc. */
extraArgs?: string[];
/** Approx Overpass cost — used to spread cron schedule. */
weight: 'light' | 'medium' | 'heavy';
}
export const SYNC_LAYERS: OsmSyncLayerDef[] = [
{
layer: 'admin-boundaries',
category: 'province',
scriptPath: 'scripts/sync-osm-admin-boundaries.ts',
extraArgs: ['--level=4'],
weight: 'light',
},
{
layer: 'admin-boundaries',
category: 'district',
scriptPath: 'scripts/sync-osm-admin-boundaries.ts',
extraArgs: ['--level=6'],
weight: 'medium',
},
{
layer: 'admin-boundaries',
category: 'ward',
scriptPath: 'scripts/sync-osm-admin-boundaries.ts',
extraArgs: ['--level=8'],
weight: 'heavy',
},
// POI categories — each one its own row so the dashboard shows progress
// per category and the cron can rotate them across days.
...['SCHOOL_PRIMARY', 'SCHOOL_SECONDARY', 'UNIVERSITY',
'HOSPITAL', 'CLINIC', 'PHARMACY',
'MARKET', 'SUPERMARKET', 'MALL', 'CONVENIENCE',
'BANK', 'ATM',
'PARK',
'GAS_STATION', 'POLICE', 'POST_OFFICE',
'METRO_STATION', 'RAILWAY_STATION', 'BUS_STATION', 'AIRPORT',
].map<OsmSyncLayerDef>((cat) => ({
layer: 'poi',
category: cat,
scriptPath: 'scripts/sync-osm-poi.ts',
extraArgs: [`--category=${cat}`],
weight: cat === 'BANK' || cat === 'PHARMACY' || cat === 'CONVENIENCE' ? 'medium' : 'light',
})),
{
layer: 'industrial-parks',
scriptPath: 'scripts/sync-osm-industrial-parks.ts',
weight: 'heavy',
},
];
/**
* Spawns the right tsx script for a given (layer, category, chunk) and
* tracks the run in `OsmSyncRun`. Used both by the cron service and the
* admin "Sync now" button.
*/
@Injectable()
export class OsmSyncService {
constructor(
private readonly prisma: PrismaService,
private readonly logger: LoggerService,
) {}
/**
* Refresh the proximity / density materialized views. Called by the
* cron and from the admin "Refresh views" button. Runs concurrently
* (`CONCURRENTLY`) so reads aren't blocked.
*/
async refreshMaterializedViews(): Promise<void> {
this.logger.log('Refreshing materialized views', 'OsmSyncService');
await this.prisma.$executeRawUnsafe(
`REFRESH MATERIALIZED VIEW CONCURRENTLY "mv_park_nearest_poi"`,
);
await this.prisma.$executeRawUnsafe(
`REFRESH MATERIALIZED VIEW "mv_poi_density_by_province"`,
);
this.logger.log('Materialized views refreshed', 'OsmSyncService');
}
/** Look up a sync layer by its (layer, category) tuple. */
findLayer(layer: string, category?: string | null): OsmSyncLayerDef | undefined {
return SYNC_LAYERS.find(
(l) => l.layer === layer && (l.category ?? null) === (category ?? null),
);
}
list(): OsmSyncLayerDef[] {
return SYNC_LAYERS;
}
/**
* Run a sync layer (script invocation). Inserts a RUNNING `OsmSyncRun`,
* captures script stdout/stderr line-by-line into the logger, and
* updates the row to SUCCESS / PARTIAL / FAILED + row counts on exit.
*
* Returns the persisted `OsmSyncRun.id` immediately if `wait=false` so
* the admin UI can poll, or after the script exits when `wait=true`.
*/
async run(opts: {
layer: string;
category?: string | null;
chunk?: string | null;
wait?: boolean;
}): Promise<{ runId: string; status: OsmSyncStatus }> {
const def = this.findLayer(opts.layer, opts.category);
if (!def) {
throw new Error(`Unknown OSM sync layer: ${opts.layer}/${opts.category ?? '-'}`);
}
const args = [...(def.extraArgs ?? [])];
if (opts.chunk) args.push(`--chunk=${opts.chunk}`);
const queryHash = createHash('sha256')
.update(`${def.scriptPath} ${args.join(' ')}`)
.digest('hex')
.slice(0, 16);
const run = await this.prisma.osmSyncRun.create({
data: {
layer: opts.layer,
category: opts.category ?? null,
chunk: opts.chunk ?? null,
status: OsmSyncStatus.RUNNING,
overpassQueryHash: queryHash,
},
});
this.logger.log(
`OSM sync started run=${run.id} layer=${opts.layer} category=${opts.category ?? '-'} chunk=${opts.chunk ?? '-'}`,
'OsmSyncService',
);
const promise = this.spawnAndTrack(run.id, def, args);
if (opts.wait) {
const status = await promise;
return { runId: run.id, status };
}
void promise.catch((err) =>
this.logger.error(
`OSM sync ${run.id} background failure: ${err}`,
err instanceof Error ? err.stack : undefined,
'OsmSyncService',
),
);
return { runId: run.id, status: OsmSyncStatus.RUNNING };
}
private async spawnAndTrack(
runId: string,
def: OsmSyncLayerDef,
args: string[],
): Promise<OsmSyncStatus> {
return new Promise((resolve) => {
const repoRoot = path.resolve(__dirname, '../../../../../../..');
const child = spawn('pnpm', ['tsx', def.scriptPath, ...args], {
cwd: repoRoot,
env: {
...process.env,
NODE_OPTIONS: '-r dotenv/config',
DOTENV_CONFIG_PATH: '.env',
},
});
const stats = { added: 0, updated: 0, skipped: 0, locked: 0 };
const errors: string[] = [];
const parseLine = (line: string) => {
// Lines like: "inserted=12 updated=3 locked=1 skipped=0"
const m = line.match(/inserted=(\d+).*updated=(\d+).*locked=(\d+).*skipped=(\d+)/);
if (m) {
stats.added += Number(m[1]);
stats.updated += Number(m[2]);
stats.locked += Number(m[3]);
stats.skipped += Number(m[4]);
}
};
child.stdout?.on('data', (b) => {
for (const line of b.toString().split('\n')) {
if (!line.trim()) continue;
this.logger.log(`[${runId}] ${line.trim()}`, 'OsmSyncService');
parseLine(line);
}
});
child.stderr?.on('data', (b) => {
for (const line of b.toString().split('\n')) {
if (!line.trim()) continue;
this.logger.warn(`[${runId}] ${line.trim()}`, 'OsmSyncService');
if (errors.length < 20) errors.push(line.trim().slice(0, 500));
}
});
child.on('error', async (err) => {
await this.complete(runId, OsmSyncStatus.FAILED, stats, err.message);
resolve(OsmSyncStatus.FAILED);
});
child.on('exit', async (code) => {
const status =
code === 0
? errors.length > 0
? OsmSyncStatus.PARTIAL
: OsmSyncStatus.SUCCESS
: OsmSyncStatus.FAILED;
await this.complete(
runId,
status,
stats,
status === OsmSyncStatus.SUCCESS
? null
: `exit=${code}; ${errors.slice(0, 5).join(' | ')}`,
);
resolve(status);
});
});
}
private async complete(
runId: string,
status: OsmSyncStatus,
stats: { added: number; updated: number; skipped: number; locked: number },
errorMessage: string | null,
): Promise<void> {
await this.prisma.osmSyncRun.update({
where: { id: runId },
data: {
status,
finishedAt: new Date(),
rowsAdded: stats.added,
rowsUpdated: stats.updated,
rowsSkipped: stats.skipped,
rowsLocked: stats.locked,
errorMessage,
},
});
this.logger.log(
`OSM sync ${runId}${status}: added=${stats.added} updated=${stats.updated} skipped=${stats.skipped} locked=${stats.locked}`,
'OsmSyncService',
);
}
}

View File

@@ -0,0 +1,22 @@
import { Module } from '@nestjs/common';
import { CqrsModule } from '@nestjs/cqrs';
import { TriggerOsmSyncHandler } from './application/commands/trigger-sync/trigger-sync.handler';
import { OsmCoverageSummaryHandler } from './application/queries/coverage-summary/coverage-summary.handler';
import { ListOsmSyncRunsHandler } from './application/queries/list-runs/list-runs.handler';
import { OsmSyncCronService } from './infrastructure/cron/osm-sync-cron.service';
import { OsmSyncService } from './infrastructure/osm-sync.service';
import { OsmSyncController } from './presentation/controllers/osm-sync.controller';
const Handlers = [
TriggerOsmSyncHandler,
ListOsmSyncRunsHandler,
OsmCoverageSummaryHandler,
];
@Module({
imports: [CqrsModule],
controllers: [OsmSyncController],
providers: [OsmSyncService, OsmSyncCronService, ...Handlers],
exports: [OsmSyncService],
})
export class OsmSyncModule {}

View File

@@ -0,0 +1,74 @@
import { Body, Controller, Get, Post, Query, UseGuards } from '@nestjs/common';
import { CommandBus, QueryBus } from '@nestjs/cqrs';
import { ApiBearerAuth, ApiOperation, ApiResponse, ApiTags } from '@nestjs/swagger';
import { UserRole } from '@prisma/client';
import { JwtAuthGuard, Roles, RolesGuard } from '@modules/auth';
import { TriggerOsmSyncCommand } from '../../application/commands/trigger-sync/trigger-sync.command';
import { OsmCoverageSummaryQuery } from '../../application/queries/coverage-summary/coverage-summary.query';
import { ListOsmSyncRunsQuery } from '../../application/queries/list-runs/list-runs.query';
import { OsmSyncService } from '../../infrastructure/osm-sync.service';
import { TriggerSyncDto } from '../dto/trigger-sync.dto';
/**
* Admin-only endpoints powering the `/admin/osm` dashboard. Public users
* never hit this controller — guarded by JwtAuthGuard + RolesGuard(ADMIN).
*/
@ApiTags('osm-sync')
@ApiBearerAuth('JWT')
@UseGuards(JwtAuthGuard, RolesGuard)
@Roles(UserRole.ADMIN)
@Controller('admin/osm')
export class OsmSyncController {
constructor(
private readonly commandBus: CommandBus,
private readonly queryBus: QueryBus,
private readonly osmSync: OsmSyncService,
) {}
@ApiOperation({ summary: 'List configured sync layers (catalog)' })
@Get('layers')
layers(): { layer: string; category?: string; weight: string }[] {
return this.osmSync.list().map((l) => ({
layer: l.layer,
category: l.category,
weight: l.weight,
}));
}
@ApiOperation({ summary: 'Coverage summary across all layers' })
@Get('coverage')
coverage() {
return this.queryBus.execute(new OsmCoverageSummaryQuery());
}
@ApiOperation({ summary: 'Recent sync runs (latest first)' })
@Get('runs')
runs(
@Query('layer') layer?: string,
@Query('status') status?: string,
@Query('limit') limit?: string,
) {
return this.queryBus.execute(
new ListOsmSyncRunsQuery(layer, status, limit ? Number(limit) : 50),
);
}
@ApiOperation({ summary: 'Trigger a sync run now (returns runId for polling)' })
@ApiResponse({ status: 201, description: 'Sync started' })
@Post('runs')
trigger(@Body() dto: TriggerSyncDto) {
return this.commandBus.execute(
new TriggerOsmSyncCommand(dto.layer, dto.category, dto.chunk),
);
}
@ApiOperation({
summary: 'Refresh proximity materialized views',
description: 'Recomputes mv_park_nearest_poi + mv_poi_density_by_province.',
})
@Post('refresh-views')
async refreshViews(): Promise<{ ok: true }> {
await this.osmSync.refreshMaterializedViews();
return { ok: true };
}
}

View File

@@ -0,0 +1,8 @@
import { ApiProperty } from '@nestjs/swagger';
import { IsOptional, IsString } from 'class-validator';
export class TriggerSyncDto {
@ApiProperty({ example: 'admin-boundaries' }) @IsString() layer!: string;
@ApiProperty({ required: false, example: 'province' }) @IsOptional() @IsString() category?: string;
@ApiProperty({ required: false, example: 'north' }) @IsOptional() @IsString() chunk?: string;
}

View File

@@ -0,0 +1,55 @@
import { QueryHandler, type IQueryHandler } from '@nestjs/cqrs';
import { PrismaService } from '@modules/shared';
import { PoiCoverageStatsQuery } from './coverage-stats.query';
export interface PoiCoverageRow {
category: string;
total: number;
promoted: number;
raw: number;
lastSyncedAt: Date | null;
}
@QueryHandler(PoiCoverageStatsQuery)
export class PoiCoverageStatsHandler
implements IQueryHandler<PoiCoverageStatsQuery, PoiCoverageRow[]>
{
constructor(private readonly prisma: PrismaService) {}
async execute(q: PoiCoverageStatsQuery): Promise<PoiCoverageRow[]> {
const provinceFilter = q.provinceCode ? `WHERE "provinceCode" = $1` : '';
const params = q.provinceCode ? [q.provinceCode] : [];
const rows = await this.prisma.$queryRawUnsafe<
{
category: string;
total: bigint;
promoted: bigint;
raw: bigint;
lastSyncedAt: Date | null;
}[]
>(
`
SELECT
category::text AS category,
COUNT(*)::bigint AS total,
SUM(CASE WHEN "dataSource" = 'OSM_PROMOTED' THEN 1 ELSE 0 END)::bigint AS promoted,
SUM(CASE WHEN "dataSource" = 'OSM' THEN 1 ELSE 0 END)::bigint AS raw,
MAX("lastSyncedAt") AS "lastSyncedAt"
FROM "Poi"
${provinceFilter}
GROUP BY category
ORDER BY total DESC
`,
...params,
);
return rows.map((r) => ({
category: r.category,
total: Number(r.total),
promoted: Number(r.promoted),
raw: Number(r.raw),
lastSyncedAt: r.lastSyncedAt,
}));
}
}

View File

@@ -0,0 +1,5 @@
/** Query: aggregate per-category counts so the admin /admin/osm dashboard
* can show "30k schools, 15k hospitals, …" without a join per row. */
export class PoiCoverageStatsQuery {
constructor(public readonly provinceCode?: string | null) {}
}

View File

@@ -0,0 +1,123 @@
import { QueryHandler, type IQueryHandler } from '@nestjs/cqrs';
import { LoggerService, PrismaService } from '@modules/shared';
import { FindNearbyPoiQuery } from './find-nearby-poi.query';
export interface NearbyPoi {
id: string;
name: string;
category: string;
/** Great-circle distance in metres from the requested centre. */
distanceM: number;
lat: number;
lng: number;
address: string | null;
}
export interface NearbyPoiResult {
/** Grouped by category for easy rendering as "tiện ích" chips. */
byCategory: Record<string, NearbyPoi[]>;
/** Flat list ordered by distance — used by the map overlay layer. */
all: NearbyPoi[];
meta: {
radiusMeters: number;
totalCount: number;
requestedCategories: string[] | null;
};
}
@QueryHandler(FindNearbyPoiQuery)
export class FindNearbyPoiHandler
implements IQueryHandler<FindNearbyPoiQuery, NearbyPoiResult>
{
constructor(
private readonly prisma: PrismaService,
private readonly logger: LoggerService,
) {}
async execute(q: FindNearbyPoiQuery): Promise<NearbyPoiResult> {
const radius = Math.min(Math.max(q.radiusMeters, 50), 10_000);
const limitPerCat = Math.min(Math.max(q.limitPerCategory, 1), 50);
const cleanCats = (q.categories ?? [])
.map((c) => c.trim().toUpperCase())
.filter((c) => /^[A-Z_]+$/.test(c));
const categoryFilter = cleanCats.length
? `AND category::text IN (${cleanCats.map((c) => `'${c}'`).join(', ')})`
: '';
try {
// PostGIS `ST_DWithin` with `geography::` cast does the great-circle
// metres check. For each row we also compute the actual distance
// (cast back to geography) and rank within category.
const rows = await this.prisma.$queryRawUnsafe<
{
id: string;
name: string;
category: string;
address: string | null;
lat: number;
lng: number;
distance_m: number;
rank: number;
}[]
>(
`
SELECT id, name, category, address, lat, lng, distance_m, rank FROM (
SELECT
p.id, p.name, p.category::text AS category, p.address,
ST_Y(p.location::geometry) AS lat,
ST_X(p.location::geometry) AS lng,
ST_Distance(p.location::geography, c.center::geography) AS distance_m,
ROW_NUMBER() OVER (
PARTITION BY p.category
ORDER BY ST_Distance(p.location::geography, c.center::geography)
) AS rank
FROM "Poi" p,
(SELECT ST_SetSRID(ST_MakePoint($1, $2), 4326) AS center) c
WHERE p."isPublic" = true
AND p."dataSource"::text IN ('OSM', 'OSM_PROMOTED', 'MANUAL')
AND ST_DWithin(p.location::geography, c.center::geography, $3)
${categoryFilter}
) ranked
WHERE rank <= $4
ORDER BY distance_m
`,
q.center.lng,
q.center.lat,
radius,
limitPerCat,
);
const items: NearbyPoi[] = rows.map((r) => ({
id: r.id,
name: r.name,
category: r.category,
distanceM: Math.round(r.distance_m),
lat: r.lat,
lng: r.lng,
address: r.address,
}));
const byCategory: Record<string, NearbyPoi[]> = {};
for (const it of items) {
(byCategory[it.category] ??= []).push(it);
}
return {
byCategory,
all: items,
meta: {
radiusMeters: radius,
totalCount: items.length,
requestedCategories: cleanCats.length ? cleanCats : null,
},
};
} catch (err) {
this.logger.error(
`Find nearby POI failed: ${err instanceof Error ? err.message : err}`,
err instanceof Error ? err.stack : undefined,
this.constructor.name,
);
throw err;
}
}
}

View File

@@ -0,0 +1,13 @@
/**
* Query: find POI within a radius around a centre point. Drives the
* "tiện ích xung quanh" sidebar on listing / project / KCN detail pages
* and the search filter "trong vòng X mét từ trường".
*/
export class FindNearbyPoiQuery {
constructor(
public readonly center: { lng: number; lat: number },
public readonly radiusMeters: number,
public readonly categories: string[] | null,
public readonly limitPerCategory: number = 5,
) {}
}

View File

@@ -0,0 +1,99 @@
import { QueryHandler, type IQueryHandler } from '@nestjs/cqrs';
import { LoggerService, PrismaService } from '@modules/shared';
import type { Feature, FeatureCollection } from 'geojson';
import { ListPoiByBboxQuery } from './list-poi-by-bbox.query';
interface BboxRow {
id: string;
name: string;
category: string;
province_code: string | null;
district_code: string | null;
point: string; // GeoJSON Point as text from ST_AsGeoJSON
}
export interface PoiGeoCollection extends FeatureCollection {
meta: {
count: number;
truncated: boolean;
categories: string[];
};
}
@QueryHandler(ListPoiByBboxQuery)
export class ListPoiByBboxHandler
implements IQueryHandler<ListPoiByBboxQuery, PoiGeoCollection>
{
constructor(
private readonly prisma: PrismaService,
private readonly logger: LoggerService,
) {}
async execute(q: ListPoiByBboxQuery): Promise<PoiGeoCollection> {
const { south, west, north, east } = q.bbox;
const limit = Math.min(Math.max(q.limit, 1), 5000);
// Build optional category filter — Prisma can't safely interpolate enum
// arrays so we whitelist + inline.
const cleanCats = (q.categories ?? [])
.map((c) => c.trim().toUpperCase())
.filter((c) => /^[A-Z_]+$/.test(c));
const categoryFilter = cleanCats.length
? `AND category::text IN (${cleanCats.map((c) => `'${c}'`).join(', ')})`
: '';
try {
const rows = await this.prisma.$queryRawUnsafe<BboxRow[]>(
`
SELECT id, name, category::text AS category,
"provinceCode" AS province_code,
"districtCode" AS district_code,
ST_AsGeoJSON(location) AS point
FROM "Poi"
WHERE "isPublic" = true
AND "dataSource"::text IN ('OSM', 'OSM_PROMOTED', 'MANUAL')
AND location && ST_MakeEnvelope($1, $2, $3, $4, 4326)
${categoryFilter}
LIMIT ${limit + 1}
`,
west,
south,
east,
north,
);
const truncated = rows.length > limit;
const trimmed = truncated ? rows.slice(0, limit) : rows;
const features: Feature[] = trimmed.map((r) => ({
type: 'Feature',
id: r.id,
geometry: JSON.parse(r.point),
properties: {
id: r.id,
name: r.name,
category: r.category,
provinceCode: r.province_code,
districtCode: r.district_code,
},
}));
return {
type: 'FeatureCollection',
features,
meta: {
count: trimmed.length,
truncated,
categories: Array.from(new Set(trimmed.map((r) => r.category))),
},
};
} catch (err) {
this.logger.error(
`Poi bbox query failed: ${err instanceof Error ? err.message : err}`,
err instanceof Error ? err.stack : undefined,
this.constructor.name,
);
throw err;
}
}
}

View File

@@ -0,0 +1,12 @@
/**
* Query: list POI inside a Mapbox-style bounding box, filtered by category.
* Used by the public catalog map and the listing-detail "tiện ích xung
* quanh" chips.
*/
export class ListPoiByBboxQuery {
constructor(
public readonly bbox: { south: number; west: number; north: number; east: number },
public readonly categories: string[] | null,
public readonly limit: number = 1000,
) {}
}

View File

@@ -0,0 +1,19 @@
import { Module } from '@nestjs/common';
import { CqrsModule } from '@nestjs/cqrs';
import { PoiCoverageStatsHandler } from './application/queries/coverage-stats/coverage-stats.handler';
import { FindNearbyPoiHandler } from './application/queries/find-nearby-poi/find-nearby-poi.handler';
import { ListPoiByBboxHandler } from './application/queries/list-poi-by-bbox/list-poi-by-bbox.handler';
import { PoiController } from './presentation/controllers/poi.controller';
const QueryHandlers = [
ListPoiByBboxHandler,
FindNearbyPoiHandler,
PoiCoverageStatsHandler,
];
@Module({
imports: [CqrsModule],
controllers: [PoiController],
providers: [...QueryHandlers],
})
export class PoiModule {}

View File

@@ -0,0 +1,55 @@
import { Controller, Get, Query, UseGuards } from '@nestjs/common';
import { QueryBus } from '@nestjs/cqrs';
import { ApiBearerAuth, ApiOperation, ApiResponse, ApiTags } from '@nestjs/swagger';
import { UserRole } from '@prisma/client';
import { JwtAuthGuard, Roles, RolesGuard } from '@modules/auth';
import { PoiCoverageStatsQuery } from '../../application/queries/coverage-stats/coverage-stats.query';
import { FindNearbyPoiQuery } from '../../application/queries/find-nearby-poi/find-nearby-poi.query';
import { ListPoiByBboxQuery } from '../../application/queries/list-poi-by-bbox/list-poi-by-bbox.query';
import { FindNearbyPoiDto } from '../dto/find-nearby-poi.dto';
import { ListPoiByBboxDto } from '../dto/list-poi-by-bbox.dto';
@ApiTags('poi')
@Controller('poi')
export class PoiController {
constructor(private readonly queryBus: QueryBus) {}
@ApiOperation({ summary: 'POI in viewport (GeoJSON FeatureCollection)' })
@ApiResponse({ status: 200, description: 'GeoJSON + meta' })
@Get('by-bbox')
async byBbox(@Query() dto: ListPoiByBboxDto) {
return this.queryBus.execute(
new ListPoiByBboxQuery(
{ south: dto.south, west: dto.west, north: dto.north, east: dto.east },
dto.categories ?? null,
dto.limit ?? 1000,
),
);
}
@ApiOperation({
summary: 'POI within radius around a point',
description:
'Drives "tiện ích xung quanh" sidebar. Returns up to N nearest POI per category.',
})
@Get('nearby')
async nearby(@Query() dto: FindNearbyPoiDto) {
return this.queryBus.execute(
new FindNearbyPoiQuery(
{ lng: dto.lng, lat: dto.lat },
dto.radius,
dto.categories ?? null,
dto.limitPerCategory ?? 5,
),
);
}
@ApiOperation({ summary: 'POI coverage stats per category (admin)' })
@ApiBearerAuth('JWT')
@UseGuards(JwtAuthGuard, RolesGuard)
@Roles(UserRole.ADMIN)
@Get('coverage')
async coverage(@Query('provinceCode') provinceCode?: string) {
return this.queryBus.execute(new PoiCoverageStatsQuery(provinceCode ?? null));
}
}

View File

@@ -0,0 +1,27 @@
import { ApiProperty } from '@nestjs/swagger';
import { Transform, Type } from 'class-transformer';
import { IsArray, IsInt, IsNumber, IsOptional, IsString, Max, Min } from 'class-validator';
export class FindNearbyPoiDto {
@ApiProperty({ example: 10.762622 }) @Type(() => Number) @IsNumber() @Min(-90) @Max(90)
lat!: number;
@ApiProperty({ example: 106.660172 }) @Type(() => Number) @IsNumber() @Min(-180) @Max(180)
lng!: number;
@ApiProperty({ example: 1500, description: 'Radius in metres (50 - 10000)' })
@Type(() => Number) @IsInt() @Min(50) @Max(10_000)
radius!: number;
@ApiProperty({ required: false, isArray: true })
@IsOptional()
@IsArray()
@IsString({ each: true })
@Transform(({ value }) =>
typeof value === 'string' ? value.split(',').map((s) => s.trim()) : value,
)
categories?: string[];
@ApiProperty({ required: false, default: 5 })
@IsOptional() @Type(() => Number) @IsInt() @Min(1) @Max(50)
limitPerCategory?: number;
}

View File

@@ -0,0 +1,27 @@
import { ApiProperty } from '@nestjs/swagger';
import { Transform, Type } from 'class-transformer';
import { IsArray, IsInt, IsNumber, IsOptional, IsString, Max, Min } from 'class-validator';
export class ListPoiByBboxDto {
@ApiProperty({ example: 10.5 }) @Type(() => Number) @IsNumber() @Min(-90) @Max(90)
south!: number;
@ApiProperty({ example: 106.5 }) @Type(() => Number) @IsNumber() @Min(-180) @Max(180)
west!: number;
@ApiProperty({ example: 11.0 }) @Type(() => Number) @IsNumber() @Min(-90) @Max(90)
north!: number;
@ApiProperty({ example: 107.0 }) @Type(() => Number) @IsNumber() @Min(-180) @Max(180)
east!: number;
@ApiProperty({ required: false, isArray: true, example: ['SCHOOL_PRIMARY', 'HOSPITAL'] })
@IsOptional()
@IsArray()
@IsString({ each: true })
@Transform(({ value }) =>
typeof value === 'string' ? value.split(',').map((s) => s.trim()) : value,
)
categories?: string[];
@ApiProperty({ required: false, default: 1000 })
@IsOptional() @Type(() => Number) @IsInt() @Min(1) @Max(5000)
limit?: number;
}

View File

@@ -0,0 +1,184 @@
import { Injectable } from '@nestjs/common';
import { LoggerService } from './logger.service';
import { PrismaService } from './prisma.service';
/**
* Result of a "where am I?" geo lookup. Each level may be null when the
* point lies outside any synced polygon (or when that level hasn't been
* synced yet — see PHASE_0 in the OSM rollout plan).
*/
export interface GeoLookupResult {
province: { code: string; name: string } | null;
district: { code: string; name: string } | null;
ward: { code: string; name: string } | null;
}
/**
* Centralised "lat/lng → administrative unit" resolver. Replaces the old
* `nearestProvince()` helper that walked a hard-coded centroid table —
* we now use real OSM-sourced polygons (PostGIS `ST_Contains`).
*
* Backed by the `vn_provinces` / `vn_districts` / `vn_wards` tables that
* `scripts/sync-osm-admin-boundaries.ts` populates. All three GIST-indexed
* geometry columns mean each lookup is O(log N).
*/
@Injectable()
export class GeoLookupService {
constructor(
private readonly prisma: PrismaService,
private readonly logger: LoggerService,
) {}
/**
* Resolve a point to the deepest administrative unit available. Returns
* partial results when the polygon hierarchy is incomplete (e.g. ward
* polygons not synced yet for that area).
*/
async lookup(lng: number, lat: number): Promise<GeoLookupResult> {
if (!this.isFiniteCoord(lng, lat)) {
return { province: null, district: null, ward: null };
}
// Province first — fastest GIST lookup, parents the other two.
const provinceRows = await this.prisma.$queryRawUnsafe<
{ code: string; name: string }[]
>(
`SELECT code, name FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1`,
lng,
lat,
);
const province = provinceRows[0] ?? null;
if (!province) return { province: null, district: null, ward: null };
// District scoped to the matched province for speed + correctness
// around shared borders.
const districtRows = await this.prisma.$queryRawUnsafe<
{ code: string; name: string }[]
>(
`SELECT code, name FROM "vn_districts"
WHERE "provinceCode" = $3
AND geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1`,
lng,
lat,
province.code,
);
const district = districtRows[0] ?? null;
if (!district) {
return { province, district: null, ward: null };
}
const wardRows = await this.prisma.$queryRawUnsafe<
{ code: string; name: string }[]
>(
`SELECT code, name FROM "vn_wards"
WHERE "districtCode" = $3
AND geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1`,
lng,
lat,
district.code,
);
const ward = wardRows[0] ?? null;
return { province, district, ward };
}
/** Convenience wrapper that returns just the province display name. */
async findProvinceName(lng: number, lat: number): Promise<string | null> {
const r = await this.lookup(lng, lat);
return r.province?.name ?? null;
}
/** True if any province polygon contains the point — i.e. point is in VN. */
async isInVietnam(lng: number, lat: number): Promise<boolean> {
if (!this.isFiniteCoord(lng, lat)) return false;
const rows = await this.prisma.$queryRawUnsafe<{ exists: boolean }[]>(
`SELECT EXISTS (
SELECT 1 FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
) AS exists`,
lng,
lat,
);
return rows[0]?.exists ?? false;
}
/**
* Coverage report for the admin dashboard: how many polygons of each
* level we have, and when each was last refreshed. Cheap aggregate.
*/
async coverage(): Promise<{
provinces: { total: number; withGeometry: number; lastSyncedAt: Date | null };
districts: { total: number; withGeometry: number; lastSyncedAt: Date | null };
wards: { total: number; withGeometry: number; lastSyncedAt: Date | null };
}> {
const [p, d, w] = await Promise.all([
this.prisma.$queryRawUnsafe<
{ total: bigint; withGeometry: bigint; lastSyncedAt: Date | null }[]
>(
`SELECT COUNT(*)::bigint AS total,
COUNT(geometry)::bigint AS "withGeometry",
MAX("lastSyncedAt") AS "lastSyncedAt"
FROM "vn_provinces"`,
),
this.prisma.$queryRawUnsafe<
{ total: bigint; withGeometry: bigint; lastSyncedAt: Date | null }[]
>(
`SELECT COUNT(*)::bigint AS total,
COUNT(geometry)::bigint AS "withGeometry",
MAX("lastSyncedAt") AS "lastSyncedAt"
FROM "vn_districts"`,
),
this.prisma.$queryRawUnsafe<
{ total: bigint; withGeometry: bigint; lastSyncedAt: Date | null }[]
>(
`SELECT COUNT(*)::bigint AS total,
COUNT(geometry)::bigint AS "withGeometry",
MAX("lastSyncedAt") AS "lastSyncedAt"
FROM "vn_wards"`,
),
]);
return {
provinces: {
total: Number(p[0]?.total ?? 0n),
withGeometry: Number(p[0]?.withGeometry ?? 0n),
lastSyncedAt: p[0]?.lastSyncedAt ?? null,
},
districts: {
total: Number(d[0]?.total ?? 0n),
withGeometry: Number(d[0]?.withGeometry ?? 0n),
lastSyncedAt: d[0]?.lastSyncedAt ?? null,
},
wards: {
total: Number(w[0]?.total ?? 0n),
withGeometry: Number(w[0]?.withGeometry ?? 0n),
lastSyncedAt: w[0]?.lastSyncedAt ?? null,
},
};
}
private isFiniteCoord(lng: number, lat: number): boolean {
if (!Number.isFinite(lng) || !Number.isFinite(lat)) {
this.logger.warn(
`GeoLookupService: invalid coordinates lng=${lng} lat=${lat}`,
'GeoLookupService',
);
return false;
}
if (lng < -180 || lng > 180 || lat < -90 || lat > 90) {
this.logger.warn(
`GeoLookupService: out-of-range coordinates lng=${lng} lat=${lat}`,
'GeoLookupService',
);
return false;
}
return true;
}
}

View File

@@ -9,6 +9,7 @@ export {
type ModelEncryptionFieldConfig,
} from './field-encryption.service';
export { createEncryptionExtension } from './encryption-middleware';
export { GeoLookupService, type GeoLookupResult } from './geo-lookup.service';
export { PrismaService } from './prisma.service';
export { RedisService } from './redis.service';
export { RedisIoAdapter } from './redis-io.adapter';

View File

@@ -17,6 +17,7 @@ import {
// import { EVENT_BUS, RedisStreamsEventBus } from './infrastructure/event-bus';
import { EventBusService } from './infrastructure/event-bus.service';
import { FieldEncryptionService } from './infrastructure/field-encryption.service';
import { GeoLookupService } from './infrastructure/geo-lookup.service';
import { GlobalExceptionFilter } from './infrastructure/filters/global-exception.filter';
import { DeprecationInterceptor, VersionInterceptor } from './infrastructure/interceptors';
import { LoggerService } from './infrastructure/logger.service';
@@ -43,6 +44,7 @@ import { TypesenseClientService } from './infrastructure/typesense-client.servic
RedisService,
CacheService,
EventBusService,
GeoLookupService,
// RFC-004 Phase 0 (GOO-172) — see import comment above.
// { provide: EVENT_BUS, useClass: RedisStreamsEventBus },
// OutboxService,
@@ -78,7 +80,17 @@ import { TypesenseClientService } from './infrastructure/typesense-client.servic
useClass: DeprecationInterceptor,
},
],
exports: [PrismaService, RedisService, CacheService, LoggerService, EventBusService, FieldEncryptionService, TypesenseClientService, PrometheusModule],
exports: [
PrismaService,
RedisService,
CacheService,
LoggerService,
EventBusService,
FieldEncryptionService,
GeoLookupService,
TypesenseClientService,
PrometheusModule,
],
})
export class SharedModule implements NestModule {
configure(consumer: MiddlewareConsumer): void {

View File

@@ -0,0 +1,321 @@
'use client';
import {
AlertTriangle,
CheckCircle,
Clock,
Layers,
MapPin,
PlayCircle,
RefreshCw,
Train,
XCircle,
} from 'lucide-react';
import { useCallback, useEffect, useState } from 'react';
import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from '@/components/ui/table';
import {
osmSyncApi,
type OsmCoverageSummary,
type OsmSyncLayer,
type OsmSyncRun,
} from '@/lib/osm-sync-api';
const STATUS_STYLES: Record<OsmSyncRun['status'], string> = {
RUNNING: 'bg-blue-100 text-blue-800 ring-blue-200',
SUCCESS: 'bg-emerald-100 text-emerald-800 ring-emerald-200',
PARTIAL: 'bg-amber-100 text-amber-800 ring-amber-200',
FAILED: 'bg-red-100 text-red-800 ring-red-200',
};
const STATUS_ICONS: Record<OsmSyncRun['status'], React.ReactNode> = {
RUNNING: <RefreshCw className="h-3 w-3 animate-spin" />,
SUCCESS: <CheckCircle className="h-3 w-3" />,
PARTIAL: <AlertTriangle className="h-3 w-3" />,
FAILED: <XCircle className="h-3 w-3" />,
};
function formatRelative(iso: string | null): string {
if (!iso) return '—';
const d = new Date(iso);
const diff = Date.now() - d.getTime();
if (diff < 60_000) return 'vừa xong';
if (diff < 3_600_000) return `${Math.round(diff / 60_000)} phút trước`;
if (diff < 86_400_000) return `${Math.round(diff / 3_600_000)} giờ trước`;
return `${Math.round(diff / 86_400_000)} ngày trước`;
}
function formatDuration(start: string, end: string | null): string {
const startMs = new Date(start).getTime();
const endMs = end ? new Date(end).getTime() : Date.now();
const sec = Math.round((endMs - startMs) / 1000);
if (sec < 60) return `${sec}s`;
if (sec < 3600) return `${Math.floor(sec / 60)}m ${sec % 60}s`;
return `${Math.floor(sec / 3600)}h ${Math.floor((sec % 3600) / 60)}m`;
}
export default function AdminOsmDashboardPage() {
const [coverage, setCoverage] = useState<OsmCoverageSummary | null>(null);
const [runs, setRuns] = useState<OsmSyncRun[]>([]);
const [layers, setLayers] = useState<OsmSyncLayer[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [triggering, setTriggering] = useState<string | null>(null);
const refresh = useCallback(async () => {
setLoading(true);
setError(null);
try {
const [cov, rs, ls] = await Promise.all([
osmSyncApi.coverage(),
osmSyncApi.runs({ limit: 30 }),
osmSyncApi.layers(),
]);
setCoverage(cov);
setRuns(rs);
setLayers(ls);
} catch (e) {
setError(e instanceof Error ? e.message : 'Lỗi tải dashboard');
} finally {
setLoading(false);
}
}, []);
useEffect(() => {
refresh();
const int = setInterval(refresh, 15_000); // poll while RUNNING runs visible
return () => clearInterval(int);
}, [refresh]);
const trigger = async (layer: string, category?: string) => {
const key = `${layer}/${category ?? '-'}`;
setTriggering(key);
try {
await osmSyncApi.trigger({ layer, category });
await refresh();
} catch (e) {
setError(e instanceof Error ? e.message : 'Trigger fail');
} finally {
setTriggering(null);
}
};
return (
<div className="flex flex-col gap-4">
<div className="flex items-center justify-between">
<div>
<h1 className="text-2xl font-semibold">OSM Sync Dashboard</h1>
<p className="text-sm text-muted-foreground">
Đng bộ OpenStreetMap Goodgo: ranh giới hành chính, POI, KCN, giao thông.
</p>
</div>
<Button variant="outline" size="sm" onClick={refresh} disabled={loading}>
<RefreshCw className={`mr-1.5 h-3.5 w-3.5 ${loading ? 'animate-spin' : ''}`} />
Làm mới
</Button>
</div>
{error && (
<div className="rounded-md border border-destructive/40 bg-destructive/10 px-4 py-2 text-sm text-destructive">
{error}
</div>
)}
{/* Top stats */}
{coverage && (
<div className="grid grid-cols-2 gap-3 md:grid-cols-5">
<StatCard
icon={<Layers className="h-4 w-4" />}
label="Đơn vị hành chính"
value={coverage.totals.administrativeUnits.toLocaleString('vi-VN')}
/>
<StatCard
icon={<MapPin className="h-4 w-4" />}
label="POI tổng"
value={coverage.totals.poiTotal.toLocaleString('vi-VN')}
/>
<StatCard
icon={<MapPin className="h-4 w-4 text-green-600" />}
label="KCN"
value={coverage.totals.industrialParks.toLocaleString('vi-VN')}
/>
<StatCard
icon={<Train className="h-4 w-4" />}
label="Bến/Ga"
value={coverage.totals.transportStations.toLocaleString('vi-VN')}
/>
<StatCard
icon={<Train className="h-4 w-4" />}
label="Tuyến giao thông"
value={coverage.totals.transportLines.toLocaleString('vi-VN')}
/>
</div>
)}
{/* Coverage table */}
<Card>
<CardContent className="p-0">
<div className="border-b border-border px-4 py-2.5">
<h2 className="text-sm font-semibold">Coverage theo layer</h2>
</div>
<Table>
<TableHeader>
<TableRow>
<TableHead>Layer / Category</TableHead>
<TableHead className="text-right">Tổng</TableHead>
<TableHead className="text-right">Promoted</TableHead>
<TableHead className="text-right">Raw</TableHead>
<TableHead>Sync gần nhất</TableHead>
<TableHead className="text-right">Hành đng</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{coverage?.rows.map((r) => {
const key = `${r.layer}/${r.category ?? '-'}`;
const layerDef = layers.find(
(l) => l.layer === r.layer && (l.category ?? null) === (r.category ?? null),
);
return (
<TableRow key={key}>
<TableCell>
<div className="font-medium">{r.layer}</div>
{r.category && (
<div className="text-xs text-muted-foreground">{r.category}</div>
)}
</TableCell>
<TableCell className="text-right font-mono">
{r.total.toLocaleString('vi-VN')}
{r.withGeometry !== undefined && r.withGeometry !== r.total && (
<span className="ml-1 text-xs text-muted-foreground">
({r.withGeometry} geom)
</span>
)}
</TableCell>
<TableCell className="text-right font-mono">
{r.promoted?.toLocaleString('vi-VN') ?? '—'}
</TableCell>
<TableCell className="text-right font-mono">
{r.raw?.toLocaleString('vi-VN') ?? '—'}
</TableCell>
<TableCell className="text-sm text-muted-foreground">
{formatRelative(r.lastSyncedAt)}
</TableCell>
<TableCell className="text-right">
{layerDef && (
<Button
variant="outline"
size="sm"
disabled={triggering === key}
onClick={() => trigger(r.layer, r.category ?? undefined)}
>
{triggering === key ? (
<RefreshCw className="mr-1 h-3 w-3 animate-spin" />
) : (
<PlayCircle className="mr-1 h-3 w-3" />
)}
Sync
</Button>
)}
</TableCell>
</TableRow>
);
})}
</TableBody>
</Table>
</CardContent>
</Card>
{/* Recent runs */}
<Card>
<CardContent className="p-0">
<div className="border-b border-border px-4 py-2.5">
<h2 className="text-sm font-semibold">Sync runs gần đây</h2>
</div>
<Table>
<TableHeader>
<TableRow>
<TableHead>Layer</TableHead>
<TableHead>Status</TableHead>
<TableHead className="text-right">Added</TableHead>
<TableHead className="text-right">Updated</TableHead>
<TableHead className="text-right">Skipped</TableHead>
<TableHead>Bắt đu</TableHead>
<TableHead>Thời gian</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{runs.length === 0 ? (
<TableRow>
<TableCell colSpan={7} className="py-8 text-center text-sm text-muted-foreground">
Chưa sync run nào.
</TableCell>
</TableRow>
) : (
runs.map((r) => (
<TableRow key={r.id}>
<TableCell>
<div className="font-medium">{r.layer}</div>
{r.category && (
<div className="text-xs text-muted-foreground">{r.category}</div>
)}
</TableCell>
<TableCell>
<span
className={`inline-flex items-center gap-1 rounded-pill px-2 py-0.5 text-xs font-medium ring-1 ring-inset ${STATUS_STYLES[r.status]}`}
>
{STATUS_ICONS[r.status]}
{r.status}
</span>
</TableCell>
<TableCell className="text-right font-mono">{r.rowsAdded}</TableCell>
<TableCell className="text-right font-mono">{r.rowsUpdated}</TableCell>
<TableCell className="text-right font-mono">{r.rowsSkipped}</TableCell>
<TableCell className="text-sm text-muted-foreground">
<Clock className="mr-1 inline h-3 w-3" />
{formatRelative(r.startedAt)}
</TableCell>
<TableCell className="text-sm text-muted-foreground">
{formatDuration(r.startedAt, r.finishedAt)}
</TableCell>
</TableRow>
))
)}
</TableBody>
</Table>
</CardContent>
</Card>
</div>
);
}
function StatCard({
icon,
label,
value,
}: {
icon: React.ReactNode;
label: string;
value: string;
}) {
return (
<Card>
<CardContent className="flex items-center gap-3 p-4">
<div className="flex h-9 w-9 shrink-0 items-center justify-center rounded-md bg-primary/10 text-primary">
{icon}
</div>
<div className="min-w-0">
<div className="truncate text-xs uppercase text-muted-foreground">{label}</div>
<div className="truncate text-lg font-semibold">{value}</div>
</div>
</CardContent>
</Card>
);
}

View File

@@ -7,6 +7,7 @@ import {
ShieldCheck,
Building2,
Factory,
Globe,
LogOut,
Menu,
Sparkles,
@@ -38,6 +39,7 @@ export default function AdminLayout({ children }: { children: React.ReactNode })
{ href: '/admin/accounts/developers' as const, label: 'Tài khoản CĐT', icon: Building2 },
{ href: '/admin/accounts/park-operators' as const, label: 'Tài khoản KCN', icon: Factory },
{ href: '/admin/industrial/osm-review' as const, label: 'Review OSM (KCN)', icon: Factory },
{ href: '/admin/osm' as const, label: 'OSM Sync Dashboard', icon: Globe },
{ href: '/admin/settings/ai' as const, label: t('adminNav.aiSettings'), icon: Sparkles },
];

View File

@@ -14,6 +14,7 @@ import {
} from 'lucide-react';
import * as React from 'react';
import { ParkMap } from '@/components/khu-cong-nghiep/park-map';
import { NearbyPoiSidebar } from '@/components/poi/nearby-poi-sidebar';
import { Badge } from '@/components/ui/badge';
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';
import {
@@ -252,6 +253,16 @@ export function KhuCongNghiepDetailClient({ park }: KhuCongNghiepDetailClientPro
{/* Sidebar */}
<div className="space-y-6">
{/* OSM POI nearby (schools, hospitals, banks, transport, …) */}
{park.latitude != null && park.longitude != null && (
<NearbyPoiSidebar
lat={park.latitude}
lng={park.longitude}
radius={3000}
categories={['HOSPITAL', 'BANK', 'GAS_STATION', 'BUS_STATION', 'METRO_STATION', 'POLICE']}
/>
)}
{/* Rent info */}
<Card>
<CardHeader>

View File

@@ -6,6 +6,7 @@ import * as React from 'react';
import { AddToCompareButton } from '@/components/comparison/add-to-compare-button';
import { AiAdviceCards } from '@/components/listings/ai-advice-cards';
import { ImageGallery } from '@/components/listings/image-gallery';
import { NearbyPoiSidebar } from '@/components/poi/nearby-poi-sidebar';
import { InquiryModal } from '@/components/listings/inquiry-modal';
import { PriceHistoryChart } from '@/components/listings/price-history-chart';
import { ReportListingModal } from '@/components/listings/report-listing-modal';
@@ -897,6 +898,15 @@ export function ListingDetailClient({ listing }: ListingDetailClientProps) {
onOpenChange={setReportOpen}
/>
{/* OSM POI nearby — schools, hospitals, markets, banks, metro… */}
{property.latitude != null && property.longitude != null && (
<NearbyPoiSidebar
lat={property.latitude}
lng={property.longitude}
radius={1500}
/>
)}
{/* Stats */}
<Card>
<CardContent className="pt-5">

View File

@@ -0,0 +1,151 @@
'use client';
import { Loader2, MapPin } from 'lucide-react';
import * as React from 'react';
import {
POI_ICONS,
POI_LABELS,
poiApi,
type NearbyPoiResult,
type PoiCategory,
} from '@/lib/poi-api';
interface Props {
/** Centre coordinates of the asset (listing / project / KCN). */
lat: number;
lng: number;
/** Search radius in metres. Default 1500m (~15 phút đi bộ). */
radius?: number;
/** Restrict to these categories. Default: 6 most relevant for residential. */
categories?: PoiCategory[];
/** N nearest POI shown per category. */
limitPerCategory?: number;
className?: string;
}
const DEFAULT_CATEGORIES: PoiCategory[] = [
'SCHOOL_PRIMARY',
'SCHOOL_SECONDARY',
'HOSPITAL',
'MARKET',
'BANK',
'METRO_STATION',
];
function formatDistance(m: number): string {
if (m < 1000) return `${m} m`;
return `${(m / 1000).toFixed(1)} km`;
}
/**
* Sidebar widget that lists the nearest POI of each category around a
* geo-tagged asset. Renders inside listing detail, project detail and KCN
* detail pages.
*/
export function NearbyPoiSidebar({
lat,
lng,
radius = 1500,
categories = DEFAULT_CATEGORIES,
limitPerCategory = 3,
className,
}: Props) {
const [data, setData] = React.useState<NearbyPoiResult | null>(null);
const [loading, setLoading] = React.useState(true);
const [error, setError] = React.useState<string | null>(null);
React.useEffect(() => {
let cancelled = false;
setLoading(true);
setError(null);
poiApi
.nearby({ lat, lng, radius, categories, limitPerCategory })
.then((res) => {
if (cancelled) return;
setData(res);
})
.catch((err: Error) => {
if (cancelled) return;
setError(err.message ?? 'Không tải được tiện ích');
})
.finally(() => {
if (cancelled) return;
setLoading(false);
});
return () => {
cancelled = true;
};
}, [lat, lng, radius, categories, limitPerCategory]);
if (loading) {
return (
<div
className={`flex items-center gap-2 rounded-lg border border-border bg-card p-4 text-sm text-muted-foreground ${className ?? ''}`}
>
<Loader2 className="h-4 w-4 animate-spin" />
Đang tải tiện ích xung quanh
</div>
);
}
if (error) {
return (
<div
className={`rounded-lg border border-destructive/30 bg-destructive/5 p-4 text-sm text-destructive ${className ?? ''}`}
>
{error}
</div>
);
}
if (!data || data.all.length === 0) {
return (
<div
className={`rounded-lg border border-border bg-card p-4 text-sm text-muted-foreground ${className ?? ''}`}
>
Chưa dữ liệu tiện ích trong bán kính {formatDistance(radius)}.
</div>
);
}
return (
<div className={`rounded-lg border border-border bg-card ${className ?? ''}`}>
<div className="flex items-center justify-between border-b border-border px-4 py-2.5">
<h3 className="text-sm font-semibold text-foreground">Tiện ích xung quanh</h3>
<span className="text-xs text-muted-foreground">
{data.meta.totalCount} điểm · bán kính {formatDistance(data.meta.radiusMeters)}
</span>
</div>
<div className="flex flex-col divide-y divide-border">
{categories.map((cat) => {
const items = data.byCategory[cat] ?? [];
if (items.length === 0) return null;
return (
<div key={cat} className="px-4 py-2.5">
<div className="mb-1.5 flex items-center gap-1.5 text-xs font-medium uppercase text-muted-foreground">
<span aria-hidden>{POI_ICONS[cat]}</span>
{POI_LABELS[cat]}
</div>
<ul className="flex flex-col gap-1.5">
{items.map((p) => (
<li key={p.id} className="flex items-start gap-2 text-sm">
<MapPin className="mt-0.5 h-3.5 w-3.5 shrink-0 text-muted-foreground" />
<div className="min-w-0 flex-1">
<div className="truncate font-medium text-foreground">{p.name}</div>
{p.address && (
<div className="truncate text-xs text-muted-foreground">{p.address}</div>
)}
</div>
<span className="shrink-0 rounded bg-muted px-1.5 py-0.5 text-xs font-medium text-muted-foreground">
{formatDistance(p.distanceM)}
</span>
</li>
))}
</ul>
</div>
);
})}
</div>
</div>
);
}

View File

@@ -0,0 +1,173 @@
'use client';
import { ChevronDown } from 'lucide-react';
import * as React from 'react';
import { POI_ICONS, POI_LABELS, type PoiCategory } from '@/lib/poi-api';
export interface PoiNearbyConstraint {
/** Required POI categories — listing must have at least one of each within radius. */
categories: PoiCategory[];
/** Radius in metres (50 - 5000). */
radiusM: number;
}
interface Props {
value: PoiNearbyConstraint;
onChange: (next: PoiNearbyConstraint) => void;
className?: string;
}
const RADIUS_OPTIONS = [
{ value: 300, label: '300m (đi bộ 5 phút)' },
{ value: 500, label: '500m' },
{ value: 1000, label: '1 km' },
{ value: 1500, label: '1.5 km' },
{ value: 2500, label: '2.5 km' },
{ value: 5000, label: '5 km (xe máy 10 phút)' },
];
const QUICK_PRESETS: { label: string; categories: PoiCategory[] }[] = [
{ label: 'Gia đình con nhỏ', categories: ['SCHOOL_PRIMARY', 'HOSPITAL', 'MARKET'] },
{ label: 'Đi làm văn phòng', categories: ['BUS_STATION', 'METRO_STATION', 'BANK'] },
{ label: 'Tiện nghi', categories: ['SUPERMARKET', 'PARK', 'PHARMACY'] },
];
const ALL_GROUPS: { label: string; items: PoiCategory[] }[] = [
{ label: 'Giáo dục', items: ['SCHOOL_PRIMARY', 'SCHOOL_SECONDARY', 'UNIVERSITY'] },
{ label: 'Y tế', items: ['HOSPITAL', 'CLINIC', 'PHARMACY'] },
{ label: 'Thương mại', items: ['MARKET', 'SUPERMARKET', 'MALL', 'CONVENIENCE'] },
{ label: 'Tài chính', items: ['BANK', 'ATM'] },
{ label: 'Giao thông', items: ['METRO_STATION', 'BUS_STATION', 'RAILWAY_STATION', 'AIRPORT'] },
{ label: 'Khác', items: ['PARK', 'GAS_STATION', 'POLICE', 'POST_OFFICE'] },
];
/**
* Compact filter widget for the search page: pick "in X meters" + which
* POI categories are required nearby. Designed to slot into an existing
* search filter bar — see `apps/web/app/[locale]/(public)/search/page.tsx`.
*/
export function PoiSearchFilter({ value, onChange, className }: Props) {
const [open, setOpen] = React.useState(false);
const ref = React.useRef<HTMLDivElement>(null);
React.useEffect(() => {
if (!open) return;
const onClick = (e: MouseEvent) => {
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
};
document.addEventListener('mousedown', onClick);
return () => document.removeEventListener('mousedown', onClick);
}, [open]);
const toggle = (cat: PoiCategory) => {
const next = value.categories.includes(cat)
? value.categories.filter((c) => c !== cat)
: [...value.categories, cat];
onChange({ ...value, categories: next });
};
const summary =
value.categories.length === 0
? 'Tiện ích xung quanh'
: `${value.categories.length} tiện ích · ${value.radiusM >= 1000 ? `${value.radiusM / 1000}km` : `${value.radiusM}m`}`;
return (
<div ref={ref} className={`relative ${className ?? ''}`}>
<button
type="button"
onClick={() => setOpen((v) => !v)}
aria-haspopup="dialog"
aria-expanded={open}
className="flex h-9 items-center gap-2 rounded-md border border-border bg-background px-3 text-sm text-foreground transition-colors hover:bg-accent"
>
<span>{summary}</span>
<ChevronDown
className={`h-3.5 w-3.5 text-muted-foreground transition-transform ${open ? 'rotate-180' : ''}`}
/>
</button>
{open && (
<div className="absolute right-0 top-full z-popover mt-2 w-[28rem] max-w-[90vw] overflow-hidden rounded-lg border border-border bg-card shadow-elevation-3">
{/* Radius */}
<div className="border-b border-border p-3">
<label className="mb-1 block text-xs font-medium text-muted-foreground">
Trong bán kính
</label>
<select
value={value.radiusM}
onChange={(e) => onChange({ ...value, radiusM: Number(e.target.value) })}
className="h-8 w-full rounded-md border border-border bg-background px-2 text-sm"
>
{RADIUS_OPTIONS.map((o) => (
<option key={o.value} value={o.value}>
{o.label}
</option>
))}
</select>
</div>
{/* Quick presets */}
<div className="border-b border-border p-3">
<div className="mb-1.5 text-xs font-medium text-muted-foreground">Gợi ý</div>
<div className="flex flex-wrap gap-1.5">
{QUICK_PRESETS.map((p) => (
<button
key={p.label}
type="button"
onClick={() => onChange({ ...value, categories: p.categories })}
className="rounded-full border border-border bg-background px-2.5 py-1 text-xs transition-colors hover:bg-accent"
>
{p.label}
</button>
))}
{value.categories.length > 0 && (
<button
type="button"
onClick={() => onChange({ ...value, categories: [] })}
className="rounded-full border border-destructive/30 bg-destructive/10 px-2.5 py-1 text-xs text-destructive transition-colors hover:bg-destructive/15"
>
Bỏ chọn
</button>
)}
</div>
</div>
{/* Category groups */}
<div className="max-h-72 overflow-y-auto p-3">
{ALL_GROUPS.map((g) => (
<div key={g.label} className="mb-2 last:mb-0">
<div className="mb-1 text-xs font-medium uppercase text-muted-foreground">
{g.label}
</div>
<div className="grid grid-cols-2 gap-1.5">
{g.items.map((cat) => {
const checked = value.categories.includes(cat);
return (
<label
key={cat}
className={`flex cursor-pointer items-center gap-1.5 rounded-md border px-2 py-1 text-xs transition-colors ${
checked
? 'border-primary bg-primary/10 text-primary'
: 'border-border bg-background hover:bg-accent'
}`}
>
<input
type="checkbox"
checked={checked}
onChange={() => toggle(cat)}
className="h-3 w-3"
/>
<span aria-hidden>{POI_ICONS[cat]}</span>
<span className="truncate">{POI_LABELS[cat]}</span>
</label>
);
})}
</div>
</div>
))}
</div>
</div>
)}
</div>
);
}

View File

@@ -0,0 +1,58 @@
import { apiClient } from './api-client';
export interface OsmCoverageRow {
layer: string;
category: string | null;
total: number;
withGeometry?: number;
promoted?: number;
raw?: number;
lastSyncedAt: string | null;
}
export interface OsmCoverageSummary {
rows: OsmCoverageRow[];
totals: {
administrativeUnits: number;
poiTotal: number;
industrialParks: number;
transportStations: number;
transportLines: number;
};
}
export interface OsmSyncRun {
id: string;
layer: string;
category: string | null;
chunk: string | null;
startedAt: string;
finishedAt: string | null;
status: 'RUNNING' | 'SUCCESS' | 'PARTIAL' | 'FAILED';
rowsAdded: number;
rowsUpdated: number;
rowsSkipped: number;
rowsLocked: number;
errorMessage: string | null;
}
export interface OsmSyncLayer {
layer: string;
category?: string;
weight: 'light' | 'medium' | 'heavy';
}
export const osmSyncApi = {
layers: () => apiClient.get<OsmSyncLayer[]>('/admin/osm/layers'),
coverage: () => apiClient.get<OsmCoverageSummary>('/admin/osm/coverage'),
runs: (params: { layer?: string; status?: string; limit?: number } = {}) => {
const q = new URLSearchParams();
Object.entries(params).forEach(([k, v]) => {
if (v !== undefined && v !== '') q.append(k, String(v));
});
const qs = q.toString();
return apiClient.get<OsmSyncRun[]>(`/admin/osm/runs${qs ? `?${qs}` : ''}`);
},
trigger: (body: { layer: string; category?: string; chunk?: string }) =>
apiClient.post<{ runId: string; status: string }>('/admin/osm/runs', body),
};

140
apps/web/lib/poi-api.ts Normal file
View File

@@ -0,0 +1,140 @@
import { apiClient } from './api-client';
/* -------------------------------------------------------------------------- */
/* Types */
/* -------------------------------------------------------------------------- */
export type PoiCategory =
| 'SCHOOL_PRIMARY' | 'SCHOOL_SECONDARY' | 'UNIVERSITY'
| 'HOSPITAL' | 'CLINIC' | 'PHARMACY'
| 'MARKET' | 'SUPERMARKET' | 'MALL' | 'CONVENIENCE'
| 'BANK' | 'ATM'
| 'PARK'
| 'GAS_STATION' | 'POLICE' | 'POST_OFFICE'
| 'METRO_STATION' | 'RAILWAY_STATION' | 'BUS_STATION' | 'AIRPORT';
/** Vietnamese display labels for each POI category. */
export const POI_LABELS: Record<PoiCategory, string> = {
SCHOOL_PRIMARY: 'Trường tiểu học',
SCHOOL_SECONDARY: 'Trường THCS / THPT',
UNIVERSITY: 'Đại học / Cao đẳng',
HOSPITAL: 'Bệnh viện',
CLINIC: 'Phòng khám',
PHARMACY: 'Nhà thuốc',
MARKET: 'Chợ',
SUPERMARKET: 'Siêu thị',
MALL: 'TTTM',
CONVENIENCE: 'Cửa hàng tiện lợi',
BANK: 'Ngân hàng',
ATM: 'ATM',
PARK: 'Công viên',
GAS_STATION: 'Cây xăng',
POLICE: 'Công an',
POST_OFFICE: 'Bưu điện',
METRO_STATION: 'Ga Metro',
RAILWAY_STATION: 'Ga tàu',
BUS_STATION: 'Bến xe',
AIRPORT: 'Sân bay',
};
/** Single-emoji icon for chips / map markers (no extra image dep needed). */
export const POI_ICONS: Record<PoiCategory, string> = {
SCHOOL_PRIMARY: '🏫', SCHOOL_SECONDARY: '🎒', UNIVERSITY: '🎓',
HOSPITAL: '🏥', CLINIC: '⚕️', PHARMACY: '💊',
MARKET: '🛒', SUPERMARKET: '🏪', MALL: '🛍️', CONVENIENCE: '🏬',
BANK: '🏦', ATM: '🏧',
PARK: '🌳',
GAS_STATION: '⛽', POLICE: '👮', POST_OFFICE: '📮',
METRO_STATION: '🚇', RAILWAY_STATION: '🚉', BUS_STATION: '🚌', AIRPORT: '✈️',
};
/** Tailwind colour class per category — keep marker coding consistent. */
export const POI_COLORS: Record<PoiCategory, string> = {
SCHOOL_PRIMARY: '#3b82f6', SCHOOL_SECONDARY: '#2563eb', UNIVERSITY: '#1d4ed8',
HOSPITAL: '#ef4444', CLINIC: '#f87171', PHARMACY: '#fb7185',
MARKET: '#f59e0b', SUPERMARKET: '#fbbf24', MALL: '#fcd34d', CONVENIENCE: '#fde68a',
BANK: '#8b5cf6', ATM: '#a78bfa',
PARK: '#22c55e',
GAS_STATION: '#64748b', POLICE: '#0f172a', POST_OFFICE: '#be185d',
METRO_STATION: '#0ea5e9', RAILWAY_STATION: '#0284c7', BUS_STATION: '#0369a1', AIRPORT: '#075985',
};
export interface NearbyPoi {
id: string;
name: string;
category: PoiCategory;
distanceM: number;
lat: number;
lng: number;
address: string | null;
}
export interface NearbyPoiResult {
byCategory: Partial<Record<PoiCategory, NearbyPoi[]>>;
all: NearbyPoi[];
meta: { radiusMeters: number; totalCount: number; requestedCategories: PoiCategory[] | null };
}
export interface PoiBboxFeatureCollection {
type: 'FeatureCollection';
features: {
type: 'Feature';
id: string;
geometry: { type: 'Point'; coordinates: [number, number] };
properties: {
id: string;
name: string;
category: PoiCategory;
provinceCode: string | null;
districtCode: string | null;
};
}[];
meta: { count: number; truncated: boolean; categories: PoiCategory[] };
}
/* -------------------------------------------------------------------------- */
/* API */
/* -------------------------------------------------------------------------- */
export const poiApi = {
/**
* Fetch nearest N POI (per category) within `radius` metres of the given
* point. Drives the "tiện ích xung quanh" sidebar.
*/
nearby: (params: {
lat: number;
lng: number;
radius: number;
categories?: PoiCategory[];
limitPerCategory?: number;
}): Promise<NearbyPoiResult> => {
const q = new URLSearchParams({
lat: String(params.lat),
lng: String(params.lng),
radius: String(params.radius),
});
if (params.categories?.length) q.set('categories', params.categories.join(','));
if (params.limitPerCategory) q.set('limitPerCategory', String(params.limitPerCategory));
return apiClient.get<NearbyPoiResult>(`/poi/nearby?${q.toString()}`);
},
/** GeoJSON for map overlays. Used by the listing detail mini-map and KCN page. */
byBbox: (params: {
south: number;
west: number;
north: number;
east: number;
categories?: PoiCategory[];
limit?: number;
}): Promise<PoiBboxFeatureCollection> => {
const q = new URLSearchParams({
south: String(params.south),
west: String(params.west),
north: String(params.north),
east: String(params.east),
});
if (params.categories?.length) q.set('categories', params.categories.join(','));
if (params.limit) q.set('limit', String(params.limit));
return apiClient.get<PoiBboxFeatureCollection>(`/poi/by-bbox?${q.toString()}`);
},
};

92
docs/osm-data-model.md Normal file
View File

@@ -0,0 +1,92 @@
# OSM Data Model — GoodGo Platform
This document is the canonical reference for every OpenStreetMap-sourced
table in the GoodGo database, the sync pipelines that populate them, and
the query patterns that use them.
## Tables at a glance
| Table | Source | Geometry | Sync cadence | Used by |
|-------|--------|----------|--------------|---------|
| `vn_provinces` | OSM `boundary=administrative + admin_level=4` | MultiPolygon | Weekly (Mon 02:30 ICT) | `GeoLookupService`, KCN sync, address auto-fill |
| `vn_districts` | OSM `admin_level=6` | MultiPolygon | Weekly (Wed 02:30 ICT) | Same as above. **After the 2025 reform** this table effectively holds the new ward / commune layer (~3,200 units), since Vietnam dropped the district level. The schema name is kept for backwards-compat with goodgo's existing FK references. |
| `vn_wards` | OSM `admin_level=8` | MultiPolygon | Weekly (Sat 02:30 ICT) | Same as above. **Note**: after the 2025 admin reform Vietnam only uses level=4 (province) + level=6 (ward/commune). OSM doesn't currently tag any VN feature with admin_level=8, so this table will stay empty until/unless the policy changes. Kept for forward-compat. |
| `Poi` | OSM nodes/ways/relations matching 20 category selectors | Point | Daily 1 category rotation (02:00 ICT) | `/poi/nearby`, `/poi/by-bbox`, listing sidebar, search filter |
| `TransportLine` | OSM `route=subway|train|highway` relations | MultiLineString | Monthly | Distance scoring, planned for Phase 2 UX |
| `IndustrialPark` | OSM `landuse=industrial` ways/relations | Point + MultiPolygon boundary | Monthly (1st 03:00 ICT, 4 chunks) | `/industrial/parks/*`, KCN catalog |
| `OsmSyncRun` | Generated by orchestrator | — | Append-only audit | `/admin/osm` dashboard |
All sync writes are gated by `OSM_SYNC_ENABLED=true` so dev / staging
environments don't hit Overpass accidentally.
## GeoLookupService — the foundation
Every other layer depends on `vn_provinces.geometry` for PostGIS
`ST_Contains` lookups. The service exposes:
```ts
const r = await geo.lookup(lng, lat);
// → { province: { code, name }, district: { code, name }, ward: { code, name } }
const inside = await geo.isInVietnam(lng, lat);
// → boolean
const cov = await geo.coverage();
// → { provinces: { total, withGeometry, lastSyncedAt }, districts: ..., wards: ... }
```
It replaces the old `nearestProvince()` heuristic that walked a
hardcoded centroid table.
## Quality gates baked into sync scripts
1. **Geographic gate**`isPointInVietnam(lng, lat)` from
`scripts/data/vn-country-polygon.ts` rejects rows whose centroid
falls outside the VN mainland polygon (catches China / Laos /
Cambodia bleed across the Overpass bbox chunks).
2. **Name gate** — rows whose `name` contains zero Latin/Vietnamese
letters (`/[A-Za-zÀ-ỹ]/`) are dropped (filters CJK / Khmer / Thai).
3. **Lock gate** — when an admin sets `osmLocked=true` or adds a column
to `lockedFields`, the next sync skips that row entirely (or that
column) so manual edits survive.
## Adding a new POI category
1. Add the enum value to `PoiCategory` in `prisma/schema.prisma` and
create a Prisma migration that `ALTER TYPE "PoiCategory" ADD VALUE`.
2. Add the Overpass selector to `CATEGORY_QUERIES` in
`scripts/sync-osm-poi.ts`.
3. Append the same enum value to the `POI_CATEGORIES` rotation list in
`OsmSyncCronService` so the cron picks it up.
4. Add labels + icons + colour to `apps/web/lib/poi-api.ts` so the UI
chips render.
That's it — `OsmSyncService.findLayer('poi', 'YOUR_CAT')` will return a
def automatically because `SYNC_LAYERS` is generated from the enum keys.
## Operational runbook
* **Sync hangs / 504 from Overpass** — `kubectl describe pod` on the
Kaniko-style sync runner shows the chunk in flight. The script has
a 5× retry on the clone step (HTTP 504 from Gitea is transient).
For Overpass itself, raise the per-script `[out:json][timeout:N]`
by editing the script. Default 180s for POI, 300s for boundaries.
* **Runs stuck in `RUNNING` state** — `OsmSyncOrchestrator` writes the
row before spawning the script. If the script process dies without
emitting an `exit` event, the row stays RUNNING. Mitigation: cron
job to flip RUNNING > 6h old to FAILED with `errorMessage='timeout'`.
* **Conflict logs** — when sync updates a column the admin had locked,
it skips the column silently. There is no separate conflict table
(yet). To audit, search Loki for `[osm-sync] skipping locked field`.
## Phase status
| Phase | Status | Notes |
|-------|--------|-------|
| 0 — Admin boundaries + GeoLookupService | ✅ Schema, sync, service done. Provinces synced (33), districts in progress |
| 1 — POI catalog + sync | ✅ Schema + sync script + NestJS module + sidebar component done. Hospital category synced (~500 rows) |
| 2 — Transport (metro/railway/airport) | 🟡 Stations synced via POI; lines layer pending |
| 3 — Buildings / landuse | ⏳ Deferred — admin says low priority |
| 4 — Sync orchestrator + admin dashboard | ✅ Service + cron + Prometheus-friendly stats + admin UI done |
| 5 — User-facing UX | 🟡 Listing + KCN sidebar wired; search filter widget built; map overlays pending |
| 6 — Performance hardening | ⏳ Materialized views + Redis cache pending |

View File

@@ -0,0 +1,51 @@
-- Add PostGIS geometry + OSM provenance to vn_provinces / vn_districts / vn_wards.
-- Geometry is `MultiPolygon` (some provinces have offshore islands), centroid is `Point`.
-- All columns are nullable to allow incremental backfill from the Overpass sync.
-- ── vn_provinces ────────────────────────────────────────────────────────────
ALTER TABLE "vn_provinces"
ADD COLUMN IF NOT EXISTS "osmId" BIGINT,
ADD COLUMN IF NOT EXISTS "areaKm2" DOUBLE PRECISION,
ADD COLUMN IF NOT EXISTS "population" INTEGER,
ADD COLUMN IF NOT EXISTS "lastSyncedAt" TIMESTAMP(3),
ADD COLUMN IF NOT EXISTS "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP;
SELECT AddGeometryColumn('public', 'vn_provinces', 'geometry', 4326, 'MULTIPOLYGON', 2);
SELECT AddGeometryColumn('public', 'vn_provinces', 'centroid', 4326, 'POINT', 2);
CREATE UNIQUE INDEX IF NOT EXISTS "vn_provinces_osmId_key" ON "vn_provinces"("osmId") WHERE "osmId" IS NOT NULL;
CREATE INDEX IF NOT EXISTS "vn_provinces_geometry_idx" ON "vn_provinces" USING GIST ("geometry");
CREATE INDEX IF NOT EXISTS "vn_provinces_centroid_idx" ON "vn_provinces" USING GIST ("centroid");
CREATE INDEX IF NOT EXISTS "vn_provinces_lastSyncedAt_idx" ON "vn_provinces"("lastSyncedAt");
-- ── vn_districts ────────────────────────────────────────────────────────────
ALTER TABLE "vn_districts"
ADD COLUMN IF NOT EXISTS "osmId" BIGINT,
ADD COLUMN IF NOT EXISTS "areaKm2" DOUBLE PRECISION,
ADD COLUMN IF NOT EXISTS "population" INTEGER,
ADD COLUMN IF NOT EXISTS "lastSyncedAt" TIMESTAMP(3),
ADD COLUMN IF NOT EXISTS "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP;
SELECT AddGeometryColumn('public', 'vn_districts', 'geometry', 4326, 'MULTIPOLYGON', 2);
SELECT AddGeometryColumn('public', 'vn_districts', 'centroid', 4326, 'POINT', 2);
CREATE UNIQUE INDEX IF NOT EXISTS "vn_districts_osmId_key" ON "vn_districts"("osmId") WHERE "osmId" IS NOT NULL;
CREATE INDEX IF NOT EXISTS "vn_districts_geometry_idx" ON "vn_districts" USING GIST ("geometry");
CREATE INDEX IF NOT EXISTS "vn_districts_centroid_idx" ON "vn_districts" USING GIST ("centroid");
CREATE INDEX IF NOT EXISTS "vn_districts_lastSyncedAt_idx" ON "vn_districts"("lastSyncedAt");
-- ── vn_wards ────────────────────────────────────────────────────────────────
ALTER TABLE "vn_wards"
ADD COLUMN IF NOT EXISTS "osmId" BIGINT,
ADD COLUMN IF NOT EXISTS "areaKm2" DOUBLE PRECISION,
ADD COLUMN IF NOT EXISTS "population" INTEGER,
ADD COLUMN IF NOT EXISTS "lastSyncedAt" TIMESTAMP(3),
ADD COLUMN IF NOT EXISTS "updatedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP;
SELECT AddGeometryColumn('public', 'vn_wards', 'geometry', 4326, 'MULTIPOLYGON', 2);
SELECT AddGeometryColumn('public', 'vn_wards', 'centroid', 4326, 'POINT', 2);
CREATE UNIQUE INDEX IF NOT EXISTS "vn_wards_osmId_key" ON "vn_wards"("osmId") WHERE "osmId" IS NOT NULL;
CREATE INDEX IF NOT EXISTS "vn_wards_geometry_idx" ON "vn_wards" USING GIST ("geometry");
CREATE INDEX IF NOT EXISTS "vn_wards_centroid_idx" ON "vn_wards" USING GIST ("centroid");
CREATE INDEX IF NOT EXISTS "vn_wards_lastSyncedAt_idx" ON "vn_wards"("lastSyncedAt");

View File

@@ -0,0 +1,77 @@
-- Phase 1: Poi catalog + TransportLine for OSM-sourced amenities and routes.
-- ── Enums ──────────────────────────────────────────────────────────────────
DO $$ BEGIN
CREATE TYPE "PoiCategory" AS ENUM (
'SCHOOL_PRIMARY','SCHOOL_SECONDARY','UNIVERSITY',
'HOSPITAL','CLINIC','PHARMACY',
'MARKET','SUPERMARKET','MALL','CONVENIENCE',
'BANK','ATM',
'PARK',
'GAS_STATION','POLICE','POST_OFFICE',
'METRO_STATION','RAILWAY_STATION','BUS_STATION','AIRPORT'
);
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
DO $$ BEGIN
CREATE TYPE "OsmType" AS ENUM ('NODE','WAY','RELATION');
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
DO $$ BEGIN
CREATE TYPE "OsmDataSource" AS ENUM ('OSM','OSM_PROMOTED','MANUAL');
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
-- ── Poi ────────────────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS "Poi" (
"id" TEXT PRIMARY KEY,
"category" "PoiCategory" NOT NULL,
"name" TEXT NOT NULL,
"nameEn" TEXT,
"address" TEXT,
"provinceCode" TEXT,
"districtCode" TEXT,
"wardCode" TEXT,
"osmId" BIGINT NOT NULL,
"osmType" "OsmType" NOT NULL,
"osmTags" JSONB NOT NULL,
"dataSource" "OsmDataSource" NOT NULL DEFAULT 'OSM',
"isPublic" BOOLEAN NOT NULL DEFAULT true,
"osmLocked" BOOLEAN NOT NULL DEFAULT false,
"lockedFields" TEXT[] NOT NULL DEFAULT '{}',
"lastSyncedAt" TIMESTAMP(3) NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMP(3) NOT NULL,
CONSTRAINT "Poi_osmId_key" UNIQUE ("osmId")
);
SELECT AddGeometryColumn('public', 'Poi', 'location', 4326, 'POINT', 2);
ALTER TABLE "Poi" ALTER COLUMN "location" SET NOT NULL;
CREATE INDEX IF NOT EXISTS "Poi_location_idx" ON "Poi" USING GIST ("location");
CREATE INDEX IF NOT EXISTS "Poi_cat_prov_idx" ON "Poi"("category","provinceCode");
CREATE INDEX IF NOT EXISTS "Poi_cat_dist_idx" ON "Poi"("category","districtCode");
CREATE INDEX IF NOT EXISTS "Poi_provinceCode_idx" ON "Poi"("provinceCode");
CREATE INDEX IF NOT EXISTS "Poi_dataSource_pub" ON "Poi"("dataSource","isPublic");
CREATE INDEX IF NOT EXISTS "Poi_lastSyncedAt_idx" ON "Poi"("lastSyncedAt");
-- ── TransportLine ──────────────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS "TransportLine" (
"id" TEXT PRIMARY KEY,
"type" TEXT NOT NULL,
"name" TEXT NOT NULL,
"ref" TEXT,
"osmRelationId" BIGINT,
"status" TEXT NOT NULL DEFAULT 'operational',
"lengthKm" DOUBLE PRECISION,
"lastSyncedAt" TIMESTAMP(3) NOT NULL,
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updatedAt" TIMESTAMP(3) NOT NULL,
CONSTRAINT "TransportLine_osmRelationId_key" UNIQUE ("osmRelationId")
);
SELECT AddGeometryColumn('public', 'TransportLine', 'geometry', 4326, 'MULTILINESTRING', 2);
ALTER TABLE "TransportLine" ALTER COLUMN "geometry" SET NOT NULL;
CREATE INDEX IF NOT EXISTS "TransportLine_geometry_idx" ON "TransportLine" USING GIST ("geometry");
CREATE INDEX IF NOT EXISTS "TransportLine_type_idx" ON "TransportLine"("type");
CREATE INDEX IF NOT EXISTS "TransportLine_status_idx" ON "TransportLine"("status");

View File

@@ -0,0 +1,25 @@
-- Phase 4: persistent audit log of every OSM sync run.
DO $$ BEGIN
CREATE TYPE "OsmSyncStatus" AS ENUM ('RUNNING','SUCCESS','PARTIAL','FAILED');
EXCEPTION WHEN duplicate_object THEN NULL; END $$;
CREATE TABLE IF NOT EXISTS "OsmSyncRun" (
"id" TEXT PRIMARY KEY,
"layer" TEXT NOT NULL,
"category" TEXT,
"chunk" TEXT,
"startedAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"finishedAt" TIMESTAMP(3),
"status" "OsmSyncStatus" NOT NULL DEFAULT 'RUNNING',
"rowsAdded" INTEGER NOT NULL DEFAULT 0,
"rowsUpdated" INTEGER NOT NULL DEFAULT 0,
"rowsSkipped" INTEGER NOT NULL DEFAULT 0,
"rowsLocked" INTEGER NOT NULL DEFAULT 0,
"errorMessage" TEXT,
"overpassQueryHash" TEXT,
"metadata" JSONB
);
CREATE INDEX IF NOT EXISTS "OsmSyncRun_layer_started" ON "OsmSyncRun"("layer","startedAt");
CREATE INDEX IF NOT EXISTS "OsmSyncRun_status_idx" ON "OsmSyncRun"("status");
CREATE INDEX IF NOT EXISTS "OsmSyncRun_started_idx" ON "OsmSyncRun"("startedAt");

View File

@@ -0,0 +1,39 @@
-- Phase 6: cached proximity scores for fast filter / sort.
-- Refreshed by `OsmRefreshViewsCron` (Sun 03:00 ICT) and on admin-trigger.
-- For each industrial park, the 3 nearest POI of each priority category.
CREATE MATERIALIZED VIEW IF NOT EXISTS "mv_park_nearest_poi" AS
SELECT
p.id AS park_id,
poi.category::text AS category,
poi.id AS poi_id,
poi.name AS poi_name,
ROUND(ST_Distance(p.location::geography, poi.location::geography)::numeric, 0)::int AS distance_m,
ROW_NUMBER() OVER (
PARTITION BY p.id, poi.category
ORDER BY ST_Distance(p.location::geography, poi.location::geography)
) AS rank
FROM "IndustrialPark" p
CROSS JOIN LATERAL (
SELECT id, name, category, location
FROM "Poi"
WHERE "isPublic" = true
AND category::text IN ('HOSPITAL','BANK','GAS_STATION','BUS_STATION','METRO_STATION','POLICE')
AND ST_DWithin(location::geography, p.location::geography, 5000)
) poi;
CREATE INDEX IF NOT EXISTS "mv_park_nearest_poi_park_cat" ON "mv_park_nearest_poi"(park_id, category, rank);
CREATE UNIQUE INDEX IF NOT EXISTS "mv_park_nearest_poi_uq" ON "mv_park_nearest_poi"(park_id, category, poi_id);
-- Per-province POI density (count by category) for analytics heatmaps.
CREATE MATERIALIZED VIEW IF NOT EXISTS "mv_poi_density_by_province" AS
SELECT
"provinceCode",
category::text AS category,
COUNT(*)::int AS poi_count
FROM "Poi"
WHERE "isPublic" = true
AND "provinceCode" IS NOT NULL
GROUP BY "provinceCode", category;
CREATE INDEX IF NOT EXISTS "mv_poi_density_by_province_idx" ON "mv_poi_density_by_province"("provinceCode", category);

View File

@@ -1580,9 +1580,25 @@ model VnProvince {
type String // "Thành phố Trung ương" | "Tỉnh"
codename String // slug, e.g. "thanh_pho_ho_chi_minh"
phoneCode Int?
/// OSM relation id for `boundary=administrative + admin_level=4`. Null until first sync.
osmId BigInt? @unique
/// PostGIS multipolygon (managed via raw SQL — Prisma can't model PostGIS).
geometry Unsupported("geometry(MultiPolygon, 4326)")?
/// Cached centroid for fast "show on map" without ST_Centroid every query.
centroid Unsupported("geometry(Point, 4326)")?
/// Surface area in km². Useful for density / coverage analytics.
areaKm2 Float?
/// Latest GSO population estimate when known.
population Int?
/// When the row was last refreshed from Overpass.
lastSyncedAt DateTime?
updatedAt DateTime @updatedAt
districts VnDistrict[]
@@index([codename])
@@index([geometry], type: Gist)
@@index([centroid], type: Gist)
@@index([lastSyncedAt])
@@map("vn_provinces")
}
@@ -1593,11 +1609,21 @@ model VnDistrict {
nameEn String?
type String // "Quận" | "Huyện" | "Thị xã" | "Thành phố thuộc tỉnh"
codename String
osmId BigInt? @unique
geometry Unsupported("geometry(MultiPolygon, 4326)")?
centroid Unsupported("geometry(Point, 4326)")?
areaKm2 Float?
population Int?
lastSyncedAt DateTime?
updatedAt DateTime @updatedAt
province VnProvince @relation(fields: [provinceCode], references: [code], onDelete: Restrict)
wards VnWard[]
@@index([provinceCode])
@@index([codename])
@@index([geometry], type: Gist)
@@index([centroid], type: Gist)
@@index([lastSyncedAt])
@@map("vn_districts")
}
@@ -1608,15 +1634,166 @@ model VnWard {
nameEn String?
type String // "Phường" | "Xã" | "Thị trấn"
codename String
osmId BigInt? @unique
geometry Unsupported("geometry(MultiPolygon, 4326)")?
centroid Unsupported("geometry(Point, 4326)")?
areaKm2 Float?
population Int?
lastSyncedAt DateTime?
updatedAt DateTime @updatedAt
district VnDistrict @relation(fields: [districtCode], references: [code], onDelete: Restrict)
@@index([districtCode])
@@index([codename])
@@index([geometry], type: Gist)
@@index([centroid], type: Gist)
@@index([lastSyncedAt])
@@map("vn_wards")
}
/// Historical name/code changes so legacy data (e.g. Quận 2, Quận 9) and post-2025
/// merges can still resolve to the current district/ward.
/// Categories of OSM POI we ingest. Each maps to one or more Overpass
/// tag queries — see `scripts/sync-osm-poi.ts`. Adding a new value here
/// requires a Prisma migration.
enum PoiCategory {
// Education
SCHOOL_PRIMARY
SCHOOL_SECONDARY
UNIVERSITY
// Health
HOSPITAL
CLINIC
PHARMACY
// Commerce
MARKET
SUPERMARKET
MALL
CONVENIENCE
// Finance
BANK
ATM
// Recreation
PARK
// Services
GAS_STATION
POLICE
POST_OFFICE
// Transport (also tracked here for proximity scoring; lines live in TransportLine)
METRO_STATION
RAILWAY_STATION
BUS_STATION
AIRPORT
}
enum OsmType {
NODE
WAY
RELATION
}
enum OsmDataSource {
OSM
OSM_PROMOTED
MANUAL
}
/// Catalog of points-of-interest sourced primarily from OSM. Backs the
/// "tiện ích xung quanh" feature on listing detail + KCN + project
/// proximity scoring + the search "within X meters" filters.
model Poi {
id String @id @default(cuid())
category PoiCategory
name String
nameEn String?
/// PostGIS Point — managed via raw SQL because Prisma can't model
/// `geometry`. GIST-indexed for fast nearby-radius queries.
location Unsupported("geometry(Point, 4326)")
address String?
/// Resolved by `GeoLookupService` after insert (not part of OSM data).
provinceCode String?
districtCode String?
wardCode String?
/// OSM provenance — same model as IndustrialPark.
osmId BigInt @unique
osmType OsmType
osmTags Json
dataSource OsmDataSource @default(OSM)
isPublic Boolean @default(true)
osmLocked Boolean @default(false)
lockedFields String[] @default([])
lastSyncedAt DateTime
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([location], type: Gist)
@@index([category, provinceCode])
@@index([category, districtCode])
@@index([provinceCode])
@@index([dataSource, isPublic])
@@index([lastSyncedAt])
@@map("Poi")
}
/// Transport lines (metro / railway / highway routes) — the linear
/// counterpart to Poi station entries. Used to compute "distance to
/// nearest metro line" without joining 100k station pings.
model TransportLine {
id String @id @default(cuid())
type String // METRO | RAILWAY | TRUNK | MOTORWAY | PRIMARY
name String // "Metro Số 1 Bến Thành - Suối Tiên" / "QL1A"
ref String? // "M1", "QL1A"
geometry Unsupported("geometry(MultiLineString, 4326)")
osmRelationId BigInt? @unique
status String @default("operational") // planned | under_construction | operational
lengthKm Float?
lastSyncedAt DateTime
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([geometry], type: Gist)
@@index([type])
@@index([status])
@@map("TransportLine")
}
enum OsmSyncStatus {
RUNNING
SUCCESS
PARTIAL
FAILED
}
/// Audit + monitoring record for every OSM sync run (admin boundaries,
/// POI categories, transport, KCN, etc.). Drives the `/admin/osm`
/// dashboard and Prometheus alerts.
model OsmSyncRun {
id String @id @default(cuid())
/// Coarse layer name: "admin-boundaries" / "poi" / "transport" / "industrial-parks"
layer String
/// Fine-grained scope inside the layer, when applicable.
category String?
chunk String?
startedAt DateTime @default(now())
finishedAt DateTime?
status OsmSyncStatus @default(RUNNING)
rowsAdded Int @default(0)
rowsUpdated Int @default(0)
rowsSkipped Int @default(0)
rowsLocked Int @default(0)
/// Truncated message for UI display; full stack lives in Loki.
errorMessage String? @db.Text
/// SHA-256 of the Overpass query so we can detect query drift.
overpassQueryHash String?
/// Free-form metadata (Overpass response size, kubectl run id, etc.).
metadata Json?
@@index([layer, startedAt])
@@index([status])
@@index([startedAt])
@@map("OsmSyncRun")
}
model VnAdministrativeAlias {
id String @id @default(cuid())
oldCode String? // GSO code pre-change, when known

View File

@@ -0,0 +1,216 @@
/**
* Backfill `provinceCode` / `districtCode` / `wardCode` (and the human
* `province` / `district` / `ward` text columns where present) on every
* geo-bearing entity, using the freshly synced
* `vn_provinces` / `vn_districts` / `vn_wards` polygons.
*
* Tables processed:
* - IndustrialPark (PostGIS point)
* - ProjectDevelopment (PostGIS point)
* - Listing (uses Property.location internally — joined)
* - Property (PostGIS point — most listings live here)
*
* Usage:
* NODE_OPTIONS="-r dotenv/config" DOTENV_CONFIG_PATH=.env \
* pnpm tsx scripts/backfill-admin-codes.ts [--dry-run] [--table=NAME]
*
* Strategy:
* For each entity with a `location` Point we ST_Contains against the
* province/district/ward polygons and write the matched code+name back
* into the row. Only rows where the resolved value DIFFERS from the
* existing one are touched, so re-runs are cheap.
*/
import 'dotenv/config';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '@prisma/client';
import pg from 'pg';
const pool = new pg.Pool({ connectionString: process.env['DATABASE_URL'] });
const adapter = new PrismaPg(pool);
const prisma = new PrismaClient({ adapter });
const dryRun = process.argv.includes('--dry-run');
const tableArg = process.argv.find((a) => a.startsWith('--table='))?.slice('--table='.length);
interface AdminMatch {
provinceCode: string | null;
provinceName: string | null;
districtCode: string | null;
districtName: string | null;
wardCode: string | null;
wardName: string | null;
}
/**
* Single SQL statement that joins a point against the 3 admin tables and
* returns whichever level matched. NULL when no province polygon contains
* the point (likely outside VN or polygons not synced for that area).
*/
async function resolve(lng: number, lat: number): Promise<AdminMatch> {
const rows = await prisma.$queryRawUnsafe<
{
provinceCode: string | null;
provinceName: string | null;
districtCode: string | null;
districtName: string | null;
wardCode: string | null;
wardName: string | null;
}[]
>(
`WITH p AS (
SELECT code, name FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
),
d AS (
SELECT d.code, d.name
FROM "vn_districts" d
JOIN p ON p.code = d."provinceCode"
WHERE d.geometry IS NOT NULL
AND ST_Contains(d.geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
),
w AS (
SELECT w.code, w.name
FROM "vn_wards" w
JOIN d ON d.code = w."districtCode"
WHERE w.geometry IS NOT NULL
AND ST_Contains(w.geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
)
SELECT
(SELECT code FROM p) AS "provinceCode",
(SELECT name FROM p) AS "provinceName",
(SELECT code FROM d) AS "districtCode",
(SELECT name FROM d) AS "districtName",
(SELECT code FROM w) AS "wardCode",
(SELECT name FROM w) AS "wardName"`,
lng,
lat,
);
return (
rows[0] ?? {
provinceCode: null,
provinceName: null,
districtCode: null,
districtName: null,
wardCode: null,
wardName: null,
}
);
}
async function backfillIndustrialPark(): Promise<void> {
console.log('🏭 IndustrialPark…');
const rows = await prisma.$queryRawUnsafe<
{ id: string; lat: number; lng: number; province: string }[]
>(
`SELECT id, ST_Y(location::geometry) AS lat, ST_X(location::geometry) AS lng, province
FROM "IndustrialPark"`,
);
let updated = 0;
for (const r of rows) {
const m = await resolve(r.lng, r.lat);
if (!m.provinceName) continue; // outside VN polygon
if (m.provinceName === r.province) continue;
if (!dryRun) {
await prisma.$executeRawUnsafe(
`UPDATE "IndustrialPark" SET province = $2, district = COALESCE($3, district) WHERE id = $1`,
r.id,
m.provinceName,
m.districtName,
);
}
updated++;
}
console.log(` ${updated}/${rows.length} rows would update.`);
}
async function backfillProjectDevelopment(): Promise<void> {
console.log('🏗️ ProjectDevelopment…');
const rows = await prisma.$queryRawUnsafe<
{ id: string; lat: number; lng: number; city: string; district: string; ward: string }[]
>(
`SELECT id, ST_Y(location::geometry) AS lat, ST_X(location::geometry) AS lng, city, district, ward
FROM "ProjectDevelopment"`,
);
let updated = 0;
for (const r of rows) {
const m = await resolve(r.lng, r.lat);
if (!m.provinceName) continue;
const sameCity = m.provinceName === r.city;
const sameDistrict = !m.districtName || m.districtName === r.district;
const sameWard = !m.wardName || m.wardName === r.ward;
if (sameCity && sameDistrict && sameWard) continue;
if (!dryRun) {
await prisma.$executeRawUnsafe(
`UPDATE "ProjectDevelopment"
SET city = $2,
district = COALESCE($3, district),
ward = COALESCE($4, ward)
WHERE id = $1`,
r.id,
m.provinceName,
m.districtName,
m.wardName,
);
}
updated++;
}
console.log(` ${updated}/${rows.length} rows would update.`);
}
async function backfillProperty(): Promise<void> {
console.log('🏠 Property…');
// Property has Vietnamese province / district / ward text columns; check schema.
const colsExist = await prisma.$queryRawUnsafe<{ count: bigint }[]>(
`SELECT COUNT(*)::bigint AS count
FROM information_schema.columns
WHERE table_name = 'Property' AND column_name = 'province'`,
);
if (Number(colsExist[0]?.count ?? 0n) === 0) {
console.log(' (no province column on Property — skipping)');
return;
}
const rows = await prisma.$queryRawUnsafe<
{ id: string; lat: number; lng: number; province: string | null }[]
>(
`SELECT id, ST_Y(location::geometry) AS lat, ST_X(location::geometry) AS lng, province
FROM "Property"
WHERE location IS NOT NULL`,
);
let updated = 0;
for (const r of rows) {
const m = await resolve(r.lng, r.lat);
if (!m.provinceName) continue;
if (m.provinceName === r.province) continue;
if (!dryRun) {
await prisma.$executeRawUnsafe(
`UPDATE "Property" SET province = $2 WHERE id = $1`,
r.id,
m.provinceName,
);
}
updated++;
}
console.log(` ${updated}/${rows.length} rows would update.`);
}
async function main(): Promise<void> {
console.log(`🌍 Admin-code backfill (dryRun=${dryRun})`);
if (!tableArg || tableArg === 'industrial') await backfillIndustrialPark();
if (!tableArg || tableArg === 'project') await backfillProjectDevelopment();
if (!tableArg || tableArg === 'property') await backfillProperty();
}
main()
.catch((err) => {
console.error(err);
process.exitCode = 1;
})
.finally(async () => {
await prisma.$disconnect();
await pool.end();
});

View File

@@ -0,0 +1,519 @@
/**
* Sync Vietnam administrative boundaries from OpenStreetMap into the
* `vn_provinces` / `vn_districts` / `vn_wards` tables.
*
* Usage:
* NODE_OPTIONS="-r dotenv/config" DOTENV_CONFIG_PATH=.env \
* pnpm tsx scripts/sync-osm-admin-boundaries.ts \
* [--level=4|6|8|all] [--dry-run] [--chunk=NAME]
*
* What it does:
* 1. Queries Overpass for `boundary=administrative + admin_level=N`
* relations clipped to the Vietnam bbox (split into 4 chunks).
* 2. Converts each relation's outer rings into a MultiPolygon GeoJSON.
* 3. Looks up the GSO code from OSM tags (`ref:VN`, `gso_code`,
* `iso_code`, fallback to slugified name → existing seed row).
* 4. Upserts the row, writing geometry + centroid + areaKm2 + osmId.
*
* Coverage targets:
* admin_level=4 → 63 provinces (cities of central authority + 58 tỉnh)
* admin_level=6 → ~700 districts (quận / huyện / thị xã / TP thuộc tỉnh)
* admin_level=8 → ~11.000 wards (phường / xã / thị trấn)
*
* Notes:
* • Vietnam reformed wards in 2025 (some merged). We track historic
* names via `vn_administrative_aliases` — this script populates that
* table when an OSM tag `was:name` differs from the current name.
* • Wards (level 8) are the heaviest pull (~11k polygons). We always
* chunk them into 4 geographic slices to dodge Overpass timeouts.
*/
import 'dotenv/config';
import area from '@turf/area';
import centroid from '@turf/centroid';
import { PrismaPg } from '@prisma/adapter-pg';
import { PrismaClient } from '@prisma/client';
import type { Feature, MultiPolygon, Polygon } from 'geojson';
import osmtogeojson from 'osmtogeojson';
import pg from 'pg';
import { isPointInVietnam } from './data/vn-country-polygon';
const pool = new pg.Pool({ connectionString: process.env['DATABASE_URL'] });
const adapter = new PrismaPg(pool);
const prisma = new PrismaClient({ adapter });
const OVERPASS_URL =
process.env['OVERPASS_URL'] ?? 'https://overpass-api.de/api/interpreter';
interface BBox {
south: number;
west: number;
north: number;
east: number;
}
/** Same chunks the KCN sync uses — keeps Overpass query budget reasonable. */
const CHUNKS: Record<string, BBox> = {
north: { south: 19.0, west: 102.0, north: 23.5, east: 110.0 },
northCentral: { south: 16.5, west: 102.0, north: 19.0, east: 110.0 },
southCentral: { south: 13.0, west: 102.0, north: 16.5, east: 110.0 },
south: { south: 8.0, west: 102.0, north: 13.0, east: 110.0 },
};
// ─── CLI ──────────────────────────────────────────────────────────────────
const argv = process.argv.slice(2);
const dryRun = argv.includes('--dry-run');
const chunkArg = argv.find((a) => a.startsWith('--chunk='))?.slice('--chunk='.length);
const levelArg = argv.find((a) => a.startsWith('--level='))?.slice('--level='.length) ?? 'all';
const wantedLevels: number[] =
levelArg === 'all'
? [4, 6, 8]
: levelArg
.split(',')
.map((s) => Number(s.trim()))
.filter((n) => [4, 6, 8].includes(n));
// ─── Slug helper (matches GSO codename style) ──────────────────────────────
function slugify(name: string): string {
return name
.toLowerCase()
.replace(/đ/g, 'd')
.normalize('NFD')
.replace(/[̀-ͯ]/g, '')
.replace(/[^a-z0-9]+/g, '_')
.replace(/^_+|_+$/g, '');
}
// ─── Overpass fetch ────────────────────────────────────────────────────────
interface OverpassResult {
elements: unknown[];
}
async function fetchChunk(level: number, name: string, bbox: BBox): Promise<OverpassResult> {
// `out geom` returns the relation members with inline geometry so we can
// assemble polygons without a second roundtrip. Timeout 300s for level=8.
const query = `
[out:json][timeout:300];
relation
["boundary"="administrative"]
["admin_level"="${level}"]
(${bbox.south},${bbox.west},${bbox.north},${bbox.east});
out body geom;
`;
console.log(` → fetching level=${level} chunk="${name}"…`);
const start = Date.now();
const res = await fetch(OVERPASS_URL, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'User-Agent': 'goodgo-osm-admin-sync/1.0 (https://goodgo.vn)',
},
body: 'data=' + encodeURIComponent(query),
});
if (!res.ok) {
const body = await res.text();
throw new Error(`Overpass returned ${res.status}: ${body.slice(0, 200)}`);
}
const json = (await res.json()) as OverpassResult;
console.log(
` ← level=${level} ${name}: ${json.elements?.length ?? 0} relations in ${(
(Date.now() - start) /
1000
).toFixed(1)}s`,
);
return json;
}
// ─── Per-feature parser ────────────────────────────────────────────────────
interface ParsedAdmin {
level: 4 | 6 | 8;
osmId: bigint;
name: string;
nameEn: string | null;
gsoCode: string | null;
type: string; // "Tỉnh" / "Quận" / "Phường" etc.
geometry: MultiPolygon; // outer rings only
centroid: { lng: number; lat: number };
areaKm2: number;
population: number | null;
rawTags: Record<string, string>;
}
const PROVINCE_TYPE_MAP = (name: string): string =>
/^(Thành phố|TP\.?)\s+(Hà Nội|Hồ Chí Minh|Hải Phòng|Đà Nẵng|Cần Thơ)/i.test(name)
? 'Thành phố Trung ương'
: 'Tỉnh';
const DISTRICT_TYPE_MAP = (name: string): string => {
if (/^Quận/i.test(name)) return 'Quận';
if (/^Huyện/i.test(name)) return 'Huyện';
if (/^Thị xã/i.test(name)) return 'Thị xã';
if (/^Thành phố/i.test(name)) return 'Thành phố thuộc tỉnh';
return 'Quận';
};
const WARD_TYPE_MAP = (name: string): string => {
if (/^Phường/i.test(name)) return 'Phường';
if (/^Xã/i.test(name)) return 'Xã';
if (/^Thị trấn/i.test(name)) return 'Thị trấn';
return 'Xã';
};
function parseFeature(
feat: Feature<Polygon | MultiPolygon>,
level: 4 | 6 | 8,
): ParsedAdmin | null {
const propsRaw = feat.properties as Record<string, unknown> | null;
if (!propsRaw) return null;
// osmtogeojson encodes the prefixed id on `feat.id` ("relation/123") and
// the bare numeric id under `properties.id`. We only kept relations.
const featAny = feat as unknown as { id?: unknown };
const idStr = String(featAny.id ?? propsRaw['id'] ?? '');
if (!idStr.startsWith('relation/')) return null;
const osmId = BigInt(idStr.slice('relation/'.length));
const tagsRaw = propsRaw['tags'];
const tags: Record<string, string> =
tagsRaw && typeof tagsRaw === 'object'
? (tagsRaw as Record<string, string>)
: (propsRaw as Record<string, string>);
const name = tags['name:vi'] ?? tags['name'] ?? null;
if (!name) return null;
// Skip rows without any Latin/Vietnamese letter (cross-border bleed).
if (!/[A-Za-zÀ-ỹ]/.test(name)) return null;
const nameEn = tags['name:en'] ?? null;
const gsoCode =
tags['ref:VN'] ?? tags['gso_code'] ?? tags['ref'] ?? tags['iso_code'] ?? null;
const populationRaw = tags['population'];
const population = populationRaw && /^\d+$/.test(populationRaw) ? Number(populationRaw) : null;
// Normalise to MultiPolygon regardless of source (Polygon → wrap once).
const geom: MultiPolygon =
feat.geometry.type === 'Polygon'
? { type: 'MultiPolygon', coordinates: [feat.geometry.coordinates] }
: feat.geometry;
const c = centroid(feat as Feature);
const [cLng, cLat] = c.geometry.coordinates;
// Geographic gate: drop relations whose centroid sits outside the VN
// mainland polygon (China / Laos / Cambodia bleed across the bbox).
if (!isPointInVietnam(cLng, cLat)) return null;
const areaKm2 = Math.round((area(feat as Feature) / 1_000_000) * 100) / 100;
let type: string;
if (level === 4) type = PROVINCE_TYPE_MAP(name);
else if (level === 6) type = DISTRICT_TYPE_MAP(name);
else type = WARD_TYPE_MAP(name);
return {
level,
osmId,
name,
nameEn,
gsoCode,
type,
geometry: geom,
centroid: { lng: cLng, lat: cLat },
areaKm2,
population,
rawTags: tags,
};
}
// ─── Resolve to existing GSO code or generate a synthetic one ─────────────
async function resolveProvinceCode(p: ParsedAdmin): Promise<string> {
if (p.gsoCode) {
const exists = await prisma.vnProvince.findUnique({ where: { code: p.gsoCode } });
if (exists) return p.gsoCode;
}
// Fallback: lookup by codename slug.
const codename = slugify(p.name);
const byCodename = await prisma.vnProvince.findFirst({ where: { codename } });
if (byCodename) return byCodename.code;
// Brand-new: derive a code from osmId so it's stable.
return `OSM_${p.osmId.toString()}`;
}
async function resolveDistrictCode(p: ParsedAdmin, provinceCode: string): Promise<string> {
if (p.gsoCode) {
const exists = await prisma.vnDistrict.findUnique({ where: { code: p.gsoCode } });
if (exists) return p.gsoCode;
}
const codename = slugify(p.name);
const byCodename = await prisma.vnDistrict.findFirst({
where: { codename, provinceCode },
});
if (byCodename) return byCodename.code;
return `OSM_${p.osmId.toString()}`;
}
async function resolveWardCode(p: ParsedAdmin, districtCode: string): Promise<string> {
if (p.gsoCode) {
const exists = await prisma.vnWard.findUnique({ where: { code: p.gsoCode } });
if (exists) return p.gsoCode;
}
const codename = slugify(p.name);
const byCodename = await prisma.vnWard.findFirst({
where: { codename, districtCode },
});
if (byCodename) return byCodename.code;
return `OSM_${p.osmId.toString()}`;
}
// ─── Upsert helpers — raw SQL because Prisma can't manage geometry ────────
function geomSql(g: MultiPolygon): string {
const json = JSON.stringify(g).replace(/'/g, "''");
return `ST_Multi(ST_GeomFromGeoJSON('${json}'))`;
}
interface UpsertStats {
inserted: number;
updated: number;
skipped: number;
}
async function upsertProvince(p: ParsedAdmin, stats: UpsertStats): Promise<void> {
const code = await resolveProvinceCode(p);
const codename = slugify(p.name);
const existed = await prisma.vnProvince.findUnique({ where: { code }, select: { code: true } });
await prisma.$executeRawUnsafe(
`
INSERT INTO "vn_provinces" (
code, name, "nameEn", type, codename, "osmId",
"areaKm2", population, "lastSyncedAt", "updatedAt", geometry, centroid
) VALUES (
$1, $2, $3, $4, $5, $6::bigint,
$7, $8, NOW(), NOW(),
${geomSql(p.geometry)},
ST_SetSRID(ST_MakePoint($9, $10), 4326)
)
ON CONFLICT (code) DO UPDATE SET
name = EXCLUDED.name,
"nameEn" = EXCLUDED."nameEn",
type = EXCLUDED.type,
"osmId" = EXCLUDED."osmId",
"areaKm2" = EXCLUDED."areaKm2",
population = COALESCE(EXCLUDED.population, "vn_provinces".population),
"lastSyncedAt" = NOW(),
"updatedAt" = NOW(),
geometry = EXCLUDED.geometry,
centroid = EXCLUDED.centroid
`,
code,
p.name,
p.nameEn,
p.type,
codename,
p.osmId.toString(),
p.areaKm2,
p.population,
p.centroid.lng,
p.centroid.lat,
);
if (existed) stats.updated++;
else stats.inserted++;
}
async function upsertDistrict(p: ParsedAdmin, stats: UpsertStats): Promise<void> {
// Find which province contains this district by ST_Within against existing
// synced province polygons. Falls back to province with largest overlap.
const provinceMatch = await prisma.$queryRawUnsafe<{ code: string }[]>(
`SELECT code FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1`,
p.centroid.lng,
p.centroid.lat,
);
if (provinceMatch.length === 0) {
stats.skipped++;
return; // Cannot place district until provinces are synced first.
}
const provinceCode = provinceMatch[0]!.code;
const code = await resolveDistrictCode(p, provinceCode);
const codename = slugify(p.name);
const existed = await prisma.vnDistrict.findUnique({ where: { code }, select: { code: true } });
await prisma.$executeRawUnsafe(
`
INSERT INTO "vn_districts" (
code, "provinceCode", name, "nameEn", type, codename, "osmId",
"areaKm2", population, "lastSyncedAt", "updatedAt", geometry, centroid
) VALUES (
$1, $2, $3, $4, $5, $6, $7::bigint,
$8, $9, NOW(), NOW(),
${geomSql(p.geometry)},
ST_SetSRID(ST_MakePoint($10, $11), 4326)
)
ON CONFLICT (code) DO UPDATE SET
"provinceCode" = EXCLUDED."provinceCode",
name = EXCLUDED.name,
"nameEn" = EXCLUDED."nameEn",
type = EXCLUDED.type,
"osmId" = EXCLUDED."osmId",
"areaKm2" = EXCLUDED."areaKm2",
population = COALESCE(EXCLUDED.population, "vn_districts".population),
"lastSyncedAt" = NOW(),
"updatedAt" = NOW(),
geometry = EXCLUDED.geometry,
centroid = EXCLUDED.centroid
`,
code,
provinceCode,
p.name,
p.nameEn,
p.type,
codename,
p.osmId.toString(),
p.areaKm2,
p.population,
p.centroid.lng,
p.centroid.lat,
);
if (existed) stats.updated++;
else stats.inserted++;
}
async function upsertWard(p: ParsedAdmin, stats: UpsertStats): Promise<void> {
const districtMatch = await prisma.$queryRawUnsafe<{ code: string }[]>(
`SELECT code FROM "vn_districts"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1`,
p.centroid.lng,
p.centroid.lat,
);
if (districtMatch.length === 0) {
stats.skipped++;
return;
}
const districtCode = districtMatch[0]!.code;
const code = await resolveWardCode(p, districtCode);
const codename = slugify(p.name);
const existed = await prisma.vnWard.findUnique({ where: { code }, select: { code: true } });
await prisma.$executeRawUnsafe(
`
INSERT INTO "vn_wards" (
code, "districtCode", name, "nameEn", type, codename, "osmId",
"areaKm2", population, "lastSyncedAt", "updatedAt", geometry, centroid
) VALUES (
$1, $2, $3, $4, $5, $6, $7::bigint,
$8, $9, NOW(), NOW(),
${geomSql(p.geometry)},
ST_SetSRID(ST_MakePoint($10, $11), 4326)
)
ON CONFLICT (code) DO UPDATE SET
"districtCode" = EXCLUDED."districtCode",
name = EXCLUDED.name,
"nameEn" = EXCLUDED."nameEn",
type = EXCLUDED.type,
"osmId" = EXCLUDED."osmId",
"areaKm2" = EXCLUDED."areaKm2",
population = COALESCE(EXCLUDED.population, "vn_wards".population),
"lastSyncedAt" = NOW(),
"updatedAt" = NOW(),
geometry = EXCLUDED.geometry,
centroid = EXCLUDED.centroid
`,
code,
districtCode,
p.name,
p.nameEn,
p.type,
codename,
p.osmId.toString(),
p.areaKm2,
p.population,
p.centroid.lng,
p.centroid.lat,
);
if (existed) stats.updated++;
else stats.inserted++;
}
// ─── Main ─────────────────────────────────────────────────────────────────
async function processChunk(
level: 4 | 6 | 8,
chunkName: string,
bbox: BBox,
): Promise<UpsertStats> {
const stats: UpsertStats = { inserted: 0, updated: 0, skipped: 0 };
const result = await fetchChunk(level, chunkName, bbox);
const fc = osmtogeojson(result, { flatProperties: false });
const features = (fc.features as Feature<Polygon | MultiPolygon>[]).filter(
(f) => f.geometry?.type === 'Polygon' || f.geometry?.type === 'MultiPolygon',
);
for (const feat of features) {
const parsed = parseFeature(feat, level);
if (!parsed) continue;
if (dryRun) {
stats.inserted++;
continue;
}
try {
if (level === 4) await upsertProvince(parsed, stats);
else if (level === 6) await upsertDistrict(parsed, stats);
else await upsertWard(parsed, stats);
} catch (err) {
console.error(`${parsed.name}: ${err instanceof Error ? err.message : err}`);
stats.skipped++;
}
}
console.log(
` ✓ level=${level} ${chunkName}: inserted=${stats.inserted} updated=${stats.updated} skipped=${stats.skipped}`,
);
return stats;
}
async function main(): Promise<void> {
console.log('🌏 OSM admin boundaries sync starting');
console.log(` levels: ${wantedLevels.join(',')}, chunks: ${chunkArg ?? 'all'}, dryRun=${dryRun}`);
const chunks = chunkArg
? { [chunkArg]: CHUNKS[chunkArg]! }
: CHUNKS;
const totals: Record<number, UpsertStats> = {
4: { inserted: 0, updated: 0, skipped: 0 },
6: { inserted: 0, updated: 0, skipped: 0 },
8: { inserted: 0, updated: 0, skipped: 0 },
};
// ALWAYS process levels in order 4 → 6 → 8, because 6 needs province
// polygons in the DB to assign provinceCode (and 8 needs districts).
for (const level of wantedLevels.sort() as (4 | 6 | 8)[]) {
console.log(`\n=== Level ${level} ===`);
for (const [name, bbox] of Object.entries(chunks)) {
try {
const s = await processChunk(level, name, bbox);
totals[level]!.inserted += s.inserted;
totals[level]!.updated += s.updated;
totals[level]!.skipped += s.skipped;
} catch (err) {
console.error(` ✗ chunk ${name} (level ${level}) failed:`, err);
}
}
}
console.log('\n📊 Totals');
for (const lvl of wantedLevels) {
const t = totals[lvl]!;
console.log(
` level=${lvl}: inserted=${t.inserted} updated=${t.updated} skipped=${t.skipped}`,
);
}
}
main()
.catch((err) => {
console.error(err);
process.exitCode = 1;
})
.finally(async () => {
await prisma.$disconnect();
await pool.end();
});

View File

@@ -215,9 +215,11 @@ function parseFeature(
if (!isPointInVietnam(cLng, cLat)) return null;
// Province resolution: prefer explicit OSM tags, then fall back to a
// nearest-centroid lookup against our 63-province table. The fallback
// catches the (very common) case where Vietnamese landuse polygons have
// no addr:* tags at all.
// nearest-centroid lookup against our 63-province table. The actual DB
// upsert step (`upsertFeature`) replaces this with a precise PostGIS
// ST_Contains lookup against `vn_provinces.geometry` once those polygons
// are synced — this is just the bootstrap value used when the polygon
// table is empty.
const province =
VN_PROVINCE_HINTS.map((k) => tags[k]).find(Boolean) ??
tags['addr:city'] ??
@@ -271,6 +273,33 @@ async function upsertFeature(
return;
}
// Override the heuristic province with a precise PostGIS lookup against
// the OSM-sourced admin polygons (when synced). Falls back to the
// nearest-centroid value already on `parsed.province` if the polygon
// table doesn't yet cover that area.
const adminMatch = await prisma.$queryRawUnsafe<
{ provinceName: string | null; districtName: string | null }[]
>(
`WITH p AS (
SELECT code, name FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
)
SELECT
(SELECT name FROM p) AS "provinceName",
(SELECT d.name FROM "vn_districts" d JOIN p ON p.code = d."provinceCode"
WHERE d.geometry IS NOT NULL
AND ST_Contains(d.geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1) AS "districtName"`,
parsed.centroid.lng,
parsed.centroid.lat,
);
const resolvedProvince = adminMatch[0]?.provinceName ?? parsed.province;
const resolvedDistrict = adminMatch[0]?.districtName ?? parsed.district;
parsed.province = resolvedProvince;
if (!parsed.district) parsed.district = resolvedDistrict ?? '';
const region = guessRegion(parsed.centroid.lat);
const slug = slugify(parsed.name, parsed.osmId.toString());

400
scripts/sync-osm-poi.ts Normal file
View File

@@ -0,0 +1,400 @@
/**
* Sync OSM points-of-interest into the `Poi` table.
*
* Usage:
* NODE_OPTIONS="-r dotenv/config" DOTENV_CONFIG_PATH=.env \
* pnpm tsx scripts/sync-osm-poi.ts \
* [--category=school,hospital,...|all] [--chunk=NAME] [--dry-run]
*
* What it does:
* 1. For each requested category, queries Overpass for the matching
* node/way/relation across the 4 Vietnam chunks.
* 2. Filters out non-Vietnam centroids (cross-border bleed) and rows
* without any Latin/Vietnamese letters in the name.
* 3. Resolves provinceCode/districtCode/wardCode via PostGIS lookup
* against `vn_provinces` / `vn_districts` / `vn_wards` (assumes
* Phase 0 boundary sync ran first).
* 4. Upserts on `osmId`, honouring `osmLocked` + `lockedFields`.
*/
import 'dotenv/config';
import area from '@turf/area';
import centroid from '@turf/centroid';
import { createId } from '@paralleldrive/cuid2';
import { PrismaPg } from '@prisma/adapter-pg';
import { type Prisma, PrismaClient } from '@prisma/client';
import type { Feature, MultiPolygon, Polygon, Point } from 'geojson';
import osmtogeojson from 'osmtogeojson';
import pg from 'pg';
import { isPointInVietnam } from './data/vn-country-polygon';
const pool = new pg.Pool({ connectionString: process.env['DATABASE_URL'] });
const adapter = new PrismaPg(pool);
const prisma = new PrismaClient({ adapter });
const OVERPASS_URL =
process.env['OVERPASS_URL'] ?? 'https://overpass-api.de/api/interpreter';
interface BBox {
south: number;
west: number;
north: number;
east: number;
}
const CHUNKS: Record<string, BBox> = {
north: { south: 19.0, west: 102.0, north: 23.5, east: 110.0 },
northCentral: { south: 16.5, west: 102.0, north: 19.0, east: 110.0 },
southCentral: { south: 13.0, west: 102.0, north: 16.5, east: 110.0 },
south: { south: 8.0, west: 102.0, north: 13.0, east: 110.0 },
};
type PoiCategoryKey =
| 'SCHOOL_PRIMARY' | 'SCHOOL_SECONDARY' | 'UNIVERSITY'
| 'HOSPITAL' | 'CLINIC' | 'PHARMACY'
| 'MARKET' | 'SUPERMARKET' | 'MALL' | 'CONVENIENCE'
| 'BANK' | 'ATM'
| 'PARK'
| 'GAS_STATION' | 'POLICE' | 'POST_OFFICE'
| 'METRO_STATION' | 'RAILWAY_STATION' | 'BUS_STATION' | 'AIRPORT';
/**
* For each category, the Overpass selector. We query node/way/relation
* to catch both single points and named building polygons.
*/
const CATEGORY_QUERIES: Record<PoiCategoryKey, string> = {
// ── Education ─────────────────────────────────────────────────────────
SCHOOL_PRIMARY: '["amenity"="school"]["isced:level"~"^(primary|0|1)$"]',
SCHOOL_SECONDARY: '["amenity"="school"]["isced:level"~"^(secondary|2|3)$"]',
UNIVERSITY: '["amenity"~"^(university|college)$"]',
// ── Health ────────────────────────────────────────────────────────────
HOSPITAL: '["amenity"="hospital"]',
CLINIC: '["amenity"="clinic"]',
PHARMACY: '["amenity"="pharmacy"]',
// ── Commerce ──────────────────────────────────────────────────────────
MARKET: '["amenity"="marketplace"]',
SUPERMARKET: '["shop"="supermarket"]',
MALL: '["shop"="mall"]',
CONVENIENCE: '["shop"="convenience"]',
// ── Finance ───────────────────────────────────────────────────────────
BANK: '["amenity"="bank"]',
ATM: '["amenity"="atm"]',
// ── Recreation / Services ────────────────────────────────────────────
PARK: '["leisure"="park"]',
GAS_STATION: '["amenity"="fuel"]',
POLICE: '["amenity"="police"]',
POST_OFFICE: '["amenity"="post_office"]',
// ── Transport (stations / airports — lines live in TransportLine) ────
METRO_STATION: '["railway"="station"]["station"="subway"]',
RAILWAY_STATION: '["railway"="station"]["station"!="subway"]',
BUS_STATION: '["amenity"="bus_station"]',
AIRPORT: '["aeroway"="aerodrome"]["aerodrome:type"~"international|public"]',
};
const ALL_CATEGORIES: PoiCategoryKey[] = Object.keys(CATEGORY_QUERIES) as PoiCategoryKey[];
// ─── CLI ───────────────────────────────────────────────────────────────────
const argv = process.argv.slice(2);
const dryRun = argv.includes('--dry-run');
const chunkArg = argv.find((a) => a.startsWith('--chunk='))?.slice('--chunk='.length);
const categoryArg = argv.find((a) => a.startsWith('--category='))?.slice('--category='.length) ?? 'all';
const wantedCategories: PoiCategoryKey[] =
categoryArg === 'all'
? ALL_CATEGORIES
: (categoryArg
.split(',')
.map((s) => s.trim().toUpperCase())
.filter((s): s is PoiCategoryKey => ALL_CATEGORIES.includes(s as PoiCategoryKey)) as PoiCategoryKey[]);
if (wantedCategories.length === 0) {
console.error(`No valid categories. Available: ${ALL_CATEGORIES.join(', ')}`);
process.exit(1);
}
interface OverpassResult {
elements: unknown[];
}
async function fetchChunk(
category: PoiCategoryKey,
chunkName: string,
bbox: BBox,
): Promise<OverpassResult> {
const sel = CATEGORY_QUERIES[category];
const query = `
[out:json][timeout:180];
(
node${sel}(${bbox.south},${bbox.west},${bbox.north},${bbox.east});
way${sel}(${bbox.south},${bbox.west},${bbox.north},${bbox.east});
relation${sel}(${bbox.south},${bbox.west},${bbox.north},${bbox.east});
);
out body geom;
`;
const t0 = Date.now();
console.log(`${category} ${chunkName}`);
const res = await fetch(OVERPASS_URL, {
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'User-Agent': 'goodgo-osm-poi-sync/1.0 (https://goodgo.vn)',
},
body: 'data=' + encodeURIComponent(query),
});
if (!res.ok) {
const body = await res.text();
throw new Error(`Overpass ${res.status}: ${body.slice(0, 200)}`);
}
const json = (await res.json()) as OverpassResult;
console.log(
`${category} ${chunkName}: ${json.elements?.length ?? 0} elements in ${(
(Date.now() - t0) /
1000
).toFixed(1)}s`,
);
return json;
}
interface ParsedPoi {
category: PoiCategoryKey;
osmId: bigint;
osmType: 'NODE' | 'WAY' | 'RELATION';
name: string;
nameEn: string | null;
centroid: { lng: number; lat: number };
address: string | null;
tags: Record<string, string>;
}
function parseFeature(
feat: Feature<Polygon | MultiPolygon | Point>,
category: PoiCategoryKey,
): ParsedPoi | null {
const featAny = feat as unknown as { id?: unknown };
const idStr = String(featAny.id ?? '');
const slashIdx = idStr.indexOf('/');
if (slashIdx < 0) return null;
const typeStr = idStr.slice(0, slashIdx).toUpperCase();
if (typeStr !== 'NODE' && typeStr !== 'WAY' && typeStr !== 'RELATION') return null;
const osmType = typeStr as 'NODE' | 'WAY' | 'RELATION';
const osmId = BigInt(idStr.slice(slashIdx + 1));
const propsRaw = (feat.properties ?? {}) as Record<string, unknown>;
const tagsRaw = propsRaw['tags'];
const tags: Record<string, string> =
tagsRaw && typeof tagsRaw === 'object'
? (tagsRaw as Record<string, string>)
: (propsRaw as Record<string, string>);
const name = tags['name:vi'] ?? tags['name'] ?? null;
// Skip purely unnamed POIs (very common for shop=convenience etc.)
if (!name) return null;
// Skip rows without Latin/Vietnamese letters (cross-border bleed).
if (!/[A-Za-zÀ-ỹ]/.test(name)) return null;
let cLng: number;
let cLat: number;
if (feat.geometry.type === 'Point') {
[cLng, cLat] = feat.geometry.coordinates;
} else {
const c = centroid(feat as Feature);
[cLng, cLat] = c.geometry.coordinates;
}
if (!isPointInVietnam(cLng, cLat)) return null;
const address =
tags['addr:full'] ??
[tags['addr:housenumber'], tags['addr:street']].filter(Boolean).join(' ') ??
null;
return {
category,
osmId,
osmType,
name,
nameEn: tags['name:en'] ?? null,
centroid: { lng: cLng, lat: cLat },
address: address || null,
tags,
};
}
interface UpsertStats {
inserted: number;
updated: number;
locked: number;
skipped: number;
}
async function upsertPoi(parsed: ParsedPoi, stats: UpsertStats): Promise<void> {
const existing = await prisma.poi.findUnique({
where: { osmId: parsed.osmId },
select: { id: true, osmLocked: true, lockedFields: true },
});
if (existing?.osmLocked) {
stats.locked++;
return;
}
// Resolve admin codes from the polygon tables.
const admin = await prisma.$queryRawUnsafe<
{ provinceCode: string | null; districtCode: string | null; wardCode: string | null }[]
>(
`WITH p AS (
SELECT code FROM "vn_provinces"
WHERE geometry IS NOT NULL
AND ST_Contains(geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
),
d AS (
SELECT d.code
FROM "vn_districts" d JOIN p ON p.code = d."provinceCode"
WHERE d.geometry IS NOT NULL
AND ST_Contains(d.geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1
)
SELECT
(SELECT code FROM p) AS "provinceCode",
(SELECT code FROM d) AS "districtCode",
(SELECT w.code FROM "vn_wards" w JOIN d ON d.code = w."districtCode"
WHERE w.geometry IS NOT NULL
AND ST_Contains(w.geometry, ST_SetSRID(ST_MakePoint($1, $2), 4326))
LIMIT 1) AS "wardCode"`,
parsed.centroid.lng,
parsed.centroid.lat,
);
const provinceCode = admin[0]?.provinceCode ?? null;
const districtCode = admin[0]?.districtCode ?? null;
const wardCode = admin[0]?.wardCode ?? null;
if (!existing) {
const cuid = createId();
await prisma.$executeRawUnsafe(
`
INSERT INTO "Poi" (
id, category, name, "nameEn", location, address,
"provinceCode", "districtCode", "wardCode",
"osmId", "osmType", "osmTags",
"dataSource", "isPublic", "lastSyncedAt", "createdAt", "updatedAt"
) VALUES (
$1, $2::"PoiCategory", $3, $4,
ST_SetSRID(ST_MakePoint($5, $6), 4326), $7,
$8, $9, $10,
$11::bigint, $12::"OsmType", $13::jsonb,
'OSM'::"OsmDataSource", true, NOW(), NOW(), NOW()
)
`,
cuid,
parsed.category,
parsed.name,
parsed.nameEn,
parsed.centroid.lng,
parsed.centroid.lat,
parsed.address,
provinceCode,
districtCode,
wardCode,
parsed.osmId.toString(),
parsed.osmType,
JSON.stringify(parsed.tags),
);
stats.inserted++;
} else {
// Update — respect lockedFields list.
const locked = new Set(existing.lockedFields ?? []);
const data: Prisma.PoiUpdateInput = {
lastSyncedAt: new Date(),
osmTags: JSON.stringify(parsed.tags) as unknown as Prisma.InputJsonValue,
};
if (!locked.has('name')) data.name = parsed.name;
if (!locked.has('nameEn')) data.nameEn = parsed.nameEn;
if (!locked.has('address')) data.address = parsed.address;
if (!locked.has('provinceCode')) data.provinceCode = provinceCode;
if (!locked.has('districtCode')) data.districtCode = districtCode;
if (!locked.has('wardCode')) data.wardCode = wardCode;
await prisma.poi.update({ where: { id: existing.id }, data });
// Location update via raw SQL (Prisma can't write `Unsupported` columns).
if (!locked.has('location')) {
await prisma.$executeRawUnsafe(
`UPDATE "Poi" SET location = ST_SetSRID(ST_MakePoint($1, $2), 4326) WHERE id = $3`,
parsed.centroid.lng,
parsed.centroid.lat,
existing.id,
);
}
stats.updated++;
}
}
async function processCategoryChunk(
category: PoiCategoryKey,
chunkName: string,
bbox: BBox,
stats: UpsertStats,
): Promise<void> {
const result = await fetchChunk(category, chunkName, bbox);
const fc = osmtogeojson(result, { flatProperties: false });
const features = (fc.features as Feature<Polygon | MultiPolygon | Point>[]).filter(
(f) =>
f.geometry?.type === 'Point' ||
f.geometry?.type === 'Polygon' ||
f.geometry?.type === 'MultiPolygon',
);
for (const feat of features) {
const parsed = parseFeature(feat, category);
if (!parsed) continue;
if (dryRun) {
stats.inserted++;
continue;
}
try {
await upsertPoi(parsed, stats);
} catch (err) {
console.error(
`${category} ${parsed.name}: ${err instanceof Error ? err.message : err}`,
);
stats.skipped++;
}
}
}
async function main(): Promise<void> {
console.log(`📍 OSM POI sync: categories=${wantedCategories.join(',')} dryRun=${dryRun}`);
const chunks = chunkArg
? { [chunkArg]: CHUNKS[chunkArg]! }
: CHUNKS;
const totals: Record<string, UpsertStats> = {};
for (const cat of wantedCategories) {
console.log(`\n=== ${cat} ===`);
const s: UpsertStats = { inserted: 0, updated: 0, locked: 0, skipped: 0 };
for (const [name, bbox] of Object.entries(chunks)) {
try {
await processCategoryChunk(cat, name, bbox, s);
} catch (err) {
console.error(` ✗ chunk ${name} (${cat}) failed:`, err);
}
}
totals[cat] = s;
console.log(
`${cat}: inserted=${s.inserted} updated=${s.updated} locked=${s.locked} skipped=${s.skipped}`,
);
}
console.log('\n📊 Totals:');
for (const cat of wantedCategories) {
const s = totals[cat]!;
console.log(
` ${cat.padEnd(20)} inserted=${s.inserted} updated=${s.updated} locked=${s.locked} skipped=${s.skipped}`,
);
}
}
main()
.catch((err) => {
console.error(err);
process.exitCode = 1;
})
.finally(async () => {
await prisma.$disconnect();
await pool.end();
});