diff --git a/infra/servers/minio.md b/infra/servers/minio.md new file mode 100644 index 00000000..40438264 --- /dev/null +++ b/infra/servers/minio.md @@ -0,0 +1,105 @@ +# MinIO Server Installation Guide + +## System Information +- **OS**: Ubuntu 24.04.3 LTS (Noble Numbat) +- **Architecture**: x86_64 +- **Public IP**: 167.114.174.113 + +## Installation Steps + +### 1. Download Binary +Download the latest MinIO binary and install it to `/usr/local/bin`. + +```bash +wget https://dl.min.io/server/minio/release/linux-amd64/minio +chmod +x minio +sudo mv minio /usr/local/bin/ +``` + +### 2. User & Group +Create a dedicated system user for MinIO. + +```bash +sudo useradd -r minio-user -s /sbin/nologin +``` + +### 3. Data Directory +Create the local storage directory and assign ownership. + +```bash +sudo mkdir /data +sudo chown minio-user:minio-user /data +``` + +### 4. License File +Create the license file at `/opt/minio/minio.license`. + +```bash +sudo mkdir -p /opt/minio +echo "eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9.eyJhaWQiOjAsImNhcCI6MCwiaWF0IjoxLjc2ODI0NDQ2MjMwMzI4ODQ3ZTksImlzcyI6InN1Ym5ldEBtaW4uaW8iLCJsaWQiOiJkYjQ5ZjU4Ny0yZmRkLTQ2NzEtYjI2Yy1kOTVlOTJkNDRhYTYiLCJvcmciOiIiLCJwbGFuIjoiRlJFRSIsInN1YiI6ImhvbmdvY2hhaTEwQGljbG91ZC5jb20iLCJ0cmlhbCI6ZmFsc2V9.zPT5238rxKwdQvMBuztjjtCkC1TKG0tkbNmTBwOVWRp7rIyJfmtZsIN3uu-se4WMl5Zk0A6t9C0LDgu3gXHjBVsBG9aZuonQdKCknWYlM7czyWQx3GSIr0rDq0zMkZOM" | sudo tee /opt/minio/minio.license +sudo chown -R minio-user:minio-user /opt/minio +``` + +## Configuration + +### Environment File: `/etc/default/minio` + +```bash +# Volume to be used for MinIO Server. +MINIO_VOLUMES="/data" + +# MinIO Server options. +MINIO_OPTS="--console-address :9001" + +# Root user for the server. +MINIO_ROOT_USER=minioadmin + +# Root secret for the server. +MINIO_ROOT_PASSWORD=Velik@2026 + +# MinIO License +# MinIO License +MINIO_LICENSE="/opt/minio/minio.license" +``` + +### Systemd Service: `/etc/systemd/system/minio.service` + +```ini +[Unit] +Description=MinIO +Documentation=https://docs.min.io +Wants=network-online.target +After=network-online.target +AssertFileIsExecutable=/usr/local/bin/minio + +[Service] +WorkingDirectory=/usr/local +User=minio-user +Group=minio-user +ProtectProc=invisible +EnvironmentFile=/etc/default/minio +ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi" +ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES +Restart=always +LimitNOFILE=65536 +TasksMax=infinity +TimeoutStopSec=infinity +SendSIGKILL=no + +[Install] +WantedBy=multi-user.target +``` + +## Management Commands + +- **Start Service**: `sudo systemctl start minio` +- **Stop Service**: `sudo systemctl stop minio` +- **Restart Service**: `sudo systemctl restart minio` +- **Check Status**: `sudo systemctl status minio` +- **View Logs**: `journalctl -u minio` + +## Access Details +- **API URL**: `http://167.114.174.113:9000` +- **Console URL**: `http://167.114.174.113:9001` +- **Username**: `minioadmin` +- **Password**: `Velik@2026` diff --git a/infra/servers/mssql.md b/infra/servers/mssql.md new file mode 100644 index 00000000..62574f29 --- /dev/null +++ b/infra/servers/mssql.md @@ -0,0 +1,64 @@ +# Hướng Dẫn Sử Dụng MSSQL Server 2025 (Ubuntu 24.04) + +## Thông Tin Cài Đặt +- **Phiên bản**: Microsoft SQL Server 2025 (RTM) - Enterprise Developer Edition +- **Giấy phép**: Miễn phí trọn đời (Free Forever) cho môi trường **Development & Test**. Không dùng cho Production. +- **Hệ điều hành**: Ubuntu 24.04 +- **Tài khoản mặc định**: `SA` +- **Mật khẩu**: `Velik@2026` + +## Kết Nối Database +Sử dụng `sqlcmd` để kết nối: + +```bash +/opt/mssql-tools18/bin/sqlcmd -S localhost -U SA -P 'Velik@2026' -C +``` + +Để chạy một câu lệnh SQL đơn giản (ví dụ: kiểm tra phiên bản): +```bash +/opt/mssql-tools18/bin/sqlcmd -S localhost -U SA -P 'Velik@2026' -C -Q "SELECT @@VERSION" +``` + +> **Lưu ý**: Cờ `-C` là bắt buộc để tin cậy chứng chỉ server (Trust Server Certificate) vì mặc định MSSQL tạo chứng chỉ tự ký. + +## Quản Lý Service +Các lệnh systemd để quản lý MSSQL Server: + +- **Kiểm tra trạng thái**: + ```bash + sudo systemctl status mssql-server + ``` +- **Khởi động**: + ```bash + sudo systemctl start mssql-server + ``` +- **Dừng**: + ```bash + sudo systemctl stop mssql-server + ``` +- **Khởi động lại**: + ```bash + sudo systemctl restart mssql-server + ``` + +## Cấu Hình & Thư Mục +- **File cấu hình**: `/var/opt/mssql/mssql.conf` +- **Thư mục dữ liệu (mặc định)**: `/var/opt/mssql/data` +- **Thư mục log lỗi**: `/var/opt/mssql/log` +- **Công cụ cấu hình**: `/opt/mssql/bin/mssql-conf` + +### Thay Đổi Password SA +Nếu cần đặt lại mật khẩu SA: +```bash +sudo systemctl stop mssql-server +sudo MSSQL_SA_PASSWORD='NewStrongPassword!' /opt/mssql/bin/mssql-conf -n set-sa-password +sudo systemctl start mssql-server +``` + +### Thay Đổi Phiên Bản (Edition) +```bash +sudo systemctl stop mssql-server +sudo /opt/mssql/bin/mssql-conf set-edition +# Chọn phiên bản mong muốn (ví dụ: Developer) +sudo systemctl start mssql-server +``` diff --git a/infra/servers/psql.md b/infra/servers/psql.md new file mode 100644 index 00000000..9852aaa6 --- /dev/null +++ b/infra/servers/psql.md @@ -0,0 +1,47 @@ +# Hướng dẫn sử dụng PostgreSQL + +## Phiên bản +PostgreSQL 18.1 + +## Thông tin tài khoản +Dưới đây là thông tin tài khoản và mật khẩu đã được cấu hình: + +| Tài khoản | Mật khẩu | Quyền hạn | +| :--- | :--- | :--- | +| `ubuntu` | `Velik@2026` | Superuser | +| `postgres` | `Velik@2026` | Superuser | + +## Cách kết nối + +### 1. Kết nối bằng tài khoản `ubuntu` (Mặc định) +Bạn đang đăng nhập vào server bằng user `ubuntu`, nên có thể kết nối trực tiếp: + +```bash +psql +``` + +### 2. Kết nối bằng tài khoản `postgres` +Để kết nối với tư cách là user hệ thống `postgres`: + +```bash +sudo -u postgres psql +``` + +Hoặc kết nối qua giao thức mạng (yêu cầu mật khẩu): + +```bash +psql -U postgres -h localhost +``` + +### 3. Kết nối từ xa (Remote Access) +Server đã được cấu hình để cho phép kết nối từ xa từ bất kỳ IP nào. + +- **Public IP**: `167.114.174.113` +- **Port**: `5432` +- **Lệnh kết nối**: + +```bash +psql -h 167.114.174.113 -U postgres +``` + +> **Lưu ý**: Mật khẩu `Velik@2026` có chứa ký tự `@` là hoàn toàn hợp lệ. diff --git a/infra/servers/redis.md b/infra/servers/redis.md new file mode 100644 index 00000000..2d89c696 --- /dev/null +++ b/infra/servers/redis.md @@ -0,0 +1,54 @@ +# Redis Installation Guide for Ubuntu 24.04 + +This guide documents the installation of the latest Redis server using the official Redis repository. + +## Installation Steps + +1. **Install Prerequisites**: + ```bash + sudo apt-get install -y lsb-release curl gpg + ``` + +2. **Add GPG Key and Repository**: + ```bash + curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg + sudo chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg + echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list + ``` + +3. **Install Redis Server**: + ```bash + sudo apt-get update + sudo apt-get install -y redis-server + ``` + +## Configuration + +### Security and Network + +- **Public IP Binding**: Enabled (Bound to `0.0.0.0`) +- **Password Protection**: Enabled +- **Password**: `Velik@2026` +- **Port**: 6379 + +## Verification + +To verify the installation and connection: + +```bash +# Check service status +sudo systemctl status redis-server + +# Test connection with password +redis-cli -a Velik@2026 ping +# Expected output: PONG + +# Test remote connection (replace IP with 167.114.174.113) +redis-cli -h 167.114.174.113 -a Velik@2026 ping +``` + +## Current Status + +- **Version**: 8.4.0 +- **Status**: Installed and Active +- **Connection**: Publicly accessible on port 6379 diff --git a/services/storage-service-net/docs/en/ARCHITECTURE.md b/services/storage-service-net/docs/en/ARCHITECTURE.md index b3bf5391..29ed0046 100644 --- a/services/storage-service-net/docs/en/ARCHITECTURE.md +++ b/services/storage-service-net/docs/en/ARCHITECTURE.md @@ -87,12 +87,80 @@ Application entry point and CQRS implementation: | Component | Purpose | |-----------|---------| -| **FilesController** | File CRUD endpoints | +| **FilesController** | File CRUD endpoints (legacy proxy upload) | +| **SignedUrlController** | Direct upload endpoints (recommended) | | **QuotaController** | User quota endpoints | -| **UploadFileCommand** | Handle file uploads | +| **SignUploadCommand** | Generate pre-signed upload URLs | +| **ConfirmUploadCommand** | Confirm direct uploads and save metadata | +| **UploadFileCommand** | Handle proxy file uploads (legacy) | | **DeleteFileCommand** | Handle file deletions | | **Query Handlers** | Handle read operations | +## Direct Upload Architecture (Recommended) + +For systems with millions of users, Direct Client Upload pattern is recommended over proxy upload. + +### Upload Patterns Comparison + +| Aspect | Proxy Upload (Legacy) | Direct Upload (Recommended) | +|--------|----------------------|----------------------------| +| **Throughput** | ~100-500/sec | ~10,000+/sec | +| **Memory per request** | 100MB (file size) | ~10KB (metadata only) | +| **Latency (100MB file)** | 30-60s | 10-20s | +| **Backend load** | High | Minimal | + +### Direct Upload Flow + +```mermaid +sequenceDiagram + participant Client + participant Storage_Service as Storage Service + participant MinIO + + rect rgb(200, 230, 200) + Note over Client,Storage_Service: 1. Request Upload URL (lightweight) + Client->>Storage_Service: POST /api/v1/storage/sign-upload + Storage_Service->>Storage_Service: Validate JWT, Check Quota + Storage_Service-->>Client: Pre-signed PUT URL + ObjectKey + end + + rect rgb(200, 200, 230) + Note over Client,MinIO: 2. Direct Upload (bypasses backend) + Client->>MinIO: PUT file binary to Pre-signed URL + MinIO-->>Client: 200 OK + end + + rect rgb(230, 230, 200) + Note over Client,Storage_Service: 3. Confirm Upload (lightweight) + Client->>Storage_Service: POST /api/v1/storage/confirm-upload + Storage_Service->>MinIO: Verify file exists + Storage_Service->>Storage_Service: Save metadata, Update quota + Storage_Service-->>Client: File metadata + end +``` + +### Path-based Access Control + +Files are organized with access level prefixes: + +``` +storage-bucket/ +├── public/{userId}/{date}/{fileId}_{filename} → Publicly accessible +├── private/{userId}/{date}/{fileId}_{filename} → Requires pre-signed URL +└── shared/{userId}/{date}/{fileId}_{filename} → Access controlled by rules +``` + +### Direct Upload Components + +| Component | Purpose | +|-----------|---------| +| **SignUploadCommand** | Validate quota, generate object key with path prefix, create pre-signed PUT URL | +| **SignUploadCommandHandler** | Handle sign-upload requests | +| **ConfirmUploadCommand** | Verify file exists, save metadata, update quota | +| **ConfirmUploadCommandHandler** | Handle confirm-upload with idempotency | +| **SignedUrlController** | `/sign-upload` and `/confirm-upload` endpoints | + + ## Storage Provider Architecture ```mermaid @@ -229,15 +297,30 @@ erDiagram ## API Endpoints +### Files (Legacy - Proxy Upload) + | Method | Endpoint | Description | |--------|----------|-------------| -| `POST` | `/api/v1/files/upload` | Upload file (max 100MB) | +| `POST` | `/api/v1/files/upload` | Upload file via backend (max 100MB) | | `GET` | `/api/v1/files` | List user files with pagination | | `GET` | `/api/v1/files/{id}` | Get file metadata | | `GET` | `/api/v1/files/{id}/download-url` | Get pre-signed download URL | | `DELETE` | `/api/v1/files/{id}` | Delete file (soft delete) | + +### Direct Upload (Recommended) + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/v1/storage/sign-upload` | Get pre-signed PUT URL for direct upload | +| `POST` | `/api/v1/storage/confirm-upload` | Confirm upload and save metadata | + +### Quota + +| Method | Endpoint | Description | +|--------|----------|-------------| | `GET` | `/api/v1/quota` | Get user storage quota | + ## Health Checks ```mermaid diff --git a/services/storage-service-net/docs/en/README.md b/services/storage-service-net/docs/en/README.md index dcfd56f5..5f0910cc 100644 --- a/services/storage-service-net/docs/en/README.md +++ b/services/storage-service-net/docs/en/README.md @@ -85,23 +85,148 @@ dotnet run --project src/StorageService.API ## API Endpoints -### Files +### Files (Legacy - Proxy Upload) | Method | Endpoint | Description | |--------|----------|-------------| -| `POST` | `/api/v1/files/upload` | Upload a file (max 100MB) | +| `POST` | `/api/v1/files/upload` | Upload a file via backend (max 100MB) | | `GET` | `/api/v1/files` | List user's files with pagination | | `GET` | `/api/v1/files/{id}` | Get file metadata by ID | | `GET` | `/api/v1/files/{id}/download-url` | Get pre-signed download URL | | `DELETE` | `/api/v1/files/{id}` | Delete a file (soft delete) | +### Direct Upload (Recommended for Scale) + +> **Benefits of Direct Upload:** +> - Backend doesn't handle file binary → 90%+ load reduction +> - Can handle millions of concurrent uploads +> - Faster upload speed (no intermediary) + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/v1/storage/sign-upload` | Get pre-signed PUT URL for direct upload | +| `POST` | `/api/v1/storage/confirm-upload` | Confirm upload and save metadata | + ### Quota | Method | Endpoint | Description | |--------|----------|-------------| | `GET` | `/api/v1/quota` | Get current user's storage quota | -## Upload Example +## Direct Upload Example (Recommended) + +### Step 1: Get Pre-signed URL + +```bash +curl -X POST "http://localhost:5002/api/v1/storage/sign-upload" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "fileName": "document.pdf", + "fileSizeBytes": 1048576, + "contentType": "application/pdf", + "accessLevel": "private" + }' +``` + +Response: +```json +{ + "success": true, + "data": { + "uploadUrl": "http://minio:9000/storage/private/user123/20260113/abc12345_document.pdf?X-Amz-...", + "objectKey": "private/user123/20260113/abc12345_document.pdf", + "expiresAt": "2026-01-13T21:49:33Z" + } +} +``` + +### Step 2: Upload Directly to MinIO + +```bash +curl -X PUT "${uploadUrl}" \ + -H "Content-Type: application/pdf" \ + --data-binary @document.pdf +``` + +### Step 3: Confirm Upload + +```bash +curl -X POST "http://localhost:5002/api/v1/storage/confirm-upload" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "objectKey": "private/user123/20260113/abc12345_document.pdf", + "fileName": "document.pdf", + "fileSizeBytes": 1048576, + "contentType": "application/pdf", + "accessLevel": "private" + }' +``` + +Response: +```json +{ + "success": true, + "data": { + "fileId": "550e8400-e29b-41d4-a716-446655440000", + "metadata": { + "id": "550e8400-e29b-41d4-a716-446655440000", + "fileName": "document.pdf", + "contentType": "application/pdf", + "fileSizeBytes": 1048576, + "provider": "MinIO", + "accessLevel": "Private", + "uploadedAt": "2026-01-13T20:49:33Z" + } + } +} +``` + +## Bucket Directory Structure + +Files are organized by access level and user ID with the following format: + +``` +{bucket}/ +├── private/{userId}/{date}/{fileId}_{filename} → Owner access only (via pre-signed URL) +├── public/{userId}/{date}/{fileId}_{filename} → Publicly accessible +└── shared/{userId}/{date}/{fileId}_{filename} → Controlled by sharing rules +``` + +### Object Key Format Details + +| Component | Description | Example | +|-----------|-------------|---------| +| `{bucket}` | Bucket name (from config) | `goodgo` | +| `{accessLevel}` | Access level prefix | `private`, `public`, `shared` | +| `{userId}` | Uploader's user ID | `user123` | +| `{date}` | Upload date (UTC) | `20260113` | +| `{fileId}` | First 8 chars of GUID | `a1b2c3d4` | +| `{filename}` | Sanitized file name | `document.pdf` | + +### Real-World Example + +``` +goodgo/ +├── private/ +│ └── user123/ +│ └── 20260113/ +│ ├── a1b2c3d4_document.pdf +│ └── e5f6g7h8_image.jpg +├── public/ +│ └── user456/ +│ └── 20260113/ +│ └── i9j0k1l2_avatar.png +└── shared/ + └── user789/ + └── 20260113/ + └── m3n4o5p6_presentation.pptx +``` + +> **Note**: The object key is returned in the `/sign-upload` response and must be sent back when calling `/confirm-upload`. + +## Legacy Upload Example (Via Backend) ```bash curl -X POST "http://localhost:5002/api/v1/files/upload" \ @@ -109,6 +234,7 @@ curl -X POST "http://localhost:5002/api/v1/files/upload" \ -F "file=@document.pdf" ``` + ## Inter-Service Communication The service communicates with IAM Service for user validation. This section explains the configuration required for proper inter-service communication in Docker environments. diff --git a/services/storage-service-net/docs/vi/ARCHITECTURE.md b/services/storage-service-net/docs/vi/ARCHITECTURE.md index b5f03491..240dce99 100644 --- a/services/storage-service-net/docs/vi/ARCHITECTURE.md +++ b/services/storage-service-net/docs/vi/ARCHITECTURE.md @@ -87,12 +87,80 @@ Entry point ứng dụng và triển khai CQRS: | Component | Mục đích | |-----------|----------| -| **FilesController** | Endpoints CRUD file | +| **FilesController** | Endpoints CRUD file (proxy upload cũ) | +| **SignedUrlController** | Endpoints direct upload (khuyến nghị) | | **QuotaController** | Endpoints quota của user | -| **UploadFileCommand** | Xử lý upload file | +| **SignUploadCommand** | Tạo pre-signed upload URLs | +| **ConfirmUploadCommand** | Xác nhận direct uploads và lưu metadata | +| **UploadFileCommand** | Xử lý proxy uploads (cũ) | | **DeleteFileCommand** | Xử lý xóa file | | **Query Handlers** | Xử lý các thao tác đọc | +## Kiến Trúc Direct Upload (Khuyến Nghị) + +Cho hệ thống với hàng triệu users, pattern Direct Client Upload được khuyến nghị thay vì proxy upload. + +### So Sánh Upload Patterns + +| Khía cạnh | Proxy Upload (Cũ) | Direct Upload (Khuyến nghị) | +|-----------|------------------|----------------------------| +| **Throughput** | ~100-500/giây | ~10,000+/giây | +| **Memory mỗi request** | 100MB (kích thước file) | ~10KB (chỉ metadata) | +| **Latency (file 100MB)** | 30-60 giây | 10-20 giây | +| **Tải backend** | Cao | Tối thiểu | + +### Luồng Direct Upload + +```mermaid +sequenceDiagram + participant Client + participant Storage_Service as Storage Service + participant MinIO + + rect rgb(200, 230, 200) + Note over Client,Storage_Service: 1. Yêu cầu Upload URL (nhẹ) + Client->>Storage_Service: POST /api/v1/storage/sign-upload + Storage_Service->>Storage_Service: Validate JWT, Kiểm tra Quota + Storage_Service-->>Client: Pre-signed PUT URL + ObjectKey + end + + rect rgb(200, 200, 230) + Note over Client,MinIO: 2. Upload trực tiếp (bỏ qua backend) + Client->>MinIO: PUT file binary vào Pre-signed URL + MinIO-->>Client: 200 OK + end + + rect rgb(230, 230, 200) + Note over Client,Storage_Service: 3. Xác nhận Upload (nhẹ) + Client->>Storage_Service: POST /api/v1/storage/confirm-upload + Storage_Service->>MinIO: Xác minh file tồn tại + Storage_Service->>Storage_Service: Lưu metadata, Cập nhật quota + Storage_Service-->>Client: File metadata + end +``` + +### Kiểm Soát Truy Cập Theo Path + +Files được tổ chức với prefix theo access level: + +``` +storage-bucket/ +├── public/{userId}/{date}/{fileId}_{filename} → Truy cập công khai +├── private/{userId}/{date}/{fileId}_{filename} → Yêu cầu pre-signed URL +└── shared/{userId}/{date}/{fileId}_{filename} → Kiểm soát bằng quy tắc +``` + +### Components Direct Upload + +| Component | Mục đích | +|-----------|----------| +| **SignUploadCommand** | Validate quota, tạo object key với path prefix, tạo pre-signed PUT URL | +| **SignUploadCommandHandler** | Xử lý yêu cầu sign-upload | +| **ConfirmUploadCommand** | Xác minh file tồn tại, lưu metadata, cập nhật quota | +| **ConfirmUploadCommandHandler** | Xử lý confirm-upload với idempotency | +| **SignedUrlController** | Endpoints `/sign-upload` và `/confirm-upload` | + + ## Kiến Trúc Storage Provider ```mermaid @@ -229,15 +297,30 @@ erDiagram ## API Endpoints +### Files (Proxy Upload Cũ) + | Method | Endpoint | Mô tả | |--------|----------|-------| -| `POST` | `/api/v1/files/upload` | Upload file (tối đa 100MB) | +| `POST` | `/api/v1/files/upload` | Upload file qua backend (tối đa 100MB) | | `GET` | `/api/v1/files` | Danh sách file với phân trang | | `GET` | `/api/v1/files/{id}` | Lấy metadata file | | `GET` | `/api/v1/files/{id}/download-url` | Lấy pre-signed download URL | | `DELETE` | `/api/v1/files/{id}` | Xóa file (soft delete) | + +### Direct Upload (Khuyến Nghị) + +| Method | Endpoint | Mô tả | +|--------|----------|-------| +| `POST` | `/api/v1/storage/sign-upload` | Lấy pre-signed PUT URL để upload trực tiếp | +| `POST` | `/api/v1/storage/confirm-upload` | Xác nhận upload và lưu metadata | + +### Quota + +| Method | Endpoint | Mô tả | +|--------|----------|-------| | `GET` | `/api/v1/quota` | Lấy quota storage của user | + ## Health Checks ```mermaid diff --git a/services/storage-service-net/docs/vi/README.md b/services/storage-service-net/docs/vi/README.md index 20a1e6d9..92733873 100644 --- a/services/storage-service-net/docs/vi/README.md +++ b/services/storage-service-net/docs/vi/README.md @@ -83,25 +83,151 @@ dotnet run --project src/StorageService.API | `IamService__CacheDurationSeconds` | TTL cache user info | `300` | | `IamService__HealthCheckCacheDurationSeconds` | TTL cache health check | `60` | + ## API Endpoints -### Files +### Files (Legacy - Proxy Upload) | Method | Endpoint | Mô tả | |--------|----------|-------| -| `POST` | `/api/v1/files/upload` | Upload file (tối đa 100MB) | +| `POST` | `/api/v1/files/upload` | Upload file qua backend (tối đa 100MB) | | `GET` | `/api/v1/files` | Danh sách files với phân trang | | `GET` | `/api/v1/files/{id}` | Lấy metadata file theo ID | | `GET` | `/api/v1/files/{id}/download-url` | Lấy pre-signed download URL | | `DELETE` | `/api/v1/files/{id}` | Xóa file (soft delete) | +### Direct Upload (Khuyến nghị cho Scale lớn) + +> **Lợi ích của Direct Upload:** +> - Backend không xử lý file binary → Giảm 90%+ tải +> - Có thể xử lý hàng triệu uploads đồng thời +> - Tốc độ upload nhanh hơn (không qua trung gian) + +| Method | Endpoint | Mô tả | +|--------|----------|-------| +| `POST` | `/api/v1/storage/sign-upload` | Lấy pre-signed PUT URL để upload trực tiếp | +| `POST` | `/api/v1/storage/confirm-upload` | Xác nhận upload và lưu metadata | + ### Quota | Method | Endpoint | Mô tả | |--------|----------|-------| | `GET` | `/api/v1/quota` | Lấy quota storage của user hiện tại | -## Ví Dụ Upload +## Ví Dụ Direct Upload (Khuyến nghị) + +### Bước 1: Lấy Pre-signed URL + +```bash +curl -X POST "http://localhost:5002/api/v1/storage/sign-upload" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "fileName": "document.pdf", + "fileSizeBytes": 1048576, + "contentType": "application/pdf", + "accessLevel": "private" + }' +``` + +Response: +```json +{ + "success": true, + "data": { + "uploadUrl": "http://minio:9000/storage/private/user123/20260113/abc12345_document.pdf?X-Amz-...", + "objectKey": "private/user123/20260113/abc12345_document.pdf", + "expiresAt": "2026-01-13T21:49:33Z" + } +} +``` + +### Bước 2: Upload trực tiếp lên MinIO + +```bash +curl -X PUT "${uploadUrl}" \ + -H "Content-Type: application/pdf" \ + --data-binary @document.pdf +``` + +### Bước 3: Xác nhận Upload + +```bash +curl -X POST "http://localhost:5002/api/v1/storage/confirm-upload" \ + -H "Authorization: Bearer YOUR_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "objectKey": "private/user123/20260113/abc12345_document.pdf", + "fileName": "document.pdf", + "fileSizeBytes": 1048576, + "contentType": "application/pdf", + "accessLevel": "private" + }' +``` + +Response: +```json +{ + "success": true, + "data": { + "fileId": "550e8400-e29b-41d4-a716-446655440000", + "metadata": { + "id": "550e8400-e29b-41d4-a716-446655440000", + "fileName": "document.pdf", + "contentType": "application/pdf", + "fileSizeBytes": 1048576, + "provider": "MinIO", + "accessLevel": "Private", + "uploadedAt": "2026-01-13T20:49:33Z" + } + } +} +``` + +## Cấu Trúc Thư Mục Trên Bucket + +Files được tổ chức theo access level và user ID với định dạng: + +``` +{bucket}/ +├── private/{userId}/{date}/{fileId}_{filename} → Chỉ owner truy cập (qua pre-signed URL) +├── public/{userId}/{date}/{fileId}_{filename} → Truy cập công khai +└── shared/{userId}/{date}/{fileId}_{filename} → Kiểm soát bởi sharing rules +``` + +### Chi Tiết Object Key Format + +| Thành phần | Mô tả | Ví dụ | +|------------|-------|-------| +| `{bucket}` | Tên bucket (config) | `goodgo` | +| `{accessLevel}` | Mức truy cập | `private`, `public`, `shared` | +| `{userId}` | ID của user upload | `user123` | +| `{date}` | Ngày upload (UTC) | `20260113` | +| `{fileId}` | 8 ký tự đầu của GUID | `a1b2c3d4` | +| `{filename}` | Tên file đã sanitize | `document.pdf` | + +### Ví Dụ Thực Tế + +``` +goodgo/ +├── private/ +│ └── user123/ +│ └── 20260113/ +│ ├── a1b2c3d4_document.pdf +│ └── e5f6g7h8_image.jpg +├── public/ +│ └── user456/ +│ └── 20260113/ +│ └── i9j0k1l2_avatar.png +└── shared/ + └── user789/ + └── 20260113/ + └── m3n4o5p6_presentation.pptx +``` + +> **Lưu ý**: Object key được trả về trong response của `/sign-upload` và cần được gửi lại khi gọi `/confirm-upload`. + +## Ví Dụ Legacy Upload (Qua Backend) ```bash curl -X POST "http://localhost:5002/api/v1/files/upload" \ @@ -109,6 +235,7 @@ curl -X POST "http://localhost:5002/api/v1/files/upload" \ -F "file=@document.pdf" ``` + ## Giao Tiếp Inter-Service Service giao tiếp với IAM Service để xác thực user. Phần này giải thích cấu hình cần thiết cho giao tiếp inter-service trong môi trường Docker. diff --git a/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommand.cs b/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommand.cs new file mode 100644 index 00000000..921fa7b8 --- /dev/null +++ b/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommand.cs @@ -0,0 +1,62 @@ +using MediatR; +using StorageService.API.Application.Queries; +using StorageService.Domain.AggregatesModel.FileAggregate; + +namespace StorageService.API.Application.Commands; + +/// +/// EN: Command to confirm a direct upload after client uploads to MinIO. +/// VI: Command để xác nhận upload trực tiếp sau khi client upload lên MinIO. +/// +/// +/// EN: Called after client successfully uploads file using pre-signed URL. +/// This saves metadata and updates quota. +/// VI: Được gọi sau khi client upload file thành công bằng pre-signed URL. +/// Lưu metadata và cập nhật quota. +/// +/// EN: User ID who uploaded / VI: ID người dùng đã upload +/// EN: Object key returned from sign-upload / VI: Object key được trả về từ sign-upload +/// EN: Original file name / VI: Tên file gốc +/// EN: MIME content type / VI: Content type MIME +/// EN: File size in bytes / VI: Kích thước file (bytes) +/// EN: Access level / VI: Mức truy cập +/// EN: Optional tenant ID / VI: Tenant ID (tùy chọn) +public record ConfirmUploadCommand( + string UserId, + string ObjectKey, + string FileName, + string ContentType, + long FileSizeBytes, + FileAccessLevel AccessLevel = FileAccessLevel.Private, + string? TenantId = null +) : IRequest; + +/// +/// EN: Result of confirm upload containing file metadata. +/// VI: Kết quả xác nhận upload chứa metadata file. +/// +/// EN: Whether confirmation was successful / VI: Xác nhận có thành công không +/// EN: File ID in database / VI: ID file trong database +/// EN: File metadata / VI: Metadata file +/// EN: Error message if failed / VI: Thông báo lỗi nếu thất bại +public record ConfirmUploadResult( + bool Success, + Guid? FileId, + FileDto? Metadata, + string? Error +) +{ + /// + /// EN: Create a successful result. + /// VI: Tạo kết quả thành công. + /// + public static ConfirmUploadResult Ok(Guid fileId, FileDto metadata) => + new(true, fileId, metadata, null); + + /// + /// EN: Create a failed result. + /// VI: Tạo kết quả thất bại. + /// + public static ConfirmUploadResult Fail(string error) => + new(false, null, null, error); +} diff --git a/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommandHandler.cs b/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommandHandler.cs new file mode 100644 index 00000000..cb7e101b --- /dev/null +++ b/services/storage-service-net/src/StorageService.API/Application/Commands/ConfirmUploadCommandHandler.cs @@ -0,0 +1,164 @@ +using MediatR; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StorageService.API.Application.Queries; +using StorageService.Domain.AggregatesModel.FileAggregate; +using StorageService.Domain.AggregatesModel.QuotaAggregate; +using StorageService.Infrastructure.Configuration; +using StorageService.Infrastructure.Storage; + +namespace StorageService.API.Application.Commands; + +/// +/// EN: Handler for ConfirmUploadCommand - confirms direct upload and saves metadata. +/// VI: Handler cho ConfirmUploadCommand - xác nhận upload trực tiếp và lưu metadata. +/// +/// +/// EN: Called after client successfully uploads file using pre-signed URL. +/// Verifies file exists in storage, saves metadata to database, and updates quota. +/// VI: Được gọi sau khi client upload file thành công bằng pre-signed URL. +/// Xác minh file tồn tại trong storage, lưu metadata vào database, và cập nhật quota. +/// +public class ConfirmUploadCommandHandler : IRequestHandler +{ + private readonly IFileRepository _fileRepository; + private readonly IQuotaRepository _quotaRepository; + private readonly IStorageProviderFactory _storageProviderFactory; + private readonly StorageSettings _settings; + private readonly ILogger _logger; + + public ConfirmUploadCommandHandler( + IFileRepository fileRepository, + IQuotaRepository quotaRepository, + IStorageProviderFactory storageProviderFactory, + IOptions settings, + ILogger logger) + { + _fileRepository = fileRepository; + _quotaRepository = quotaRepository; + _storageProviderFactory = storageProviderFactory; + _settings = settings.Value; + _logger = logger; + } + + public async Task Handle(ConfirmUploadCommand request, CancellationToken cancellationToken) + { + try + { + _logger.LogInformation( + "Confirm upload for user {UserId}, objectKey: {ObjectKey}", + request.UserId, request.ObjectKey); + + // EN: Validate object key belongs to user + // VI: Kiểm tra object key thuộc về user + if (!ValidateObjectKeyOwnership(request.ObjectKey, request.UserId)) + { + _logger.LogWarning( + "Object key {ObjectKey} does not belong to user {UserId}", + request.ObjectKey, request.UserId); + return ConfirmUploadResult.Fail("Invalid object key."); + } + + var bucketName = _settings.DefaultBucket; + var provider = _storageProviderFactory.GetProvider(); + + // EN: Verify file exists in storage / VI: Xác minh file tồn tại trong storage + var exists = await provider.ExistsAsync(bucketName, request.ObjectKey, cancellationToken); + if (!exists) + { + _logger.LogWarning( + "File not found in storage: {Bucket}/{ObjectKey}", + bucketName, request.ObjectKey); + return ConfirmUploadResult.Fail("File not found in storage. Upload may have failed."); + } + + // EN: Check if already confirmed (idempotency) + // VI: Kiểm tra đã xác nhận chưa (idempotency) + var existingFile = await _fileRepository.GetByObjectKeyAsync(request.ObjectKey, cancellationToken); + if (existingFile != null) + { + _logger.LogInformation( + "Upload already confirmed for objectKey: {ObjectKey}, fileId: {FileId}", + request.ObjectKey, existingFile.Id); + return ConfirmUploadResult.Ok(existingFile.Id, existingFile.ToDto()); + } + + // EN: Get or create quota / VI: Lấy hoặc tạo quota + var quota = await _quotaRepository.GetOrCreateAsync(request.UserId, cancellationToken); + + // EN: Double-check quota (in case of concurrent uploads) + // VI: Kiểm tra lại quota (phòng trường hợp upload đồng thời) + if (!quota.CanUpload(request.FileSizeBytes)) + { + _logger.LogWarning( + "Quota exceeded for user {UserId} during confirmation", + request.UserId); + + // EN: Delete the uploaded file since quota exceeded + // VI: Xóa file đã upload vì vượt quota + await provider.DeleteAsync(bucketName, request.ObjectKey, cancellationToken); + + return ConfirmUploadResult.Fail( + "Storage quota exceeded. The uploaded file has been removed."); + } + + // EN: Create file metadata / VI: Tạo metadata file + var storageFile = new StorageFile( + request.FileName, + bucketName, + request.ObjectKey, + request.ContentType, + request.FileSizeBytes, + request.UserId, + provider.ProviderType, + request.AccessLevel, + request.TenantId, + checksum: null); + + await _fileRepository.AddAsync(storageFile, cancellationToken); + + // EN: Update quota / VI: Cập nhật quota + quota.AddUsage(request.FileSizeBytes); + _quotaRepository.Update(quota); + + // EN: Save changes / VI: Lưu thay đổi + await _fileRepository.UnitOfWork.SaveEntitiesAsync(cancellationToken); + + _logger.LogInformation( + "Upload confirmed successfully: fileId={FileId}, objectKey={ObjectKey}, user={UserId}", + storageFile.Id, request.ObjectKey, request.UserId); + + return ConfirmUploadResult.Ok(storageFile.Id, storageFile.ToDto()); + } + catch (Exception ex) + { + _logger.LogError(ex, + "Error confirming upload for user {UserId}, objectKey: {ObjectKey}", + request.UserId, request.ObjectKey); + return ConfirmUploadResult.Fail("An error occurred while confirming upload."); + } + } + + /// + /// EN: Validate that object key belongs to the specified user. + /// VI: Xác minh object key thuộc về user được chỉ định. + /// + /// + /// EN: Object key format: {accessLevel}/{userId}/{date}/{fileId}_{filename} + /// VI: Định dạng object key: {accessLevel}/{userId}/{date}/{fileId}_{filename} + /// + private static bool ValidateObjectKeyOwnership(string objectKey, string userId) + { + if (string.IsNullOrEmpty(objectKey)) + return false; + + var parts = objectKey.Split('/'); + if (parts.Length < 3) + return false; + + // EN: Format: {prefix}/{userId}/{date}/... + // VI: Định dạng: {prefix}/{userId}/{date}/... + var ownerUserId = parts[1]; + return ownerUserId == userId; + } +} diff --git a/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommand.cs b/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommand.cs new file mode 100644 index 00000000..4c7ffd14 --- /dev/null +++ b/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommand.cs @@ -0,0 +1,61 @@ +using MediatR; +using StorageService.Domain.AggregatesModel.FileAggregate; + +namespace StorageService.API.Application.Commands; + +/// +/// EN: Command to request a pre-signed URL for direct upload. +/// VI: Command để yêu cầu pre-signed URL cho upload trực tiếp. +/// +/// +/// EN: This enables Direct Client Upload pattern where files are uploaded directly to MinIO, +/// bypassing the backend to handle millions of concurrent uploads efficiently. +/// VI: Pattern này cho phép upload trực tiếp lên MinIO, bỏ qua backend để xử lý +/// hàng triệu uploads đồng thời một cách hiệu quả. +/// +/// EN: User ID requesting upload / VI: ID người dùng yêu cầu upload +/// EN: Original file name / VI: Tên file gốc +/// EN: MIME content type / VI: Content type MIME +/// EN: File size in bytes for quota check / VI: Kích thước file (bytes) để kiểm tra quota +/// EN: Access level: Public, Private, or Shared / VI: Mức truy cập: Public, Private, hoặc Shared +/// EN: Optional tenant ID / VI: Tenant ID (tùy chọn) +public record SignUploadCommand( + string UserId, + string FileName, + string ContentType, + long FileSizeBytes, + FileAccessLevel AccessLevel = FileAccessLevel.Private, + string? TenantId = null +) : IRequest; + +/// +/// EN: Result of sign upload request containing pre-signed URL. +/// VI: Kết quả yêu cầu sign upload chứa pre-signed URL. +/// +/// EN: Whether the request was successful / VI: Yêu cầu có thành công không +/// EN: Pre-signed URL for PUT request / VI: Pre-signed URL cho PUT request +/// EN: Object key in storage / VI: Object key trong storage +/// EN: URL expiration time / VI: Thời gian hết hạn URL +/// EN: Error message if failed / VI: Thông báo lỗi nếu thất bại +public record SignUploadResult( + bool Success, + string? UploadUrl, + string? ObjectKey, + DateTime? ExpiresAt, + string? Error +) +{ + /// + /// EN: Create a successful result. + /// VI: Tạo kết quả thành công. + /// + public static SignUploadResult Ok(string uploadUrl, string objectKey, DateTime expiresAt) => + new(true, uploadUrl, objectKey, expiresAt, null); + + /// + /// EN: Create a failed result. + /// VI: Tạo kết quả thất bại. + /// + public static SignUploadResult Fail(string error) => + new(false, null, null, null, error); +} diff --git a/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommandHandler.cs b/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommandHandler.cs new file mode 100644 index 00000000..0839f298 --- /dev/null +++ b/services/storage-service-net/src/StorageService.API/Application/Commands/SignUploadCommandHandler.cs @@ -0,0 +1,152 @@ +using MediatR; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Options; +using StorageService.Domain.AggregatesModel.FileAggregate; +using StorageService.Domain.AggregatesModel.QuotaAggregate; +using StorageService.Infrastructure.Configuration; +using StorageService.Infrastructure.Storage; + +namespace StorageService.API.Application.Commands; + +/// +/// EN: Handler for SignUploadCommand - generates pre-signed URL for direct upload. +/// VI: Handler cho SignUploadCommand - tạo pre-signed URL cho upload trực tiếp. +/// +/// +/// EN: This handler validates quota, generates object key with path prefix based on access level, +/// and returns a pre-signed PUT URL for client to upload directly to MinIO. +/// VI: Handler này kiểm tra quota, tạo object key với path prefix theo access level, +/// và trả về pre-signed PUT URL để client upload trực tiếp lên MinIO. +/// +public class SignUploadCommandHandler : IRequestHandler +{ + private readonly IQuotaRepository _quotaRepository; + private readonly IStorageProviderFactory _storageProviderFactory; + private readonly StorageSettings _settings; + private readonly ILogger _logger; + + public SignUploadCommandHandler( + IQuotaRepository quotaRepository, + IStorageProviderFactory storageProviderFactory, + IOptions settings, + ILogger logger) + { + _quotaRepository = quotaRepository; + _storageProviderFactory = storageProviderFactory; + _settings = settings.Value; + _logger = logger; + } + + public async Task Handle(SignUploadCommand request, CancellationToken cancellationToken) + { + try + { + _logger.LogInformation( + "Sign upload request for user {UserId}, file: {FileName}, size: {Size} bytes", + request.UserId, request.FileName, request.FileSizeBytes); + + // EN: Validate file size / VI: Kiểm tra kích thước file + if (request.FileSizeBytes > _settings.MaxFileSizeBytes) + { + _logger.LogWarning( + "File size {Size} exceeds maximum {Max} for user {UserId}", + request.FileSizeBytes, _settings.MaxFileSizeBytes, request.UserId); + + return SignUploadResult.Fail( + $"File size exceeds maximum allowed ({_settings.MaxFileSizeBytes / 1024 / 1024}MB)"); + } + + // EN: Check user quota / VI: Kiểm tra quota của user + var quota = await _quotaRepository.GetOrCreateAsync(request.UserId, cancellationToken); + if (!quota.CanUpload(request.FileSizeBytes)) + { + _logger.LogWarning( + "Quota exceeded for user {UserId}. Used: {Used}/{Max}", + request.UserId, quota.UsedStorageBytes, quota.MaxStorageBytes); + + return SignUploadResult.Fail( + "Storage quota exceeded. Please upgrade your plan or delete some files."); + } + + // EN: Generate object key with path prefix based on access level + // VI: Tạo object key với path prefix theo access level + var objectKey = GenerateObjectKey(request.UserId, request.FileName, request.AccessLevel); + var bucketName = _settings.DefaultBucket; + var expirationSeconds = _settings.PreSignedUrlExpirationSeconds; + + // EN: Ensure bucket exists / VI: Đảm bảo bucket tồn tại + var provider = _storageProviderFactory.GetProvider(); + await provider.EnsureBucketExistsAsync(bucketName, cancellationToken); + + // EN: Generate pre-signed PUT URL / VI: Tạo pre-signed PUT URL + var uploadUrl = await provider.GetPreSignedUploadUrlAsync( + bucketName, + objectKey, + expirationSeconds, + cancellationToken); + + var expiresAt = DateTime.UtcNow.AddSeconds(expirationSeconds); + + _logger.LogInformation( + "Generated pre-signed URL for user {UserId}, objectKey: {ObjectKey}, expires: {ExpiresAt}", + request.UserId, objectKey, expiresAt); + + return SignUploadResult.Ok(uploadUrl, objectKey, expiresAt); + } + catch (Exception ex) + { + _logger.LogError(ex, "Error generating pre-signed URL for user {UserId}", request.UserId); + return SignUploadResult.Fail("An error occurred while generating upload URL."); + } + } + + /// + /// EN: Generate object key with path prefix based on access level. + /// VI: Tạo object key với path prefix theo access level. + /// + /// + /// EN: Path structure: + /// - public/{userId}/{date}/{fileId}_{filename} → Publicly accessible + /// - private/{userId}/{date}/{fileId}_{filename} → Requires pre-signed URL + /// - shared/{userId}/{date}/{fileId}_{filename} → Access controlled by sharing rules + /// VI: Cấu trúc path: + /// - public/{userId}/{date}/{fileId}_{filename} → Truy cập công khai + /// - private/{userId}/{date}/{fileId}_{filename} → Yêu cầu pre-signed URL + /// - shared/{userId}/{date}/{fileId}_{filename} → Kiểm soát bởi quy tắc chia sẻ + /// + private static string GenerateObjectKey(string userId, string fileName, FileAccessLevel accessLevel) + { + var prefix = accessLevel switch + { + FileAccessLevel.Public => "public", + FileAccessLevel.Shared => "shared", + _ => "private" + }; + + var date = DateTime.UtcNow.ToString("yyyyMMdd"); + var fileId = Guid.NewGuid().ToString("N")[..8]; + var sanitizedName = SanitizeFileName(fileName); + + return $"{prefix}/{userId}/{date}/{fileId}_{sanitizedName}"; + } + + /// + /// EN: Sanitize file name to remove invalid characters. + /// VI: Làm sạch tên file để loại bỏ ký tự không hợp lệ. + /// + private static string SanitizeFileName(string fileName) + { + var invalidChars = Path.GetInvalidFileNameChars(); + var sanitized = string.Join("_", fileName.Split(invalidChars, StringSplitOptions.RemoveEmptyEntries)); + + // EN: Limit file name length / VI: Giới hạn độ dài tên file + if (sanitized.Length > 100) + { + var extension = Path.GetExtension(sanitized); + var nameWithoutExt = Path.GetFileNameWithoutExtension(sanitized); + sanitized = nameWithoutExt[..Math.Min(nameWithoutExt.Length, 90)] + extension; + } + + return sanitized; + } +} diff --git a/services/storage-service-net/src/StorageService.API/Controllers/SignedUrlController.cs b/services/storage-service-net/src/StorageService.API/Controllers/SignedUrlController.cs new file mode 100644 index 00000000..75d529c6 --- /dev/null +++ b/services/storage-service-net/src/StorageService.API/Controllers/SignedUrlController.cs @@ -0,0 +1,266 @@ +using MediatR; +using Microsoft.AspNetCore.Authorization; +using Microsoft.AspNetCore.Mvc; +using StorageService.API.Application.Commands; +using StorageService.API.Application.Queries; +using StorageService.Domain.AggregatesModel.FileAggregate; +using Swashbuckle.AspNetCore.Annotations; +using System.Security.Claims; +using Asp.Versioning; + +namespace StorageService.API.Controllers; + +/// +/// EN: Controller for direct upload using pre-signed URLs. +/// VI: Controller cho upload trực tiếp sử dụng pre-signed URLs. +/// +/// +/// EN: This controller implements the Direct Client Upload pattern where files are uploaded +/// directly to MinIO, bypassing the backend to handle millions of concurrent uploads. +/// +/// Flow: +/// 1. Client calls POST /sign-upload to get pre-signed PUT URL +/// 2. Client uploads file directly to MinIO using the pre-signed URL +/// 3. Client calls POST /confirm-upload to save metadata +/// +/// VI: Controller này triển khai pattern Direct Client Upload, nơi files được upload +/// trực tiếp lên MinIO, bỏ qua backend để xử lý hàng triệu uploads đồng thời. +/// +/// Luồng: +/// 1. Client gọi POST /sign-upload để lấy pre-signed PUT URL +/// 2. Client upload file trực tiếp lên MinIO bằng pre-signed URL +/// 3. Client gọi POST /confirm-upload để lưu metadata +/// +[ApiController] +[ApiVersion("1.0")] +[Route("api/v{version:apiVersion}/storage")] +[SwaggerTag("Direct Upload - Pre-signed URL based upload for high scalability")] +public class SignedUrlController : ControllerBase +{ + private readonly IMediator _mediator; + private readonly ILogger _logger; + + public SignedUrlController(IMediator mediator, ILogger logger) + { + _mediator = mediator; + _logger = logger; + } + + /// + /// EN: Request a pre-signed URL for direct upload. + /// VI: Yêu cầu pre-signed URL để upload trực tiếp. + /// + /// + /// EN: Returns a pre-signed PUT URL that allows the client to upload directly to MinIO. + /// The URL is valid for a limited time (default: 1 hour). + /// + /// VI: Trả về pre-signed PUT URL cho phép client upload trực tiếp lên MinIO. + /// URL có hiệu lực trong thời gian giới hạn (mặc định: 1 giờ). + /// + [HttpPost("sign-upload")] + [Authorize] + [SwaggerOperation( + Summary = "Get pre-signed upload URL", + Description = "Request a pre-signed PUT URL for direct file upload to storage")] + [SwaggerResponse(200, "Pre-signed URL generated successfully")] + [SwaggerResponse(400, "Invalid request or quota exceeded")] + [SwaggerResponse(401, "Unauthorized")] + public async Task>> SignUpload( + [FromBody] SignUploadRequest request, + CancellationToken cancellationToken = default) + { + var userId = GetUserId(); + if (string.IsNullOrEmpty(userId)) + return Unauthorized(new ApiResponse + { + Success = false, + Error = "User ID not found" + }); + + // EN: Validate request / VI: Validate request + if (string.IsNullOrWhiteSpace(request.FileName)) + return BadRequest(new ApiResponse + { + Success = false, + Error = "File name is required" + }); + + if (request.FileSizeBytes <= 0) + return BadRequest(new ApiResponse + { + Success = false, + Error = "File size must be greater than 0" + }); + + var accessLevel = ParseAccessLevel(request.AccessLevel); + + var command = new SignUploadCommand( + userId, + request.FileName, + request.ContentType ?? "application/octet-stream", + request.FileSizeBytes, + accessLevel); + + var result = await _mediator.Send(command, cancellationToken); + + if (!result.Success) + return BadRequest(new ApiResponse + { + Success = false, + Error = result.Error + }); + + return Ok(new ApiResponse + { + Success = true, + Data = new SignUploadResponse( + result.UploadUrl!, + result.ObjectKey!, + result.ExpiresAt!.Value) + }); + } + + /// + /// EN: Confirm upload after client uploads to MinIO. + /// VI: Xác nhận upload sau khi client upload lên MinIO. + /// + /// + /// EN: Call this endpoint after successfully uploading the file using the pre-signed URL. + /// This saves the file metadata and updates the user's quota. + /// + /// VI: Gọi endpoint này sau khi upload file thành công bằng pre-signed URL. + /// Lưu metadata file và cập nhật quota của user. + /// + [HttpPost("confirm-upload")] + [Authorize] + [SwaggerOperation( + Summary = "Confirm file upload", + Description = "Confirm upload completion and save file metadata")] + [SwaggerResponse(200, "Upload confirmed successfully")] + [SwaggerResponse(400, "File not found in storage or validation failed")] + [SwaggerResponse(401, "Unauthorized")] + public async Task>> ConfirmUpload( + [FromBody] ConfirmUploadRequest request, + CancellationToken cancellationToken = default) + { + var userId = GetUserId(); + if (string.IsNullOrEmpty(userId)) + return Unauthorized(new ApiResponse + { + Success = false, + Error = "User ID not found" + }); + + // EN: Validate request / VI: Validate request + if (string.IsNullOrWhiteSpace(request.ObjectKey)) + return BadRequest(new ApiResponse + { + Success = false, + Error = "Object key is required" + }); + + if (string.IsNullOrWhiteSpace(request.FileName)) + return BadRequest(new ApiResponse + { + Success = false, + Error = "File name is required" + }); + + var accessLevel = ParseAccessLevel(request.AccessLevel); + + var command = new ConfirmUploadCommand( + userId, + request.ObjectKey, + request.FileName, + request.ContentType ?? "application/octet-stream", + request.FileSizeBytes, + accessLevel); + + var result = await _mediator.Send(command, cancellationToken); + + if (!result.Success) + return BadRequest(new ApiResponse + { + Success = false, + Error = result.Error + }); + + return Ok(new ApiResponse + { + Success = true, + Data = new ConfirmUploadResponse( + result.FileId!.Value, + result.Metadata!) + }); + } + + private string? GetUserId() => User.FindFirstValue(ClaimTypes.NameIdentifier); + + private static FileAccessLevel ParseAccessLevel(string? accessLevel) => + accessLevel?.ToLowerInvariant() switch + { + "public" => FileAccessLevel.Public, + "shared" => FileAccessLevel.Shared, + _ => FileAccessLevel.Private + }; +} + +#region Request/Response DTOs + +/// +/// EN: Request to sign upload URL. +/// VI: Request để ký URL upload. +/// +/// EN: Original file name / VI: Tên file gốc +/// EN: File size in bytes / VI: Kích thước file (bytes) +/// EN: MIME content type / VI: Content type MIME +/// EN: Access level: public, private, shared / VI: Mức truy cập: public, private, shared +public record SignUploadRequest( + string FileName, + long FileSizeBytes, + string? ContentType = null, + string? AccessLevel = "private" +); + +/// +/// EN: Response containing pre-signed upload URL. +/// VI: Response chứa pre-signed URL upload. +/// +/// EN: Pre-signed PUT URL / VI: Pre-signed PUT URL +/// EN: Object key to use for confirm-upload / VI: Object key để sử dụng cho confirm-upload +/// EN: URL expiration time / VI: Thời gian hết hạn URL +public record SignUploadResponse( + string UploadUrl, + string ObjectKey, + DateTime ExpiresAt +); + +/// +/// EN: Request to confirm upload. +/// VI: Request để xác nhận upload. +/// +/// EN: Object key from sign-upload response / VI: Object key từ response sign-upload +/// EN: Original file name / VI: Tên file gốc +/// EN: File size in bytes / VI: Kích thước file (bytes) +/// EN: MIME content type / VI: Content type MIME +/// EN: Access level / VI: Mức truy cập +public record ConfirmUploadRequest( + string ObjectKey, + string FileName, + long FileSizeBytes, + string? ContentType = null, + string? AccessLevel = "private" +); + +/// +/// EN: Response after confirming upload. +/// VI: Response sau khi xác nhận upload. +/// +/// EN: File ID in database / VI: ID file trong database +/// EN: File metadata / VI: Metadata file +public record ConfirmUploadResponse( + Guid FileId, + FileDto Metadata +); + +#endregion