diff --git a/services/storage-service-net/README.md b/services/storage-service-net/README.md index 5f8770c1..f4690be0 100644 --- a/services/storage-service-net/README.md +++ b/services/storage-service-net/README.md @@ -64,14 +64,39 @@ dotnet run --project src/StorageService.API ## API Endpoints +### Files + | Method | Endpoint | Description | |--------|----------|-------------| -| `POST` | `/api/v1/files/upload` | Upload file | -| `GET` | `/api/v1/files` | List user files | -| `GET` | `/api/v1/files/{id}` | Get file by ID | -| `GET` | `/api/v1/files/{id}/download-url` | Get pre-signed download URL | -| `DELETE` | `/api/v1/files/{id}` | Delete file | -| `GET` | `/api/v1/quota` | Get user quota | +| `POST` | `/api/v1/files/upload` | Upload file via backend | +| `GET` | `/api/v1/files` | List user files with pagination | +| `GET` | `/api/v1/files/{id}` | Get file metadata by ID | +| `GET` | `/api/v1/files/{id}/download-url` | Get download URL (presigned for private) | +| `DELETE` | `/api/v1/files/{id}` | Soft delete file | + +### Direct Upload (Recommended) + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/v1/storage/sign-upload` | Get pre-signed PUT URL | +| `POST` | `/api/v1/storage/confirm-upload` | Confirm upload and save metadata | + +### File Sharing + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `POST` | `/api/v1/files/{id}/share` | Create share link for file | +| `GET` | `/api/v1/files/shared/{token}` | Access file via share token | +| `DELETE` | `/api/v1/files/{id}/share` | Revoke share link | + +### Quota + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/api/v1/quota` | Get current user quota | +| `GET` | `/api/v1/quota/me` | Get quota for authenticated user | + +> **Note:** For private/shared files, `download-url` returns a pre-signed URL with AWS Signature V4. See [docs/en/README.md](docs/en/README.md) for details. ## Database Migrations / Migration Database diff --git a/services/storage-service-net/docs/en/ARCHITECTURE.md b/services/storage-service-net/docs/en/ARCHITECTURE.md index 674a5fd4..e414c38a 100644 --- a/services/storage-service-net/docs/en/ARCHITECTURE.md +++ b/services/storage-service-net/docs/en/ARCHITECTURE.md @@ -400,6 +400,88 @@ WHERE id = '{file-id}'; | **ConfirmUploadCommandHandler** | Handle confirm-upload with idempotency | | **SignedUrlController** | `/sign-upload` and `/confirm-upload` endpoints | +## Download URL Architecture + +The Storage Service generates different download URLs based on file access levels. + +### URL Generation Flow + +```mermaid +sequenceDiagram + participant Client + participant API as Storage Service + participant MinIO + + Client->>API: GET /api/v1/files/{id}/download-url + API->>API: Validate ownership + + alt accessLevel = Public + API-->>Client: Direct URL (no signature) + else accessLevel = Private/Shared + API->>MinIO: GetPreSignedDownloadUrl() + MinIO-->>API: Pre-signed URL (AWS Signature V4) + API-->>Client: Pre-signed URL with expiration + end +``` + +### Access Level URL Types + +| Access Level | Storage Prefix | URL Type | Expiration | +|--------------|----------------|----------|------------| +| **Public** | `public/` | Direct URL | Never | +| **Private** | `private/` | Pre-signed URL | Configurable (default: 1 hour) | +| **Shared** | `shared/` | Pre-signed URL | Configurable (default: 1 hour) | + +### Pre-signed URL Structure + +For private/shared files, URLs include AWS Signature Version 4 parameters: + +``` +http://minio:9000/bucket/private/2026/01/13/{uuid}.pdf + ?X-Amz-Algorithm=AWS4-HMAC-SHA256 + &X-Amz-Credential={accessKey}%2F{date}%2F{region}%2Fs3%2Faws4_request + &X-Amz-Date={timestamp} + &X-Amz-Expires={seconds} + &X-Amz-SignedHeaders=host + &X-Amz-Signature={signature} +``` + +| Parameter | Purpose | +|-----------|---------| +| `X-Amz-Algorithm` | Signing algorithm (always AWS4-HMAC-SHA256) | +| `X-Amz-Credential` | Access key + scope (date/region/service) | +| `X-Amz-Date` | Timestamp when signature was created | +| `X-Amz-Expires` | URL validity in seconds | +| `X-Amz-SignedHeaders` | Headers included in signature calculation | +| `X-Amz-Signature` | Cryptographic signature to verify URL integrity | + +### Security Considerations + +1. **Time-Limited Access**: Pre-signed URLs expire after configured time +2. **Tamper-Proof**: Any URL modification invalidates the signature +3. **Credential Protection**: MinIO access keys never exposed to client +4. **Unique URLs**: Each request generates a new signature + +### GetDownloadUrl Implementation + +```csharp +public async Task GetDownloadUrlAsync(StorageFile file) +{ + if (file.AccessLevel == FileAccessLevel.Public) + { + // Direct URL for public files + return $"{_settings.PublicEndpoint}/{file.BucketName}/{file.ObjectKey}"; + } + + // Pre-signed URL for private/shared files + return await _storageProvider.GetPreSignedDownloadUrlAsync( + file.BucketName, + file.ObjectKey, + _settings.PreSignedUrlExpirationSeconds + ); +} +``` + ## Multipart Upload Architecture (Large Files) For files larger than 100MB, use Multipart Upload to upload in chunks. diff --git a/services/storage-service-net/docs/en/README.md b/services/storage-service-net/docs/en/README.md index 93f6facd..f2ada9c0 100644 --- a/services/storage-service-net/docs/en/README.md +++ b/services/storage-service-net/docs/en/README.md @@ -183,6 +183,100 @@ Response: } ``` +## Pre-signed URLs & Access Levels + +The Storage Service generates different types of download URLs based on the file's `accessLevel`. + +### Access Level Overview + +| Access Level | Storage Path Prefix | Download URL Type | Use Case | +|--------------|---------------------|-------------------|----------| +| **Public** | `public/` | Direct URL (no signature) | Public assets, images, CDN | +| **Private** | `private/` | Pre-signed URL (AWS Signature V4) | User files, documents | +| **Shared** | `shared/` | Pre-signed URL (AWS Signature V4) | Shared files with link | + +### Public Files + +Public files can be accessed directly without authentication: + +``` +http://minio:9000/storage/public/2026/01/13/abc123.png +``` + +**Characteristics:** +- No signature required +- URL never expires +- Anyone with the URL can access +- Best for: avatars, thumbnails, public downloads + +### Private/Shared Files (Pre-signed URLs) + +Private and shared files require a **pre-signed URL** with AWS Signature Version 4: + +``` +http://minio:9000/storage/private/2026/01/13/xyz789.pdf + ?X-Amz-Algorithm=AWS4-HMAC-SHA256 + &X-Amz-Credential=minioadmin%2F20260113%2Fus-east-1%2Fs3%2Faws4_request + &X-Amz-Date=20260113T180024Z + &X-Amz-Expires=3600 + &X-Amz-SignedHeaders=host + &X-Amz-Signature=2ce827d357d105fc1cf88240dee407e5ea72... +``` + +**Signature Parameters Explained:** + +| Parameter | Description | Example | +|-----------|-------------|---------| +| `X-Amz-Algorithm` | Signing algorithm | `AWS4-HMAC-SHA256` | +| `X-Amz-Credential` | Access key + date/region/service | `minioadmin/20260113/us-east-1/s3/aws4_request` | +| `X-Amz-Date` | Signature creation timestamp | `20260113T180024Z` | +| `X-Amz-Expires` | URL validity in seconds | `3600` (1 hour) | +| `X-Amz-SignedHeaders` | Headers included in signature | `host` | +| `X-Amz-Signature` | The cryptographic signature | `2ce827d3...` | + +**Security Benefits:** +- URL expires after configured time (default: 1 hour) +- Signature prevents URL tampering +- No MinIO credentials exposed to client +- Each URL is unique and single-use in practice + +### Get Download URL API + +```bash +curl -X GET "http://localhost:5002/api/v1/files/{id}/download-url" \ + -H "Authorization: Bearer YOUR_TOKEN" +``` + +**Response for Public file:** +```json +{ + "success": true, + "data": { + "downloadUrl": "http://minio:9000/storage/public/2026/01/13/abc123.png", + "expiresAt": null + } +} +``` + +**Response for Private file:** +```json +{ + "success": true, + "data": { + "downloadUrl": "http://minio:9000/storage/private/2026/01/13/xyz789.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...", + "expiresAt": "2026-01-13T19:00:24Z" + } +} +``` + +### Configuration + +| Variable | Description | Default | +|----------|-------------|---------| +| `Storage__PreSignedUrlExpirationSeconds` | Pre-signed URL validity period | `3600` (1 hour) | + +> **Note:** For very large files or slow connections, consider increasing the expiration time. + ## Logical Folder Architecture (Data Sovereignty) > ⚠️ **IMPORTANT**: Following the **Data Sovereignty** principle in microservices, folders are a **logical concept in the Database**, NOT dependent on bucket structure. diff --git a/services/storage-service-net/docs/vi/ARCHITECTURE.md b/services/storage-service-net/docs/vi/ARCHITECTURE.md index 2e310be0..08d0ad38 100644 --- a/services/storage-service-net/docs/vi/ARCHITECTURE.md +++ b/services/storage-service-net/docs/vi/ARCHITECTURE.md @@ -400,6 +400,88 @@ WHERE id = '{file-id}'; | **ConfirmUploadCommandHandler** | Xử lý confirm-upload với idempotency | | **SignedUrlController** | Endpoints `/sign-upload` và `/confirm-upload` | +## Kiến Trúc Download URL + +Storage Service tạo các loại download URL khác nhau dựa trên access level của file. + +### Luồng Tạo URL + +```mermaid +sequenceDiagram + participant Client + participant API as Storage Service + participant MinIO + + Client->>API: GET /api/v1/files/{id}/download-url + API->>API: Validate ownership + + alt accessLevel = Public + API-->>Client: Direct URL (không có signature) + else accessLevel = Private/Shared + API->>MinIO: GetPreSignedDownloadUrl() + MinIO-->>API: Pre-signed URL (AWS Signature V4) + API-->>Client: Pre-signed URL với thời hạn + end +``` + +### Loại URL Theo Access Level + +| Access Level | Storage Prefix | Loại URL | Hết hạn | +|--------------|----------------|----------|------------| +| **Public** | `public/` | Direct URL | Không bao giờ | +| **Private** | `private/` | Pre-signed URL | Có thể cấu hình (mặc định: 1 giờ) | +| **Shared** | `shared/` | Pre-signed URL | Có thể cấu hình (mặc định: 1 giờ) | + +### Cấu Trúc Pre-signed URL + +Với files private/shared, URLs bao gồm các tham số AWS Signature Version 4: + +``` +http://minio:9000/bucket/private/2026/01/13/{uuid}.pdf + ?X-Amz-Algorithm=AWS4-HMAC-SHA256 + &X-Amz-Credential={accessKey}%2F{date}%2F{region}%2Fs3%2Faws4_request + &X-Amz-Date={timestamp} + &X-Amz-Expires={seconds} + &X-Amz-SignedHeaders=host + &X-Amz-Signature={signature} +``` + +| Tham số | Mục đích | +|---------|----------| +| `X-Amz-Algorithm` | Thuật toán ký (luôn là AWS4-HMAC-SHA256) | +| `X-Amz-Credential` | Access key + scope (date/region/service) | +| `X-Amz-Date` | Thời điểm tạo signature | +| `X-Amz-Expires` | Thời hạn URL tính bằng giây | +| `X-Amz-SignedHeaders` | Headers được bao gồm trong tính toán signature | +| `X-Amz-Signature` | Chữ ký mã hóa để xác minh tính toàn vẹn URL | + +### Cân Nhắc Bảo Mật + +1. **Truy Cập Có Thời Hạn**: Pre-signed URLs hết hạn sau thời gian cấu hình +2. **Chống Chỉnh Sửa**: Bất kỳ sửa đổi URL nào đều làm signature không hợp lệ +3. **Bảo Vệ Credentials**: Access keys MinIO không bao giờ lộ cho client +4. **URLs Duy Nhất**: Mỗi request tạo ra signature mới + +### Implementation GetDownloadUrl + +```csharp +public async Task GetDownloadUrlAsync(StorageFile file) +{ + if (file.AccessLevel == FileAccessLevel.Public) + { + // Direct URL cho files public + return $"{_settings.PublicEndpoint}/{file.BucketName}/{file.ObjectKey}"; + } + + // Pre-signed URL cho files private/shared + return await _storageProvider.GetPreSignedDownloadUrlAsync( + file.BucketName, + file.ObjectKey, + _settings.PreSignedUrlExpirationSeconds + ); +} +``` + ## Kiến Trúc Multipart Upload (File Lớn) Cho files lớn hơn 100MB, sử dụng Multipart Upload để upload theo chunks. diff --git a/services/storage-service-net/docs/vi/README.md b/services/storage-service-net/docs/vi/README.md index 49e00c54..dd552c84 100644 --- a/services/storage-service-net/docs/vi/README.md +++ b/services/storage-service-net/docs/vi/README.md @@ -184,6 +184,100 @@ Response: } ``` +## Pre-signed URLs & Access Levels + +Storage Service tạo các loại download URL khác nhau dựa trên `accessLevel` của file. + +### Tổng Quan Access Level + +| Access Level | Storage Path Prefix | Loại Download URL | Use Case | +|--------------|---------------------|-------------------|----------| +| **Public** | `public/` | Direct URL (không có signature) | Assets công khai, hình ảnh, CDN | +| **Private** | `private/` | Pre-signed URL (AWS Signature V4) | Files user, documents | +| **Shared** | `shared/` | Pre-signed URL (AWS Signature V4) | Files chia sẻ qua link | + +### Files Public + +Files public có thể truy cập trực tiếp không cần authentication: + +``` +http://minio:9000/storage/public/2026/01/13/abc123.png +``` + +**Đặc điểm:** +- Không cần signature +- URL không bao giờ hết hạn +- Bất kỳ ai có URL đều có thể truy cập +- Phù hợp cho: avatars, thumbnails, downloads công khai + +### Files Private/Shared (Pre-signed URLs) + +Files private và shared yêu cầu **pre-signed URL** với AWS Signature Version 4: + +``` +http://minio:9000/storage/private/2026/01/13/xyz789.pdf + ?X-Amz-Algorithm=AWS4-HMAC-SHA256 + &X-Amz-Credential=minioadmin%2F20260113%2Fus-east-1%2Fs3%2Faws4_request + &X-Amz-Date=20260113T180024Z + &X-Amz-Expires=3600 + &X-Amz-SignedHeaders=host + &X-Amz-Signature=2ce827d357d105fc1cf88240dee407e5ea72... +``` + +**Giải Thích Các Tham Số Signature:** + +| Tham số | Mô tả | Ví dụ | +|---------|-------|---------| +| `X-Amz-Algorithm` | Thuật toán ký | `AWS4-HMAC-SHA256` | +| `X-Amz-Credential` | Access key + date/region/service | `minioadmin/20260113/us-east-1/s3/aws4_request` | +| `X-Amz-Date` | Thời điểm tạo signature | `20260113T180024Z` | +| `X-Amz-Expires` | Thời hạn URL (giây) | `3600` (1 giờ) | +| `X-Amz-SignedHeaders` | Headers được ký | `host` | +| `X-Amz-Signature` | Chữ ký mã hóa | `2ce827d3...` | + +**Lợi Ích Bảo Mật:** +- URL hết hạn sau thời gian cấu hình (mặc định: 1 giờ) +- Signature ngăn chỉnh sửa URL +- Không lộ credentials MinIO cho client +- Mỗi URL là duy nhất và sử dụng một lần trong thực tế + +### API Lấy Download URL + +```bash +curl -X GET "http://localhost:5002/api/v1/files/{id}/download-url" \ + -H "Authorization: Bearer YOUR_TOKEN" +``` + +**Response cho file Public:** +```json +{ + "success": true, + "data": { + "downloadUrl": "http://minio:9000/storage/public/2026/01/13/abc123.png", + "expiresAt": null + } +} +``` + +**Response cho file Private:** +```json +{ + "success": true, + "data": { + "downloadUrl": "http://minio:9000/storage/private/2026/01/13/xyz789.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=...", + "expiresAt": "2026-01-13T19:00:24Z" + } +} +``` + +### Cấu Hình + +| Biến | Mô tả | Mặc định | +|------|-------|----------| +| `Storage__PreSignedUrlExpirationSeconds` | Thời hạn pre-signed URL | `3600` (1 giờ) | + +> **Lưu ý:** Với files rất lớn hoặc kết nối chậm, hãy cân nhắc tăng thời hạn. + ## Kiến Trúc Logical Folder (Data Sovereignty) > ⚠️ **QUAN TRỌNG**: Theo nguyên tắc **Data Sovereignty** trong microservices, folder là **logical concept trong Database**, KHÔNG phụ thuộc vào bucket structure.