pos-system/docs/vi/architecture/storage-proposal.md

# Đề Xuất Kiến Trúc Storage Service

Tài liệu này mô tả kiến trúc cho Storage Service, microservice quản lý lưu trữ file với hỗ trợ multi-provider.

## Tổng Quan: Storage Service

**Storage Service** cung cấp:
- **Multi-provider Storage** - MinIO (S3-compatible) và Aliyun OSS
- **Direct Client Upload** - Pre-signed URLs để bypass backend (khuyến nghị)
- **Multipart Upload** - Hỗ trợ files lớn > 100MB
- **Logical Folder Architecture** - Folder trong DB, flat storage trong bucket
- **Pre-signed URLs** - Download/Upload URLs có thời hạn với AWS Signature V4
- **User Quota Management** - Giới hạn dung lượng và số file
- **Inter-service Communication** - Xác thực JWT qua IAM Service với caching
- **CQRS Pattern** - MediatR cho Commands/Queries
- **Clean Architecture** - Domain, Infrastructure, API layers

> [!NOTE]
> Storage Service đã được triển khai với .NET 10, Clean Architecture tại `services/storage-service-net/`

---

## 1. Phạm Vi Storage Service

### 1.1 File Management (Quản Lý File)

#### A. Upload Operations ✅
- Direct Client Upload (pre-signed PUT URL) - **Đã triển khai**
- Proxy Upload qua backend - **Đã triển khai**
- Multipart Upload cho files lớn - **Đã triển khai**
- Access Level (Private, Public, Shared) - **Đã triển khai**

#### B. Download Operations ✅
- Pre-signed Download URLs - **Đã triển khai**
- Direct URLs cho Public files - **Đã triển khai**
- AWS Signature V4 security - **Đã triển khai**

#### C. File Lifecycle ✅
- Soft delete - **Đã triển khai**
- File metadata management - **Đã triển khai**
- User ownership validation - **Đã triển khai**

#### D. Folder Management ✅
- ✅ Logical folder hierarchy - **Đã triển khai**
- ✅ Folder CRUD operations - **Đã triển khai**
- ✅ Rename folder (O(1)) - **Đã triển khai**

### 1.2 Storage Infrastructure ✅

#### A. Multi-provider Support
- ✅ MinIO (S3-compatible) - **Đã triển khai**
- ✅ Aliyun OSS - **Đã triển khai**
- ✅ Runtime provider switching - **Đã triển khai**

#### B. Quota Management ✅
- ✅ Per-user storage limits - **Đã triển khai**
- ✅ File count limits - **Đã triển khai**
- ✅ Usage tracking - **Đã triển khai**

### 1.3 Advanced Features ✅

#### A. File Sharing ✅
- ✅ Share links với expiration - **Đã triển khai**
- ✅ Permission-based sharing (View, Download, Edit, Admin) - **Đã triển khai**
- ✅ Password-protected links - **Đã triển khai**
- ✅ Max downloads limit - **Đã triển khai**

#### B. File Versioning ✅
- ✅ Version history - **Đã triển khai**
- ✅ Rollback/Restore support - **Đã triển khai**
- ✅ Download specific version - **Đã triển khai**

#### C. Search & Discovery (Planned)
- 🔄 Full-text search trong metadata
- 🔄 Tag-based filtering
- 🔄 Advanced search queries

---

## 2. Kiến Trúc Module Structure (Thực Tế)

```
services/storage-service-net/
├── src/
│   ├── StorageService.API/              # Presentation Layer
│   │   ├── Controllers/                  # FilesController, SignedUrlController, QuotaController
│   │   ├── Application/                  # CQRS Commands, Queries, Handlers
│   │   │   ├── Commands/                 # UploadFile, SignUpload, ConfirmUpload, Delete
│   │   │   ├── Queries/                  # GetFile, GetFiles, GetDownloadUrl, GetQuota
│   │   │   └── Validators/               # FluentValidation validators
│   │   └── Program.cs                    # App entry point
│   ├── StorageService.Domain/           # Domain Layer
│   │   ├── AggregatesModel/              # StorageFile, UserStorageQuota
│   │   ├── Events/                       # FileUploadedEvent, FileDeletedEvent
│   │   ├── Exceptions/                   # FileNotFoundException, QuotaExceededException
│   │   └── SeedWork/                     # Entity, IAggregateRoot, IRepository
│   └── StorageService.Infrastructure/   # Infrastructure Layer
│       ├── Providers/                    # MinioStorageProvider, AliyunOssStorageProvider
│       ├── StorageServiceContext.cs      # DbContext
│       ├── Repositories/                 # FileRepository, QuotaRepository
│       └── Services/                     # IamServiceClient
├── tests/
│   ├── StorageService.UnitTests/
│   └── StorageService.FunctionalTests/
├── docs/
├── Dockerfile
└── StorageService.slnx
```

### Sơ Đồ Kiến Trúc Clean Architecture

```mermaid
graph TD
    classDef base fill:#202020,stroke:#505050,color:#fff,stroke-width:1px;
    classDef core fill:#1a237e,stroke:#3949ab,color:#fff,stroke-width:1px;
    classDef newModule fill:#1b5e20,stroke:#43a047,color:#fff,stroke-width:1px;
    classDef database fill:#4a148c,stroke:#7b1fa2,color:#fff,stroke-width:1px;
    classDef storage fill:#b71c1c,stroke:#f44336,color:#fff,stroke-width:1px;

    STORAGE[Storage Service]:::core

    subgraph FileOps [File Operations]
        direction TB
        UPLOAD[Upload Handler]:::newModule
        DOWNLOAD[Download Handler]:::newModule
        MULTIPART[Multipart Upload]:::newModule
    end

    subgraph StorageBackend [Storage Backend]
        direction TB
        MINIO[MinIO Provider]:::storage
        OSS[Aliyun OSS Provider]:::storage
    end

    subgraph DataLayer [Data Layer]
        direction TB
        QUOTA[Quota Service]:::newModule
        FOLDER[Folder Service]:::newModule
    end

    DB[(PostgreSQL)]:::database
    IAMSVC[IAM Service]:::core

    STORAGE --> FileOps
    STORAGE --> DataLayer
    FileOps --> StorageBackend
    DataLayer -.-> DB
    STORAGE --> IAMSVC
```

---

## 3. Database Schema

### 3.1 Core Tables (Đã Triển Khai)

- **storage_files**: Metadata files (id, file_name, storage_key, content_type, size, access_level, provider)
- **user_storage_quotas**: Quota per user (max_storage, used_storage, max_files, current_files)

### 3.2 Extended Tables (Đã Triển Khai)

- **folders**: Logical folder hierarchy ✅
- **file_shares**: Share links và permissions ✅
- **file_versions**: Version history ✅
- **multipart_uploads**: Tracking upload sessions ✅
- **multipart_upload_parts**: Individual parts ✅

---

## 4. API Endpoints (Thực Tế)

### 4.1 Direct Upload APIs ✅ (Khuyến Nghị)

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `POST` | `/api/v1/storage/sign-upload` | Lấy pre-signed PUT URL để upload trực tiếp | ✅ |
| `POST` | `/api/v1/storage/confirm-upload` | Xác nhận upload và lưu metadata | ✅ |

### 4.2 Legacy Upload APIs ✅ (Proxy)

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `POST` | `/api/v1/files/upload` | Upload file qua backend (tối đa 100MB) | ✅ |

### 4.3 File Management APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `GET` | `/api/v1/files` | Danh sách files với phân trang | ✅ |
| `GET` | `/api/v1/files/{id}` | Lấy metadata file theo ID | ✅ |
| `GET` | `/api/v1/files/{id}/download-url` | Lấy pre-signed download URL | ✅ |
| `DELETE` | `/api/v1/files/{id}` | Xóa file (soft delete) | ✅ |

### 4.4 Quota APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `GET` | `/api/v1/quota` | Lấy quota storage của user hiện tại | ✅ |

### 4.5 Multipart Upload APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `POST` | `/api/v1/files/multipart/initiate` | Khởi tạo upload session | ✅ |
| `POST` | `/api/v1/files/multipart/upload-part` | Upload 1 chunk | ✅ |
| `POST` | `/api/v1/files/multipart/complete` | Hoàn thành và merge parts | ✅ |
| `DELETE` | `/api/v1/files/multipart/abort` | Hủy upload, cleanup | ✅ |
| `GET` | `/api/v1/files/multipart/{uploadId}` | Kiểm tra tiến độ | ✅ |

### 4.6 Folder APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `POST` | `/api/v1/storage/folders` | Tạo folder mới | ✅ |
| `GET` | `/api/v1/storage/folders` | List folders (root hoặc children) | ✅ |
| `GET` | `/api/v1/storage/folders/{id}` | Lấy folder info | ✅ |
| `PUT` | `/api/v1/storage/folders/{id}` | Đổi tên folder (O(1)) | ✅ |
| `DELETE` | `/api/v1/storage/folders/{id}` | Xóa folder (soft delete) | ✅ |

### 4.7 File Sharing APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `POST` | `/api/v1/storage/files/{id}/shares` | Tạo share link | ✅ |
| `GET` | `/api/v1/storage/files/{id}/shares` | List shares của file | ✅ |
| `DELETE` | `/api/v1/storage/shares/{id}` | Thu hồi share link | ✅ |
| `GET` | `/api/v1/storage/shares/public/{token}` | Truy cập file được chia sẻ (public) | ❌ |

### 4.8 File Versioning APIs ✅

| Method | Endpoint | Mô tả | Auth |
|--------|----------|-------|------|
| `GET` | `/api/v1/storage/files/{id}/versions` | Lấy danh sách versions | ✅ |
| `GET` | `/api/v1/storage/files/{id}/versions/{verId}/download` | Download version cụ thể | ✅ |
| `POST` | `/api/v1/storage/files/{id}/versions/{verId}/restore` | Khôi phục về version cũ | ✅ |

---

## 5. Implementation Roadmap

### Phase 1: Core Storage ✅ (Completed)
- ✅ Multi-provider storage (MinIO, Aliyun OSS)
- ✅ Proxy upload qua backend
- ✅ Pre-signed download URLs
- ✅ User quota management
- ✅ File CRUD operations
- ✅ Inter-service communication với IAM

### Phase 2: Direct Upload ✅ (Completed)
- ✅ Pre-signed PUT URLs cho direct upload
- ✅ Confirm upload flow
- ✅ Access level support (Private, Public, Shared)
- ✅ AWS Signature V4 security

### Phase 3: Large File Support ✅ (Completed)
- ✅ Multipart upload initiation
- ✅ Part-by-part upload
- ✅ Upload completion/abort
- ✅ Progress tracking

### Phase 4: Logical Folders ✅ (Completed)
- ✅ Folder hierarchy trong database
- ✅ Folder CRUD APIs
- ✅ Rename folder O(1)
- ✅ Soft delete

### Phase 5: File Sharing ✅ (Completed)
- ✅ Share link generation
- ✅ Expiring share links
- ✅ Password protection
- ✅ Permission levels (View, Download, Edit, Admin)
- ✅ Max downloads limit

### Phase 6: File Versioning ✅ (Completed)
- ✅ Version tracking
- ✅ Restore/Rollback support
- ✅ Download specific versions

### Phase 7: Search & Discovery (Planned)
- 🔄 Full-text search trong metadata
- 🔄 Tag-based filtering
- 🔄 Advanced search queries

---

## 6. Kiến Trúc Đặc Biệt

### 6.1 Direct Upload Pattern

```mermaid
sequenceDiagram
    participant Client
    participant Storage as Storage Service
    participant MinIO

    Client->>Storage: POST /sign-upload (metadata)
    Storage->>Storage: Validate JWT, Check Quota
    Storage-->>Client: Pre-signed PUT URL + ObjectKey
    Client->>MinIO: PUT file binary (direct)
    MinIO-->>Client: 200 OK
    Client->>Storage: POST /confirm-upload (objectKey)
    Storage->>MinIO: Verify file exists
    Storage->>Storage: Save metadata, Update quota
    Storage-->>Client: File metadata
```

**Lợi ích:**
- Backend không xử lý file binary → Giảm 90%+ tải
- Có thể xử lý hàng triệu uploads đồng thời
- Tốc độ upload nhanh hơn

### 6.2 Logical Folder Architecture

| Layer | Trách nhiệm |
|-------|-------------|
| **Database** | Quản lý folder hierarchy, paths, permissions |
| **Bucket** | Lưu trữ file binary với flat UUID keys |

**Lợi ích:**
- Rename folder: O(1) - chỉ update DB
- Move file: O(1) - chỉ update folder_id
- UUID keys không predictable → Bảo mật cao
- Dễ migrate storage provider

---

## 7. Lợi Ích

### Cho Developers
- ✅ Multi-provider abstraction
- ✅ Clean Architecture
- ✅ CQRS Pattern với MediatR
- ✅ Pre-signed URL handling

### Cho End Users
- ✅ Fast direct uploads
- ✅ Large file support (multipart)
- ✅ Clear quota visibility
- ✅ Secure file access

---

## Kết Luận

Storage Service là một microservice hoàn chỉnh cho file storage với:
- **Direct Upload Pattern** cho high scalability
- **Multi-provider Support** (MinIO, Aliyun OSS)
- **Multipart Upload** cho large files
- **Logical Folder Architecture** theo Data Sovereignty principles
- **Pre-signed URLs** với AWS Signature V4 security

Các tính năng tiếp theo (Folders, Sharing, Versioning) sẽ được triển khai trong các phase tới.