# Storage Service Architecture

> Detailed architecture documentation for the Storage Service microservice.

## Architecture Overview

```mermaid
graph TB
    subgraph "API Layer"
        C[Controllers]
        CMD[Commands]
        Q[Queries]
        B[Behaviors]
    end
    
    subgraph "Domain Layer"
        SF[StorageFile]
        SQ[UserStorageQuota]
        DE[Domain Events]
        RI[Repository Interfaces]
    end
    
    subgraph "Infrastructure Layer"
        SP[Storage Providers]
        IAM[IAM Client]
        R[Repositories]
        CTX[DbContext]
    end
    
    subgraph "External Services"
        MINIO[(MinIO)]
        OSS[(Aliyun OSS)]
        IAMS[IAM Service]
        DB[(PostgreSQL)]
    end
    
    C --> CMD
    C --> Q
    CMD --> B --> SF
    Q --> R
    R --> CTX --> DB
    SF --> DE
    
    CMD --> SP
    SP --> MINIO
    SP --> OSS
    
    C --> IAM --> IAMS
    
    style C fill:#4a90d9,stroke:#2d5986,color:#fff
    style SF fill:#50c878,stroke:#2d8659,color:#fff
    style MINIO fill:#c73b3b,stroke:#922b2b,color:#fff
    style OSS fill:#ff6b35,stroke:#cc5500,color:#fff
    style IAMS fill:#9b59b6,stroke:#7d3c98,color:#fff
```

## Layer Responsibilities

### 1. Domain Layer (StorageService.Domain)

The heart of the application containing pure business logic.

| Component | Purpose |
|-----------|---------|
| **StorageFile** | Aggregate root for file metadata and lifecycle |
| **UserStorageQuota** | Aggregate root for user storage limits and usage |
| **StorageProvider** | Enum: MinIO, AliyunOSS |
| **FileAccessLevel** | Enum: Private, Public, Shared |
| **Domain Events** | FileUploadedDomainEvent, FileDeletedDomainEvent, UserQuotaUpdatedDomainEvent |

### 2. Infrastructure Layer (StorageService.Infrastructure)

Technical implementations and external integrations:

| Component | Purpose |
|-----------|---------|
| **MinioStorageProvider** | MinIO S3-compatible storage operations |
| **AliyunOssStorageProvider** | Alibaba Cloud OSS operations |
| **StorageProviderFactory** | Runtime provider selection based on config |
| **HttpIamServiceClient** | Inter-service communication with IAM |
| **FileRepository** | EF Core repository for StorageFile |
| **QuotaRepository** | EF Core repository for UserStorageQuota |

### 3. API Layer (StorageService.API)

Application entry point and CQRS implementation:

| Component | Purpose |
|-----------|---------|
| **FilesController** | File CRUD endpoints (legacy proxy upload) |
| **SignedUrlController** | Direct upload endpoints (recommended) |
| **QuotaController** | User quota endpoints |
| **SignUploadCommand** | Generate pre-signed upload URLs |
| **ConfirmUploadCommand** | Confirm direct uploads and save metadata |
| **UploadFileCommand** | Handle proxy file uploads (legacy) |
| **DeleteFileCommand** | Handle file deletions |
| **Query Handlers** | Handle read operations |

### 4. Admin Backoffice Layer

Controllers and Commands for Admin storage management:

| Component | Purpose |
|-----------|---------|
| **AdminQuotaController** | Manage user quotas (GET all, PUT update) |
| **AdminFilesController** | View/delete all users' files |
| **AdminSharesController** | View/revoke violating file shares |
| **AdminStatisticsController** | Storage statistics dashboard |
| **UpdateUserQuotaCommand** | Admin update quota limits |
| **AdminDeleteFileCommand** | Admin delete file (bypass ownership) |
| **AdminRevokeShareCommand** | Admin revoke violating share |
| **Admin Query Handlers** | Queries with pagination and filtering |

> **Authorization**: All Admin endpoints require `Admin` or `SuperAdmin` role.

## Direct Upload Architecture (Recommended)

For systems with millions of users, Direct Client Upload pattern is recommended over proxy upload.

### Upload Patterns Comparison

| Aspect | Proxy Upload (Legacy) | Direct Upload (Recommended) |
|--------|----------------------|----------------------------|
| **Throughput** | ~100-500/sec | ~10,000+/sec |
| **Memory per request** | 100MB (file size) | ~10KB (metadata only) |
| **Latency (100MB file)** | 30-60s | 10-20s |
| **Backend load** | High | Minimal |

### Direct Upload Flow

```mermaid
sequenceDiagram
    participant Client
    participant Storage_Service as Storage Service
    participant MinIO
    
    rect rgb(200, 230, 200)
        Note over Client,Storage_Service: 1. Request Upload URL (lightweight)
        Client->>Storage_Service: POST /api/v1/storage/sign-upload
        Storage_Service->>Storage_Service: Validate JWT, Check Quota
        Storage_Service-->>Client: Pre-signed PUT URL + ObjectKey
    end
    
    rect rgb(200, 200, 230)
        Note over Client,MinIO: 2. Direct Upload (bypasses backend)
        Client->>MinIO: PUT file binary to Pre-signed URL
        MinIO-->>Client: 200 OK
    end
    
    rect rgb(230, 230, 200)
        Note over Client,Storage_Service: 3. Confirm Upload (lightweight)
        Client->>Storage_Service: POST /api/v1/storage/confirm-upload
        Storage_Service->>MinIO: Verify file exists
        Storage_Service->>Storage_Service: Save metadata, Update quota
        Storage_Service-->>Client: File metadata
    end
```

### Logical Folder Architecture (Data Sovereignty)

> ⚠️ **IMPORTANT**: Following the **Data Sovereignty** principle in microservices, Storage Service must fully own its data model.

#### ❌ Anti-pattern: Relying on Bucket Structure

```
BAD APPROACH - Folder structure reflected in bucket:
storage-bucket/
├── users/john/documents/report.pdf
├── users/john/images/photo.jpg
└── users/mary/work/presentation.pptx

PROBLEMS:
- Renaming folder "documents" → "docs" = Moving millions of files (O(n))
- Moving file = Copy + Delete on bucket (slow, risky)
- Predictable paths → Vulnerable to path traversal attacks
- Doesn't scale with millions of users
- Difficult to migrate to another storage provider
```

#### ✅ Correct Approach: Logical Separation

**Principles:**
1. **Database** = Logical structure (folders, hierarchy, permissions)
2. **Bucket** = Physical storage (flat UUID keys)

```mermaid
graph TB
    subgraph "Logical Layer - PostgreSQL Database"
        F[Folders Table]
        FL[Files Table]
        F -->|parent_id| F
        FL -->|folder_id| F
    end
    
    subgraph "Physical Layer - MinIO Bucket"
        B[Flat UUID Structure]
        B1[private/2026/01/13/uuid1.pdf]
        B2[private/2026/01/13/uuid2.jpg]
        B3[public/2026/01/14/uuid3.png]
    end
    
    FL -.->|storage_key| B
    
    style F fill:#3498db,color:#fff
    style FL fill:#2ecc71,color:#fff
    style B fill:#e74c3c,color:#fff
```

#### Database Schema

```sql
-- Folders: Hierarchical tree structure
CREATE TABLE folders (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id VARCHAR(255) NOT NULL,
    parent_id UUID REFERENCES folders(id) ON DELETE CASCADE,
    name VARCHAR(255) NOT NULL,
    path VARCHAR(1000) NOT NULL,        -- Materialized path: /docs/work/2024
    level INT NOT NULL DEFAULT 0,       -- Tree depth
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW(),
    
    UNIQUE (user_id, parent_id, name),
    INDEX idx_user_path (user_id, path)
);

-- Files: Link to logical folders
CREATE TABLE storage_files (
    id UUID PRIMARY KEY,
    user_id VARCHAR(255) NOT NULL,
    folder_id UUID REFERENCES folders(id) ON DELETE SET NULL,
    file_name VARCHAR(255) NOT NULL,
    storage_key VARCHAR(500) UNIQUE,    -- Physical UUID key in bucket
    content_type VARCHAR(100),
    file_size_bytes BIGINT,
    access_level VARCHAR(20),           -- Private, Public, Shared
    bucket_name VARCHAR(255),
    provider INT,
    uploaded_at TIMESTAMP,
    is_deleted BOOLEAN DEFAULT false,
    
    INDEX idx_folder_id (folder_id),
    INDEX idx_storage_key (storage_key)
);
```

#### Physical Key Generation Strategy

```csharp
public class StorageKeyGenerator
{
    /// <summary>
    /// Generate UUID-based key (NOT based on folder path)
    /// Pattern: {prefix}/{year}/{month}/{day}/{uuid}.{ext}
    /// </summary>
    public string GenerateKey(FileAccessLevel access, string fileName)
    {
        var now = DateTime.UtcNow;
        var prefix = access == FileAccessLevel.Public ? "public" : "private";
        var uuid = Guid.NewGuid().ToString("N"); // 32 chars, no hyphens
        var ext = Path.GetExtension(fileName);
        
        // Example: private/2026/01/13/d290f1ee6c544b0190e6d701748f0851.pdf
        return $"{prefix}/{now:yyyy}/{now:MM}/{now:dd}/{uuid}{ext}";
    }
}
```

#### Bucket Structure (Flat Physical)

```
goodgo/
├── private/
│   ├── 2026/
│   │   └── 01/
│   │       └── 13/
│   │           ├── d290f1ee6c544b0190e6d701748f0851.pdf
│   │           ├── a7f3b2c19d8e4f01bcd5e92f7a4d8b63.jpg
│   │           └── f9e4c1a2b7d36e5f8a0c4d9e2b1f7a8c.docx
├── public/
│   └── 2026/
│       └── 01/
│           └── 14/
│               └── c3d8f2e1a9b47c6e5d0f8a3b2e1c7d9f.png
└── shared/
    └── 2026/
        └── 01/
            └── 15/
                └── e1f2c3d4a5b6c7e8d9f0a1b2c3d4e5f6.xlsx

NO user/documents/... folders in bucket!
```

#### Workflows

**1. Create Folder (Database Only)**
```mermaid
sequenceDiagram
    Client->>API: POST /api/v1/folders {name: "documents"}
    API->>Database: INSERT INTO folders (name, path)
    Database-->>API: folder_id
    API-->>Client: {id, name, path: "/documents"}
    Note over Client,API: Bucket is NOT touched
```

**2. Upload File to Folder**
```mermaid
sequenceDiagram
    Client->>API: POST /sign-upload {fileName, folderId}
    API->>Database: Validate folder ownership
    API->>StorageKeyGen: Generate UUID key
    StorageKeyGen-->>API: private/2026/01/13/{uuid}.pdf
    API->>MinIO: Get pre-signed PUT URL
    MinIO-->>API: pre-signed URL
    API->>Database: INSERT file (folder_id, storage_key)
    API-->>Client: {uploadUrl, objectKey}
    Client->>MinIO: PUT file binary
    Client->>API: POST /confirm-upload
```

**3. List Files in Folder**
```sql
-- Database query (fast, indexed)
SELECT * FROM storage_files 
WHERE folder_id = '{folder-uuid}' 
  AND is_deleted = false
ORDER BY uploaded_at DESC;

-- Result returned to client:
{
  "folder": {
    "id": "folder-123",
    "path": "/documents/work"
  },
  "files": [
    {
      "id": "file-456",
      "name": "report.pdf",
      "logicalPath": "/documents/work/report.pdf",  -- Displayed to user
      "storageKey": "private/2026/01/13/{uuid}.pdf" -- Physical key
    }
  ]
}
```

**4. Rename Folder**
```sql
-- Only UPDATE database (instant, O(1))
UPDATE folders 
SET name = 'docs', 
    path = '/docs'  -- Update materialized path
WHERE id = '{folder-id}';

-- Update descendant paths
UPDATE folders
SET path = REPLACE(path, '/documents', '/docs')
WHERE path LIKE '/documents/%';

-- Files KEEP their storage_key - DON'T TOUCH bucket!
```

**5. Move File Between Folders**
```sql
-- Only UPDATE database (instant, O(1))
UPDATE storage_files
SET folder_id = '{new-folder-id}'
WHERE id = '{file-id}';

-- Physical file STAYS at old storage_key
-- No Copy/Delete needed in bucket
```

#### Performance Comparison

| Operation | Anti-pattern (Bucket-based) | Correct (Logical) | Improvement |
|-----------|----------------------------|-------------------|-------------|
| **Create folder** | N/A (virtual) | INSERT to DB | Instant |
| **Rename folder** | Copy + Delete millions of files | UPDATE 1-N rows in DB | ~1000x faster |
| **Move file** | Copy + Delete 1 file | UPDATE 1 row in DB | ~100x faster |
| **List files** | Bucket prefix scan | Indexed DB query | ~50x faster |
| **Delete folder** | Delete millions of files | Cascade delete + background cleanup | Async |

#### Data Sovereignty Benefits

1. **Performance:**
   - Rename folder: O(1) instead of O(n)
   - Move file: O(1) instead of O(1) bucket copy
   - List files: Indexed query instead of bucket scan

2. **Security:**
   - UUID keys are unpredictable
   - No path traversal vulnerabilities
   - Access control in database

3. **Scalability:**
   - Database handles metadata (fast)
   - Bucket handles blobs (simple)
   - Easy horizontal scaling

4. **Flexibility:**
   - Easy to add features (sharing, versioning)
   - Migration between storage providers
   - Support multiple buckets/regions

5. **Developer Experience:**
   - Client sees beautiful folder tree
   - Backend works with simple UUIDs
   - Clean separation of concerns

### Direct Upload Components

| Component | Purpose |
|-----------|---------|
| **SignUploadCommand** | Validate quota, generate object key with path prefix, create pre-signed PUT URL |
| **SignUploadCommandHandler** | Handle sign-upload requests |
| **ConfirmUploadCommand** | Verify file exists, save metadata, update quota |
| **ConfirmUploadCommandHandler** | Handle confirm-upload with idempotency |
| **SignedUrlController** | `/sign-upload` and `/confirm-upload` endpoints |

## Download URL Architecture

The Storage Service generates different download URLs based on file access levels.

### URL Generation Flow

```mermaid
sequenceDiagram
    participant Client
    participant API as Storage Service
    participant MinIO

    Client->>API: GET /api/v1/files/{id}/download-url
    API->>API: Validate ownership
    
    alt accessLevel = Public
        API-->>Client: Direct URL (no signature)
    else accessLevel = Private/Shared
        API->>MinIO: GetPreSignedDownloadUrl()
        MinIO-->>API: Pre-signed URL (AWS Signature V4)
        API-->>Client: Pre-signed URL with expiration
    end
```

### Access Level URL Types

| Access Level | Storage Prefix | URL Type | Expiration |
|--------------|----------------|----------|------------|
| **Public** | `public/` | Direct URL | Never |
| **Private** | `private/` | Pre-signed URL | Configurable (default: 1 hour) |
| **Shared** | `shared/` | Pre-signed URL | Configurable (default: 1 hour) |

### Pre-signed URL Structure

For private/shared files, URLs include AWS Signature Version 4 parameters:

```
http://minio:9000/bucket/private/2026/01/13/{uuid}.pdf
  ?X-Amz-Algorithm=AWS4-HMAC-SHA256
  &X-Amz-Credential={accessKey}%2F{date}%2F{region}%2Fs3%2Faws4_request
  &X-Amz-Date={timestamp}
  &X-Amz-Expires={seconds}
  &X-Amz-SignedHeaders=host
  &X-Amz-Signature={signature}
```

| Parameter | Purpose |
|-----------|---------|
| `X-Amz-Algorithm` | Signing algorithm (always AWS4-HMAC-SHA256) |
| `X-Amz-Credential` | Access key + scope (date/region/service) |
| `X-Amz-Date` | Timestamp when signature was created |
| `X-Amz-Expires` | URL validity in seconds |
| `X-Amz-SignedHeaders` | Headers included in signature calculation |
| `X-Amz-Signature` | Cryptographic signature to verify URL integrity |

### Security Considerations

1. **Time-Limited Access**: Pre-signed URLs expire after configured time
2. **Tamper-Proof**: Any URL modification invalidates the signature
3. **Credential Protection**: MinIO access keys never exposed to client
4. **Unique URLs**: Each request generates a new signature

### GetDownloadUrl Implementation

```csharp
public async Task<string> GetDownloadUrlAsync(StorageFile file)
{
    if (file.AccessLevel == FileAccessLevel.Public)
    {
        // Direct URL for public files
        return $"{_settings.PublicEndpoint}/{file.BucketName}/{file.ObjectKey}";
    }
    
    // Pre-signed URL for private/shared files
    return await _storageProvider.GetPreSignedDownloadUrlAsync(
        file.BucketName,
        file.ObjectKey,
        _settings.PreSignedUrlExpirationSeconds
    );
}
```

## Multipart Upload Architecture (Large Files)

For files larger than 100MB, use Multipart Upload to upload in chunks.

### Upload Methods Comparison

| Aspect | Direct Upload | Multipart Upload |
|--------|---------------|------------------|
| **File size** | < 100MB | > 100MB (up to 5GB+) |
| **Mechanism** | Single PUT request | Multiple part uploads |
| **Resume support** | No | Yes (per part) |
| **Progress tracking** | No | Yes (per part) |
| **Use case** | Small/medium files | Large files, video, archives |

### Multipart Upload Flow

```mermaid
sequenceDiagram
    participant Client
    participant API as Storage Service
    participant DB as PostgreSQL
    participant MinIO

    rect rgb(200, 230, 200)
        Note over Client,API: 1. Initiate Upload
        Client->>API: POST /api/v1/files/multipart/initiate
        API->>DB: Create MultipartUpload record
        API->>MinIO: InitiateMultipartUpload
        MinIO-->>API: Provider UploadId
        API-->>Client: {uploadId, objectKey, totalChunks}
    end

    rect rgb(200, 200, 230)
        Note over Client,MinIO: 2. Upload Parts (repeat for each chunk)
        loop For each part 1..N
            Client->>API: POST /api/v1/files/multipart/upload-part
            API->>MinIO: UploadPart(partNumber, data)
            MinIO-->>API: ETag
            API->>DB: Save part info (partNumber, etag)
            API-->>Client: {success, etag}
        end
    end

    rect rgb(230, 200, 200)
        Note over Client,API: 3. Optional: Check Progress
        Client->>API: GET /api/v1/files/multipart/{uploadId}
        API->>DB: Get upload + parts
        API-->>Client: {progress: 75%, uploadedChunks: 3/4}
    end

    rect rgb(230, 230, 200)
        Note over Client,API: 4. Complete Upload
        Client->>API: POST /api/v1/files/multipart/complete
        API->>MinIO: CompleteMultipartUpload(parts[])
        MinIO-->>API: OK
        API->>DB: Create StorageFile, Update quota
        API-->>Client: {fileId, objectKey}
    end
```

### Multipart Upload Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/files/multipart/initiate` | Start upload session |
| `POST` | `/api/v1/files/multipart/upload-part` | Upload 1 chunk |
| `POST` | `/api/v1/files/multipart/complete` | Complete and merge parts |
| `DELETE` | `/api/v1/files/multipart/abort` | Cancel upload, cleanup |
| `GET` | `/api/v1/files/multipart/{uploadId}` | Check progress |

### Multipart Upload Components

| Component | Purpose |
|-----------|---------|
| **InitiateMultipartUploadCommand** | Create upload session, generate object key |
| **UploadPartCommand** | Upload 1 part to storage provider |
| **CompleteMultipartUploadCommand** | Merge parts, create StorageFile record |
| **AbortMultipartUploadCommand** | Cleanup parts, mark as aborted |
| **GetMultipartUploadProgressQuery** | Get upload progress |
| **MultipartUploadController** | API endpoints for multipart |

## Storage Provider Architecture

```mermaid
graph TD
    subgraph "Storage Provider Factory"
        F[StorageProviderFactory]
        C[Configuration]
    end
    
    subgraph "Providers"
        MP[MinioStorageProvider]
        AP[AliyunOssStorageProvider]
    end
    
    subgraph "Storage Backends"
        MINIO[(MinIO)]
        OSS[(Aliyun OSS)]
    end
    
    C --> |STORAGE_PROVIDER=minio| F
    C --> |STORAGE_PROVIDER=aliyun| F
    F --> |GetProvider| MP
    F --> |GetProvider| AP
    MP --> MINIO
    AP --> OSS
    
    style F fill:#4a90d9,stroke:#2d5986,color:#fff
    style MP fill:#c73b3b,stroke:#922b2b,color:#fff
    style AP fill:#ff6b35,stroke:#cc5500,color:#fff
```

### Storage Provider Interface

```csharp
public interface IStorageProvider
{
    Task<UploadResult> UploadAsync(Stream stream, string objectKey, ...);
    Task<Stream> DownloadAsync(string bucketName, string objectKey);
    Task DeleteAsync(string bucketName, string objectKey);
    Task<bool> ExistsAsync(string bucketName, string objectKey);
    Task<string> GetPreSignedDownloadUrlAsync(string bucketName, string objectKey, int expirationSeconds);
    Task<string> GetPreSignedUploadUrlAsync(string bucketName, string objectKey, int expirationSeconds);
}
```

## Inter-Service Communication

```mermaid
sequenceDiagram
    participant Client
    participant Storage as Storage Service
    participant Cache as In-Memory Cache
    participant IAM as IAM Service
    
    Client->>Storage: Upload File (JWT)
    Storage->>Cache: Check user cache
    alt Cache Hit
        Cache-->>Storage: User info
    else Cache Miss
        Storage->>IAM: GET /api/v1/users/me
        Note over Storage,IAM: Headers: Authorization, X-Service-Name
        IAM-->>Storage: User info
        Storage->>Cache: Store (5 min TTL)
    end
    Storage->>Storage: Validate quota
    Storage->>Storage: Upload to provider
    Storage-->>Client: Upload result
```

### IAM Client Features

| Feature | Description |
|---------|-------------|
| **Caching** | In-memory cache for user info (5 min TTL) |
| **Health Check** | Check IAM availability with caching (1 min TTL) |
| **Polly Resilience** | Retry (3x exponential) + Circuit Breaker |
| **Permission Check** | HasPermissionAsync, HasRoleAsync |

### Available IAM Client Methods

```csharp
// User Operations
Task<IamUserInfo?> ValidateUserAsync(string accessToken);
Task<IamUserInfo?> GetUserByIdAsync(string userId, string accessToken);
Task<bool> UserExistsAsync(string userId, string accessToken);
Task<IReadOnlyList<string>> GetUserRolesAsync(string userId, string accessToken);
Task<IReadOnlyList<string>> GetUserPermissionsAsync(string userId, string accessToken);
Task<bool> HasPermissionAsync(string userId, string permission, string accessToken);
Task<bool> HasRoleAsync(string userId, string role, string accessToken);

// Health Check
Task<IamHealthStatus> CheckHealthAsync();
Task<bool> IsAvailableAsync();

// Cache Management
void InvalidateUserCache(string userId);
void ClearCache();
```

## Database Schema

```mermaid
erDiagram
    storage_files {
        uuid id PK
        varchar file_name
        varchar bucket_name
        varchar object_key
        varchar content_type
        bigint file_size_bytes
        varchar user_id
        varchar tenant_id
        int provider
        int access_level
        timestamp uploaded_at
        timestamp expires_at
        varchar checksum
        boolean is_deleted
        timestamp deleted_at
    }
    
    user_storage_quotas {
        uuid id PK
        varchar user_id UK
        bigint max_storage_bytes
        bigint used_storage_bytes
        int max_file_count
        int current_file_count
        varchar quota_tier
        timestamp last_updated_at
        timestamp created_at
    }
```

## API Endpoints

### Files (Legacy - Proxy Upload)

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/files/upload` | Upload file via backend (max 100MB) |
| `GET` | `/api/v1/files` | List user files with pagination |
| `GET` | `/api/v1/files/{id}` | Get file metadata |
| `GET` | `/api/v1/files/{id}/download-url` | Get pre-signed download URL |
| `DELETE` | `/api/v1/files/{id}` | Delete file (soft delete) |

### Direct Upload (Recommended)

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/storage/sign-upload` | Get pre-signed PUT URL for direct upload |
| `POST` | `/api/v1/storage/confirm-upload` | Confirm upload and save metadata |

### Quota

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/quota` | Get user storage quota |

### Admin API (Requires Role: Admin)

#### Quota Management

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/quotas` | Get all users' quotas |
| `GET` | `/api/v1/admin/quotas/{userId}` | Get quota for specific user |
| `PUT` | `/api/v1/admin/quotas/{userId}` | Update quota limits |

#### Files Management

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/files` | Get all files with filtering |
| `DELETE` | `/api/v1/admin/files/{id}` | Delete file for policy violation |

#### Shares Management

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/shares` | Get all shares |
| `DELETE` | `/api/v1/admin/shares/{id}` | Revoke share for violation |

#### Statistics

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/admin/statistics` | Dashboard aggregated statistics |
| `GET` | `/api/v1/admin/statistics/users-near-limit` | Users near quota limit (>80%) |

## File Versioning Architecture

The Storage Service maintains version history for files, allowing users to track changes and restore previous versions.

### Version Flow

```mermaid
sequenceDiagram
    participant Client
    participant API as Storage Service
    participant DB as PostgreSQL
    participant MinIO

    rect rgb(44, 62, 80)
        Note over Client,API: 1. Upload New Version
        Client->>API: PUT /files/{id} (new file)
        API->>DB: Create FileVersion (version N+1)
        API->>MinIO: Store new object with version key
        API->>DB: Update StorageFile.ObjectKey
        API-->>Client: {versionNumber, fileId}
    end

    rect rgb(39, 174, 96)
        Note over Client,API: 2. List Versions
        Client->>API: GET /files/{id}/versions
        API->>DB: SELECT * FROM file_versions WHERE file_id = ?
        API-->>Client: [{version: 1, ...}, {version: 2, ...}]
    end

    rect rgb(230, 126, 34)
        Note over Client,API: 3. Restore Version
        Client->>API: POST /files/{id}/versions/{N}/restore
        API->>DB: Get FileVersion N
        API->>DB: Update StorageFile from version
        API-->>Client: {restored: true, versionNumber: N}
    end
```

### FileVersion Entity

| Property | Type | Description |
|----------|------|-------------|
| `Id` | `Guid` | Version ID |
| `FileId` | `Guid` | Parent file reference |
| `VersionNumber` | `int` | Sequential version number |
| `ObjectKey` | `string` | Storage key for this version |
| `SizeBytes` | `long` | File size for this version |
| `ContentType` | `string` | MIME type |
| `Checksum` | `string?` | Optional hash |
| `CreatedAt` | `DateTime` | Version creation time |
| `CreatedBy` | `string` | User who created version |
| `IsCurrent` | `bool` | Whether this is the active version |

---

## File Sharing Architecture

The Storage Service provides secure file sharing via token-based access with optional password protection, expiration, and download limits.

### Share Flow

```mermaid
sequenceDiagram
    participant Owner
    participant API as Storage Service
    participant DB as PostgreSQL
    participant Guest

    rect rgb(44, 62, 80)
        Note over Owner,API: 1. Create Share
        Owner->>API: POST /storage/files/{id}/shares
        API->>DB: Create FileShare (token, options)
        API-->>Owner: {shareToken, shareUrl}
    end

    rect rgb(39, 174, 96)
        Note over Guest,API: 2. Access Share (Public)
        Guest->>API: GET /storage/shares/public/{token}
        API->>DB: Validate share (expired? revoked? limit?)
        API->>DB: Optional password check
        API->>DB: Increment download count
        API-->>Guest: {downloadUrl, fileMetadata}
    end

    rect rgb(192, 57, 43)
        Note over Owner,API: 3. Revoke Share
        Owner->>API: DELETE /storage/shares/{id}
        API->>DB: Set status = Revoked
        API-->>Owner: 204 No Content
    end
```

### FileShare Entity

| Property | Type | Description |
|----------|------|-------------|
| `Id` | `Guid` | Share ID |
| `FileId` | `Guid` | File being shared |
| `SharedBy` | `string` | User who created share |
| `SharedWith` | `string?` | Specific user (null = public link) |
| `Permission` | `SharePermission` | View, Download, Edit, Admin |
| `ShareToken` | `string` | Unique URL-safe token |
| `PasswordHash` | `string?` | Optional password (PBKDF2) |
| `ExpiresAt` | `DateTime?` | Optional expiration |
| `MaxDownloads` | `int?` | Optional download limit |
| `DownloadCount` | `int` | Current download count |
| `Status` | `FileShareStatus` | Active, Expired, Revoked, LimitReached |

### Share Permissions

| Permission | Can View | Can Download | Can Edit | Can Manage |
|------------|----------|--------------|----------|------------|
| **View** | ✅ | ❌ | ❌ | ❌ |
| **Download** | ✅ | ✅ | ❌ | ❌ |
| **Edit** | ✅ | ✅ | ✅ | ❌ |
| **Admin** | ✅ | ✅ | ✅ | ✅ |

---

## Health Checks

```mermaid
graph TD
    HC[Health Check Endpoint]
    HC --> |/health/live| L[Liveness]
    HC --> |/health/ready| R[Readiness]
    
    R --> DB[(PostgreSQL)]
    R --> MINIO[(MinIO)]
    R --> IAM[IAM Service]
    
    style HC fill:#2C3E50,color:#ECF0F1,stroke:#34495E,stroke-width:3px
    style L fill:#27AE60,color:#ECF0F1,stroke:#229954,stroke-width:2px
    style R fill:#E67E22,color:#ECF0F1,stroke:#D35400,stroke-width:2px
    style DB fill:#34495E,color:#ECF0F1,stroke:#2C3E50,stroke-width:2px
    style MINIO fill:#C0392B,color:#ECF0F1,stroke:#A93226,stroke-width:2px
    style IAM fill:#8E44AD,color:#ECF0F1,stroke:#7D3C98,stroke-width:2px
```

## Deployment Architecture

### Docker Compose (Local)

```yaml
services:
  storage-api:
    build: .
    ports: ["5002:8080"]
    depends_on:
      - postgres
      - redis
      - minio
    environment:
      - Storage__Provider=minio
      - Storage__MinIO__Endpoint=minio:9000
  
  minio:
    image: minio/minio:latest
    ports: ["9000:9000", "9001:9001"]
```

### Traefik Integration

```yaml
labels:
  - "traefik.enable=true"
  - "traefik.http.routers.storage-service.rule=PathPrefix(`/api/v1/files`) || PathPrefix(`/api/v1/quota`)"
  - "traefik.http.services.storage-service.loadbalancer.server.port=8080"
```

## Security Considerations

1. **Authentication**: JWT Bearer validation via IAM Service
2. **Authorization**: User ownership check on files
3. **Input Validation**: File size limits, content type validation
4. **Pre-signed URLs**: Time-limited access to files
5. **Soft Delete**: Files are marked deleted, not immediately removed

## Performance Optimization

1. **Caching**: In-memory cache for IAM user info
2. **Pre-signed URLs**: Direct client-to-storage downloads
3. **Streaming Upload**: Stream-based file handling
4. **Async Operations**: All I/O operations are async
5. **Connection Pooling**: HTTP client with Polly policies

## References

- [MinIO Documentation](https://min.io/docs/minio/)
- [Aliyun OSS Documentation](https://www.alibabacloud.com/help/oss)
- [Polly Resilience](https://github.com/App-vNext/Polly)
- [Clean Architecture](https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html)