admin/pos-system
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
48bb30b0096b4df60e94b563376825f9cc2ca921
pos-system/services/storage-service-net/docs/en/ARCHITECTURE.md

30 KiB
Raw Blame History

Storage Service Architecture

Detailed architecture documentation for the Storage Service microservice.

Architecture Overview

graph TB
    subgraph "API Layer"
        C[Controllers]
        CMD[Commands]
        Q[Queries]
        B[Behaviors]
    end
    
    subgraph "Domain Layer"
        SF[StorageFile]
        SQ[UserStorageQuota]
        DE[Domain Events]
        RI[Repository Interfaces]
    end
    
    subgraph "Infrastructure Layer"
        SP[Storage Providers]
        IAM[IAM Client]
        R[Repositories]
        CTX[DbContext]
    end
    
    subgraph "External Services"
        MINIO[(MinIO)]
        OSS[(Aliyun OSS)]
        IAMS[IAM Service]
        DB[(PostgreSQL)]
    end
    
    C --> CMD
    C --> Q
    CMD --> B --> SF
    Q --> R
    R --> CTX --> DB
    SF --> DE
    
    CMD --> SP
    SP --> MINIO
    SP --> OSS
    
    C --> IAM --> IAMS
    
    style C fill:#4a90d9,stroke:#2d5986,color:#fff
    style SF fill:#50c878,stroke:#2d8659,color:#fff
    style MINIO fill:#c73b3b,stroke:#922b2b,color:#fff
    style OSS fill:#ff6b35,stroke:#cc5500,color:#fff
    style IAMS fill:#9b59b6,stroke:#7d3c98,color:#fff

Layer Responsibilities

1. Domain Layer (StorageService.Domain)

The heart of the application containing pure business logic.

Component Purpose
StorageFile Aggregate root for file metadata and lifecycle
UserStorageQuota Aggregate root for user storage limits and usage
StorageProvider Enum: MinIO, AliyunOSS
FileAccessLevel Enum: Private, Public, Shared
Domain Events FileUploadedDomainEvent, FileDeletedDomainEvent, UserQuotaUpdatedDomainEvent

2. Infrastructure Layer (StorageService.Infrastructure)

Technical implementations and external integrations:

Component Purpose
MinioStorageProvider MinIO S3-compatible storage operations
AliyunOssStorageProvider Alibaba Cloud OSS operations
StorageProviderFactory Runtime provider selection based on config
HttpIamServiceClient Inter-service communication with IAM
FileRepository EF Core repository for StorageFile
QuotaRepository EF Core repository for UserStorageQuota

3. API Layer (StorageService.API)

Application entry point and CQRS implementation:

Component Purpose
FilesController File CRUD endpoints (legacy proxy upload)
SignedUrlController Direct upload endpoints (recommended)
QuotaController User quota endpoints
SignUploadCommand Generate pre-signed upload URLs
ConfirmUploadCommand Confirm direct uploads and save metadata
UploadFileCommand Handle proxy file uploads (legacy)
DeleteFileCommand Handle file deletions
Query Handlers Handle read operations

4. Admin Backoffice Layer

Controllers and Commands for Admin storage management:

Component Purpose
AdminQuotaController Manage user quotas (GET all, PUT update)
AdminFilesController View/delete all users' files
AdminSharesController View/revoke violating file shares
AdminStatisticsController Storage statistics dashboard
UpdateUserQuotaCommand Admin update quota limits
AdminDeleteFileCommand Admin delete file (bypass ownership)
AdminRevokeShareCommand Admin revoke violating share
Admin Query Handlers Queries with pagination and filtering

Authorization: All Admin endpoints require Admin or SuperAdmin role.

Direct Upload Architecture (Recommended)

For systems with millions of users, Direct Client Upload pattern is recommended over proxy upload.

Upload Patterns Comparison

Aspect Proxy Upload (Legacy) Direct Upload (Recommended)
Throughput ~100-500/sec ~10,000+/sec
Memory per request 100MB (file size) ~10KB (metadata only)
Latency (100MB file) 30-60s 10-20s
Backend load High Minimal

Direct Upload Flow

sequenceDiagram
    participant Client
    participant Storage_Service as Storage Service
    participant MinIO
    
    rect rgb(200, 230, 200)
        Note over Client,Storage_Service: 1. Request Upload URL (lightweight)
        Client->>Storage_Service: POST /api/v1/storage/sign-upload
        Storage_Service->>Storage_Service: Validate JWT, Check Quota
        Storage_Service-->>Client: Pre-signed PUT URL + ObjectKey
    end
    
    rect rgb(200, 200, 230)
        Note over Client,MinIO: 2. Direct Upload (bypasses backend)
        Client->>MinIO: PUT file binary to Pre-signed URL
        MinIO-->>Client: 200 OK
    end
    
    rect rgb(230, 230, 200)
        Note over Client,Storage_Service: 3. Confirm Upload (lightweight)
        Client->>Storage_Service: POST /api/v1/storage/confirm-upload
        Storage_Service->>MinIO: Verify file exists
        Storage_Service->>Storage_Service: Save metadata, Update quota
        Storage_Service-->>Client: File metadata
    end

Logical Folder Architecture (Data Sovereignty)

⚠️ IMPORTANT: Following the Data Sovereignty principle in microservices, Storage Service must fully own its data model.

❌ Anti-pattern: Relying on Bucket Structure

BAD APPROACH - Folder structure reflected in bucket:
storage-bucket/
├── users/john/documents/report.pdf
├── users/john/images/photo.jpg
└── users/mary/work/presentation.pptx

PROBLEMS:
- Renaming folder "documents" → "docs" = Moving millions of files (O(n))
- Moving file = Copy + Delete on bucket (slow, risky)
- Predictable paths → Vulnerable to path traversal attacks
- Doesn't scale with millions of users
- Difficult to migrate to another storage provider

✅ Correct Approach: Logical Separation

Principles:

  1. Database = Logical structure (folders, hierarchy, permissions)
  2. Bucket = Physical storage (flat UUID keys)
graph TB
    subgraph "Logical Layer - PostgreSQL Database"
        F[Folders Table]
        FL[Files Table]
        F -->|parent_id| F
        FL -->|folder_id| F
    end
    
    subgraph "Physical Layer - MinIO Bucket"
        B[Flat UUID Structure]
        B1[private/2026/01/13/uuid1.pdf]
        B2[private/2026/01/13/uuid2.jpg]
        B3[public/2026/01/14/uuid3.png]
    end
    
    FL -.->|storage_key| B
    
    style F fill:#3498db,color:#fff
    style FL fill:#2ecc71,color:#fff
    style B fill:#e74c3c,color:#fff

Database Schema

-- Folders: Hierarchical tree structure
CREATE TABLE folders (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id VARCHAR(255) NOT NULL,
    parent_id UUID REFERENCES folders(id) ON DELETE CASCADE,
    name VARCHAR(255) NOT NULL,
    path VARCHAR(1000) NOT NULL,        -- Materialized path: /docs/work/2024
    level INT NOT NULL DEFAULT 0,       -- Tree depth
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW(),
    
    UNIQUE (user_id, parent_id, name),
    INDEX idx_user_path (user_id, path)
);

-- Files: Link to logical folders
CREATE TABLE storage_files (
    id UUID PRIMARY KEY,
    user_id VARCHAR(255) NOT NULL,
    folder_id UUID REFERENCES folders(id) ON DELETE SET NULL,
    file_name VARCHAR(255) NOT NULL,
    storage_key VARCHAR(500) UNIQUE,    -- Physical UUID key in bucket
    content_type VARCHAR(100),
    file_size_bytes BIGINT,
    access_level VARCHAR(20),           -- Private, Public, Shared
    bucket_name VARCHAR(255),
    provider INT,
    uploaded_at TIMESTAMP,
    is_deleted BOOLEAN DEFAULT false,
    
    INDEX idx_folder_id (folder_id),
    INDEX idx_storage_key (storage_key)
);

Physical Key Generation Strategy

public class StorageKeyGenerator
{
    /// <summary>
    /// Generate UUID-based key (NOT based on folder path)
    /// Pattern: {prefix}/{year}/{month}/{day}/{uuid}.{ext}
    /// </summary>
    public string GenerateKey(FileAccessLevel access, string fileName)
    {
        var now = DateTime.UtcNow;
        var prefix = access == FileAccessLevel.Public ? "public" : "private";
        var uuid = Guid.NewGuid().ToString("N"); // 32 chars, no hyphens
        var ext = Path.GetExtension(fileName);
        
        // Example: private/2026/01/13/d290f1ee6c544b0190e6d701748f0851.pdf
        return $"{prefix}/{now:yyyy}/{now:MM}/{now:dd}/{uuid}{ext}";
    }
}

Bucket Structure (Flat Physical)

goodgo/
├── private/
│   ├── 2026/
│   │   └── 01/
│   │       └── 13/
│   │           ├── d290f1ee6c544b0190e6d701748f0851.pdf
│   │           ├── a7f3b2c19d8e4f01bcd5e92f7a4d8b63.jpg
│   │           └── f9e4c1a2b7d36e5f8a0c4d9e2b1f7a8c.docx
├── public/
│   └── 2026/
│       └── 01/
│           └── 14/
│               └── c3d8f2e1a9b47c6e5d0f8a3b2e1c7d9f.png
└── shared/
    └── 2026/
        └── 01/
            └── 15/
                └── e1f2c3d4a5b6c7e8d9f0a1b2c3d4e5f6.xlsx

NO user/documents/... folders in bucket!

Workflows

1. Create Folder (Database Only)

sequenceDiagram
    Client->>API: POST /api/v1/folders {name: "documents"}
    API->>Database: INSERT INTO folders (name, path)
    Database-->>API: folder_id
    API-->>Client: {id, name, path: "/documents"}
    Note over Client,API: Bucket is NOT touched

2. Upload File to Folder

sequenceDiagram
    Client->>API: POST /sign-upload {fileName, folderId}
    API->>Database: Validate folder ownership
    API->>StorageKeyGen: Generate UUID key
    StorageKeyGen-->>API: private/2026/01/13/{uuid}.pdf
    API->>MinIO: Get pre-signed PUT URL
    MinIO-->>API: pre-signed URL
    API->>Database: INSERT file (folder_id, storage_key)
    API-->>Client: {uploadUrl, objectKey}
    Client->>MinIO: PUT file binary
    Client->>API: POST /confirm-upload

3. List Files in Folder

-- Database query (fast, indexed)
SELECT * FROM storage_files 
WHERE folder_id = '{folder-uuid}' 
  AND is_deleted = false
ORDER BY uploaded_at DESC;

-- Result returned to client:
{
  "folder": {
    "id": "folder-123",
    "path": "/documents/work"
  },
  "files": [
    {
      "id": "file-456",
      "name": "report.pdf",
      "logicalPath": "/documents/work/report.pdf",  -- Displayed to user
      "storageKey": "private/2026/01/13/{uuid}.pdf" -- Physical key
    }
  ]
}

4. Rename Folder

-- Only UPDATE database (instant, O(1))
UPDATE folders 
SET name = 'docs', 
    path = '/docs'  -- Update materialized path
WHERE id = '{folder-id}';

-- Update descendant paths
UPDATE folders
SET path = REPLACE(path, '/documents', '/docs')
WHERE path LIKE '/documents/%';

-- Files KEEP their storage_key - DON'T TOUCH bucket!

5. Move File Between Folders

-- Only UPDATE database (instant, O(1))
UPDATE storage_files
SET folder_id = '{new-folder-id}'
WHERE id = '{file-id}';

-- Physical file STAYS at old storage_key
-- No Copy/Delete needed in bucket

Performance Comparison

Operation Anti-pattern (Bucket-based) Correct (Logical) Improvement
Create folder N/A (virtual) INSERT to DB Instant
Rename folder Copy + Delete millions of files UPDATE 1-N rows in DB ~1000x faster
Move file Copy + Delete 1 file UPDATE 1 row in DB ~100x faster
List files Bucket prefix scan Indexed DB query ~50x faster
Delete folder Delete millions of files Cascade delete + background cleanup Async

Data Sovereignty Benefits

  1. Performance:

    • Rename folder: O(1) instead of O(n)
    • Move file: O(1) instead of O(1) bucket copy
    • List files: Indexed query instead of bucket scan

  2. Security:

    • UUID keys are unpredictable
    • No path traversal vulnerabilities
    • Access control in database

  3. Scalability:

    • Database handles metadata (fast)
    • Bucket handles blobs (simple)
    • Easy horizontal scaling

  4. Flexibility:

    • Easy to add features (sharing, versioning)
    • Migration between storage providers
    • Support multiple buckets/regions

  5. Developer Experience:

    • Client sees beautiful folder tree
    • Backend works with simple UUIDs
    • Clean separation of concerns

Direct Upload Components

Component Purpose
SignUploadCommand Validate quota, generate object key with path prefix, create pre-signed PUT URL
SignUploadCommandHandler Handle sign-upload requests
ConfirmUploadCommand Verify file exists, save metadata, update quota
ConfirmUploadCommandHandler Handle confirm-upload with idempotency
SignedUrlController /sign-upload and /confirm-upload endpoints

Download URL Architecture

The Storage Service generates different download URLs based on file access levels.

URL Generation Flow

sequenceDiagram
    participant Client
    participant API as Storage Service
    participant MinIO

    Client->>API: GET /api/v1/files/{id}/download-url
    API->>API: Validate ownership
    
    alt accessLevel = Public
        API-->>Client: Direct URL (no signature)
    else accessLevel = Private/Shared
        API->>MinIO: GetPreSignedDownloadUrl()
        MinIO-->>API: Pre-signed URL (AWS Signature V4)
        API-->>Client: Pre-signed URL with expiration
    end

Access Level URL Types

Access Level Storage Prefix URL Type Expiration
Public public/ Direct URL Never
Private private/ Pre-signed URL Configurable (default: 1 hour)
Shared shared/ Pre-signed URL Configurable (default: 1 hour)

Pre-signed URL Structure

For private/shared files, URLs include AWS Signature Version 4 parameters:

http://minio:9000/bucket/private/2026/01/13/{uuid}.pdf
  ?X-Amz-Algorithm=AWS4-HMAC-SHA256
  &X-Amz-Credential={accessKey}%2F{date}%2F{region}%2Fs3%2Faws4_request
  &X-Amz-Date={timestamp}
  &X-Amz-Expires={seconds}
  &X-Amz-SignedHeaders=host
  &X-Amz-Signature={signature}
Parameter Purpose
X-Amz-Algorithm Signing algorithm (always AWS4-HMAC-SHA256)
X-Amz-Credential Access key + scope (date/region/service)
X-Amz-Date Timestamp when signature was created
X-Amz-Expires URL validity in seconds
X-Amz-SignedHeaders Headers included in signature calculation
X-Amz-Signature Cryptographic signature to verify URL integrity

Security Considerations

  1. Time-Limited Access: Pre-signed URLs expire after configured time
  2. Tamper-Proof: Any URL modification invalidates the signature
  3. Credential Protection: MinIO access keys never exposed to client
  4. Unique URLs: Each request generates a new signature

GetDownloadUrl Implementation

public async Task<string> GetDownloadUrlAsync(StorageFile file)
{
    if (file.AccessLevel == FileAccessLevel.Public)
    {
        // Direct URL for public files
        return $"{_settings.PublicEndpoint}/{file.BucketName}/{file.ObjectKey}";
    }
    
    // Pre-signed URL for private/shared files
    return await _storageProvider.GetPreSignedDownloadUrlAsync(
        file.BucketName,
        file.ObjectKey,
        _settings.PreSignedUrlExpirationSeconds
    );
}

Multipart Upload Architecture (Large Files)

For files larger than 100MB, use Multipart Upload to upload in chunks.

Upload Methods Comparison

Aspect Direct Upload Multipart Upload
File size < 100MB > 100MB (up to 5GB+)
Mechanism Single PUT request Multiple part uploads
Resume support No Yes (per part)
Progress tracking No Yes (per part)
Use case Small/medium files Large files, video, archives

Multipart Upload Flow

sequenceDiagram
    participant Client
    participant API as Storage Service
    participant DB as PostgreSQL
    participant MinIO

    rect rgb(200, 230, 200)
        Note over Client,API: 1. Initiate Upload
        Client->>API: POST /api/v1/files/multipart/initiate
        API->>DB: Create MultipartUpload record
        API->>MinIO: InitiateMultipartUpload
        MinIO-->>API: Provider UploadId
        API-->>Client: {uploadId, objectKey, totalChunks}
    end

    rect rgb(200, 200, 230)
        Note over Client,MinIO: 2. Upload Parts (repeat for each chunk)
        loop For each part 1..N
            Client->>API: POST /api/v1/files/multipart/upload-part
            API->>MinIO: UploadPart(partNumber, data)
            MinIO-->>API: ETag
            API->>DB: Save part info (partNumber, etag)
            API-->>Client: {success, etag}
        end
    end

    rect rgb(230, 200, 200)
        Note over Client,API: 3. Optional: Check Progress
        Client->>API: GET /api/v1/files/multipart/{uploadId}
        API->>DB: Get upload + parts
        API-->>Client: {progress: 75%, uploadedChunks: 3/4}
    end

    rect rgb(230, 230, 200)
        Note over Client,API: 4. Complete Upload
        Client->>API: POST /api/v1/files/multipart/complete
        API->>MinIO: CompleteMultipartUpload(parts[])
        MinIO-->>API: OK
        API->>DB: Create StorageFile, Update quota
        API-->>Client: {fileId, objectKey}
    end

Multipart Upload Endpoints

Method Endpoint Description
POST /api/v1/files/multipart/initiate Start upload session
POST /api/v1/files/multipart/upload-part Upload 1 chunk
POST /api/v1/files/multipart/complete Complete and merge parts
DELETE /api/v1/files/multipart/abort Cancel upload, cleanup
GET /api/v1/files/multipart/{uploadId} Check progress

Multipart Upload Components

Component Purpose
InitiateMultipartUploadCommand Create upload session, generate object key
UploadPartCommand Upload 1 part to storage provider
CompleteMultipartUploadCommand Merge parts, create StorageFile record
AbortMultipartUploadCommand Cleanup parts, mark as aborted
GetMultipartUploadProgressQuery Get upload progress
MultipartUploadController API endpoints for multipart

Storage Provider Architecture

graph TD
    subgraph "Storage Provider Factory"
        F[StorageProviderFactory]
        C[Configuration]
    end
    
    subgraph "Providers"
        MP[MinioStorageProvider]
        AP[AliyunOssStorageProvider]
    end
    
    subgraph "Storage Backends"
        MINIO[(MinIO)]
        OSS[(Aliyun OSS)]
    end
    
    C --> |STORAGE_PROVIDER=minio| F
    C --> |STORAGE_PROVIDER=aliyun| F
    F --> |GetProvider| MP
    F --> |GetProvider| AP
    MP --> MINIO
    AP --> OSS
    
    style F fill:#4a90d9,stroke:#2d5986,color:#fff
    style MP fill:#c73b3b,stroke:#922b2b,color:#fff
    style AP fill:#ff6b35,stroke:#cc5500,color:#fff

Storage Provider Interface

public interface IStorageProvider
{
    Task<UploadResult> UploadAsync(Stream stream, string objectKey, ...);
    Task<Stream> DownloadAsync(string bucketName, string objectKey);
    Task DeleteAsync(string bucketName, string objectKey);
    Task<bool> ExistsAsync(string bucketName, string objectKey);
    Task<string> GetPreSignedDownloadUrlAsync(string bucketName, string objectKey, int expirationSeconds);
    Task<string> GetPreSignedUploadUrlAsync(string bucketName, string objectKey, int expirationSeconds);
}

Inter-Service Communication

sequenceDiagram
    participant Client
    participant Storage as Storage Service
    participant Cache as In-Memory Cache
    participant IAM as IAM Service
    
    Client->>Storage: Upload File (JWT)
    Storage->>Cache: Check user cache
    alt Cache Hit
        Cache-->>Storage: User info
    else Cache Miss
        Storage->>IAM: GET /api/v1/users/me
        Note over Storage,IAM: Headers: Authorization, X-Service-Name
        IAM-->>Storage: User info
        Storage->>Cache: Store (5 min TTL)
    end
    Storage->>Storage: Validate quota
    Storage->>Storage: Upload to provider
    Storage-->>Client: Upload result

IAM Client Features

Feature Description
Caching In-memory cache for user info (5 min TTL)
Health Check Check IAM availability with caching (1 min TTL)
Polly Resilience Retry (3x exponential) + Circuit Breaker
Permission Check HasPermissionAsync, HasRoleAsync

Available IAM Client Methods

// User Operations
Task<IamUserInfo?> ValidateUserAsync(string accessToken);
Task<IamUserInfo?> GetUserByIdAsync(string userId, string accessToken);
Task<bool> UserExistsAsync(string userId, string accessToken);
Task<IReadOnlyList<string>> GetUserRolesAsync(string userId, string accessToken);
Task<IReadOnlyList<string>> GetUserPermissionsAsync(string userId, string accessToken);
Task<bool> HasPermissionAsync(string userId, string permission, string accessToken);
Task<bool> HasRoleAsync(string userId, string role, string accessToken);

// Health Check
Task<IamHealthStatus> CheckHealthAsync();
Task<bool> IsAvailableAsync();

// Cache Management
void InvalidateUserCache(string userId);
void ClearCache();

Database Schema

erDiagram
    storage_files {
        uuid id PK
        varchar file_name
        varchar bucket_name
        varchar object_key
        varchar content_type
        bigint file_size_bytes
        varchar user_id
        varchar tenant_id
        int provider
        int access_level
        timestamp uploaded_at
        timestamp expires_at
        varchar checksum
        boolean is_deleted
        timestamp deleted_at
    }
    
    user_storage_quotas {
        uuid id PK
        varchar user_id UK
        bigint max_storage_bytes
        bigint used_storage_bytes
        int max_file_count
        int current_file_count
        varchar quota_tier
        timestamp last_updated_at
        timestamp created_at
    }

API Endpoints

Files (Legacy - Proxy Upload)

Method Endpoint Description
POST /api/v1/files/upload Upload file via backend (max 100MB)
GET /api/v1/files List user files with pagination
GET /api/v1/files/{id} Get file metadata
GET /api/v1/files/{id}/download-url Get pre-signed download URL
DELETE /api/v1/files/{id} Delete file (soft delete)

Direct Upload (Recommended)

Method Endpoint Description
POST /api/v1/storage/sign-upload Get pre-signed PUT URL for direct upload
POST /api/v1/storage/confirm-upload Confirm upload and save metadata

Quota

Method Endpoint Description
GET /api/v1/quota Get user storage quota

Admin API (Requires Role: Admin)

Quota Management

Method Endpoint Description
GET /api/v1/admin/quotas Get all users' quotas
GET /api/v1/admin/quotas/{userId} Get quota for specific user
PUT /api/v1/admin/quotas/{userId} Update quota limits

Files Management

Method Endpoint Description
GET /api/v1/admin/files Get all files with filtering
DELETE /api/v1/admin/files/{id} Delete file for policy violation

Shares Management

Method Endpoint Description
GET /api/v1/admin/shares Get all shares
DELETE /api/v1/admin/shares/{id} Revoke share for violation

Statistics

Method Endpoint Description
GET /api/v1/admin/statistics Dashboard aggregated statistics
GET /api/v1/admin/statistics/users-near-limit Users near quota limit (>80%)

File Versioning Architecture

The Storage Service maintains version history for files, allowing users to track changes and restore previous versions.

Version Flow

sequenceDiagram
    participant Client
    participant API as Storage Service
    participant DB as PostgreSQL
    participant MinIO

    rect rgb(44, 62, 80)
        Note over Client,API: 1. Upload New Version
        Client->>API: PUT /files/{id} (new file)
        API->>DB: Create FileVersion (version N+1)
        API->>MinIO: Store new object with version key
        API->>DB: Update StorageFile.ObjectKey
        API-->>Client: {versionNumber, fileId}
    end

    rect rgb(39, 174, 96)
        Note over Client,API: 2. List Versions
        Client->>API: GET /files/{id}/versions
        API->>DB: SELECT * FROM file_versions WHERE file_id = ?
        API-->>Client: [{version: 1, ...}, {version: 2, ...}]
    end

    rect rgb(230, 126, 34)
        Note over Client,API: 3. Restore Version
        Client->>API: POST /files/{id}/versions/{N}/restore
        API->>DB: Get FileVersion N
        API->>DB: Update StorageFile from version
        API-->>Client: {restored: true, versionNumber: N}
    end

FileVersion Entity

Property Type Description
Id Guid Version ID
FileId Guid Parent file reference
VersionNumber int Sequential version number
ObjectKey string Storage key for this version
SizeBytes long File size for this version
ContentType string MIME type
Checksum string? Optional hash
CreatedAt DateTime Version creation time
CreatedBy string User who created version
IsCurrent bool Whether this is the active version

File Sharing Architecture

The Storage Service provides secure file sharing via token-based access with optional password protection, expiration, and download limits.

Share Flow

sequenceDiagram
    participant Owner
    participant API as Storage Service
    participant DB as PostgreSQL
    participant Guest

    rect rgb(44, 62, 80)
        Note over Owner,API: 1. Create Share
        Owner->>API: POST /storage/files/{id}/shares
        API->>DB: Create FileShare (token, options)
        API-->>Owner: {shareToken, shareUrl}
    end

    rect rgb(39, 174, 96)
        Note over Guest,API: 2. Access Share (Public)
        Guest->>API: GET /storage/shares/public/{token}
        API->>DB: Validate share (expired? revoked? limit?)
        API->>DB: Optional password check
        API->>DB: Increment download count
        API-->>Guest: {downloadUrl, fileMetadata}
    end

    rect rgb(192, 57, 43)
        Note over Owner,API: 3. Revoke Share
        Owner->>API: DELETE /storage/shares/{id}
        API->>DB: Set status = Revoked
        API-->>Owner: 204 No Content
    end

FileShare Entity

Property Type Description
Id Guid Share ID
FileId Guid File being shared
SharedBy string User who created share
SharedWith string? Specific user (null = public link)
Permission SharePermission View, Download, Edit, Admin
ShareToken string Unique URL-safe token
PasswordHash string? Optional password (PBKDF2)
ExpiresAt DateTime? Optional expiration
MaxDownloads int? Optional download limit
DownloadCount int Current download count
Status FileShareStatus Active, Expired, Revoked, LimitReached

Share Permissions

Permission Can View Can Download Can Edit Can Manage
View ✅ ❌ ❌ ❌
Download ✅ ✅ ❌ ❌
Edit ✅ ✅ ✅ ❌
Admin ✅ ✅ ✅ ✅

Health Checks

graph TD
    HC[Health Check Endpoint]
    HC --> |/health/live| L[Liveness]
    HC --> |/health/ready| R[Readiness]
    
    R --> DB[(PostgreSQL)]
    R --> MINIO[(MinIO)]
    R --> IAM[IAM Service]
    
    style HC fill:#2C3E50,color:#ECF0F1,stroke:#34495E,stroke-width:3px
    style L fill:#27AE60,color:#ECF0F1,stroke:#229954,stroke-width:2px
    style R fill:#E67E22,color:#ECF0F1,stroke:#D35400,stroke-width:2px
    style DB fill:#34495E,color:#ECF0F1,stroke:#2C3E50,stroke-width:2px
    style MINIO fill:#C0392B,color:#ECF0F1,stroke:#A93226,stroke-width:2px
    style IAM fill:#8E44AD,color:#ECF0F1,stroke:#7D3C98,stroke-width:2px

Deployment Architecture

Docker Compose (Local)

services:
  storage-api:
    build: .
    ports: ["5002:8080"]
    depends_on:
      - postgres
      - redis
      - minio
    environment:
      - Storage__Provider=minio
      - Storage__MinIO__Endpoint=minio:9000
  
  minio:
    image: minio/minio:latest
    ports: ["9000:9000", "9001:9001"]

Traefik Integration

labels:
  - "traefik.enable=true"
  - "traefik.http.routers.storage-service.rule=PathPrefix(`/api/v1/files`) || PathPrefix(`/api/v1/quota`)"
  - "traefik.http.services.storage-service.loadbalancer.server.port=8080"

Security Considerations

  1. Authentication: JWT Bearer validation via IAM Service
  2. Authorization: User ownership check on files
  3. Input Validation: File size limits, content type validation
  4. Pre-signed URLs: Time-limited access to files
  5. Soft Delete: Files are marked deleted, not immediately removed

Performance Optimization

  1. Caching: In-memory cache for IAM user info
  2. Pre-signed URLs: Direct client-to-storage downloads
  3. Streaming Upload: Stream-based file handling
  4. Async Operations: All I/O operations are async
  5. Connection Pooling: HTTP client with Polly policies

References

Reference in New Issue View Git Blame Copy Permalink