# StorageService - Service Documentation > Auto-generated from source code audit on 2026-03-13 ## Overview **StorageService** is a file storage microservice providing S3-compatible object storage with support for direct uploads, multipart uploads, file sharing, file versioning, logical folders, and per-user storage quotas. It supports two storage backends: **MinIO** (primary) and **Aliyun OSS**. - **Port**: 5002 (Development) - **Database**: `storage_service` (PostgreSQL / Neon) - **Storage Backend**: MinIO (S3-compatible) at `167.114.174.113:9000` - **Cache**: Redis at `167.114.174.113:6379` - **Default Bucket**: `goodgo` - **Max File Size**: 100MB (configurable), validator limit 500MB - **Auth**: JWT Bearer via IAM Service - **Framework**: .NET 10.0, Clean Architecture + CQRS --- ## API Endpoints ### Files (`/api/v1/files`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | POST | `/api/v1/files/upload` | Bearer | Upload a file (max 100MB) | | GET | `/api/v1/files` | Bearer | List user files (skip, take, search) | | GET | `/api/v1/files/{fileId}` | Bearer | Get file metadata by ID | | GET | `/api/v1/files/{fileId}/download-url` | Bearer | Get pre-signed download URL | | GET | `/api/v1/files/{fileId}/cdn-url` | Bearer | Get CDN URL (public) or fallback pre-signed URL | | DELETE | `/api/v1/files/{fileId}` | Bearer | Soft delete a file | ### Direct Upload - Pre-Signed URL (`/api/v1/storage`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | POST | `/api/v1/storage/sign-upload` | Bearer | Get pre-signed PUT URL for direct upload to MinIO | | POST | `/api/v1/storage/confirm-upload` | Bearer | Confirm upload and save metadata after direct upload | ### Multipart Upload (`/api/v1/files/multipart`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | POST | `/api/v1/files/multipart/initiate` | Bearer | Initiate multipart upload session | | POST | `/api/v1/files/multipart/upload-part` | Bearer | Upload a single part (max 20MB per part) | | POST | `/api/v1/files/multipart/complete` | Bearer | Complete and merge all parts | | DELETE | `/api/v1/files/multipart/abort` | Bearer | Abort and cleanup | | GET | `/api/v1/files/multipart/{uploadId}` | Bearer | Get upload progress | ### File Sharing (`/api/v1/storage`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | POST | `/api/v1/storage/files/{fileId}/shares` | Bearer | Create share link (with optional password, expiry, max downloads) | | GET | `/api/v1/storage/files/{fileId}/shares` | Bearer | Get all shares for a file | | DELETE | `/api/v1/storage/shares/{shareId}` | Bearer | Revoke a share link | | GET | `/api/v1/storage/shares/public/{token}` | Anonymous | Access shared file by token (public endpoint) | ### File Versioning (`/api/v1/storage/files/{fileId}/versions`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/storage/files/{fileId}/versions` | Bearer | Get all versions of a file | | GET | `/api/v1/storage/files/{fileId}/versions/{versionId}/download` | Bearer | Get download URL for specific version | | POST | `/api/v1/storage/files/{fileId}/versions/{versionId}/restore` | Bearer | Restore file to a specific version | ### Folders (`/api/v1/storage/folders`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | POST | `/api/v1/storage/folders` | Bearer | Create folder (root or child) | | GET | `/api/v1/storage/folders` | Bearer | List folders (root or children of parentId) | | GET | `/api/v1/storage/folders/{folderId}` | Bearer | Get folder by ID | | PUT | `/api/v1/storage/folders/{folderId}` | Bearer | Rename folder (O(1) operation) | | DELETE | `/api/v1/storage/folders/{folderId}` | Bearer | Soft delete folder | ### Quota (`/api/v1/quota`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/quota` | Bearer | Get current user's storage quota | ### Admin - Files (`/api/v1/admin/files`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/admin/files` | Admin/SuperAdmin | Get all files with filtering and pagination | | DELETE | `/api/v1/admin/files/{fileId}` | Admin/SuperAdmin | Delete file (bypasses ownership, requires reason) | ### Admin - Quotas (`/api/v1/admin/quotas`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/admin/quotas` | Admin/SuperAdmin | Get all users' quotas with filtering | | GET | `/api/v1/admin/quotas/{userId}` | Admin/SuperAdmin | Get quota for specific user | | PUT | `/api/v1/admin/quotas/{userId}` | Admin/SuperAdmin | Update user quota limits | ### Admin - Shares (`/api/v1/admin/shares`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/admin/shares` | Admin/SuperAdmin | Get all file shares with filtering | | DELETE | `/api/v1/admin/shares/{shareId}` | Admin/SuperAdmin | Revoke share (requires reason) | ### Admin - Statistics (`/api/v1/admin/statistics`) | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/api/v1/admin/statistics` | Admin/SuperAdmin | Get aggregated storage statistics | | GET | `/api/v1/admin/statistics/users-near-limit` | Admin/SuperAdmin | Get users with usage >= 80% | ### Health Checks | Method | Route | Auth | Description | |--------|-------|------|-------------| | GET | `/health` | None | Full health check (incl. PostgreSQL) | | GET | `/health/live` | None | Liveness probe (app running) | | GET | `/health/ready` | None | Readiness probe | --- ## Commands ### UploadFileCommand - **Input**: `Stream FileStream, string FileName, string ContentType, long FileSizeBytes, string UserId, string? TenantId, FileAccessLevel AccessLevel` - **Logic**: Check file size limit -> Check user quota -> Generate object key (`{accessLevel}/{userId}/{date}/{uniqueId}_{filename}`) -> Upload to storage provider -> Save StorageFile entity -> Update quota -> SaveEntities - **Validator**: FileName (required, max 255, no `..` `/` `\`), ContentType (required, max 100), FileSizeBytes (> 0, <= 500MB), UserId (required, max 128), TenantId (max 128) - **Result**: `UploadFileResult(bool Success, Guid? FileId, string? ObjectKey, string? Error)` ### SignUploadCommand - **Input**: `string UserId, string FileName, string ContentType, long FileSizeBytes, FileAccessLevel AccessLevel, string? TenantId` - **Logic**: Validate file size -> Check user quota -> Generate object key -> Ensure bucket exists -> Generate pre-signed PUT URL - **Validator**: Same rules as UploadFileCommand - **Result**: `SignUploadResult(bool Success, string? UploadUrl, string? ObjectKey, DateTime? ExpiresAt, string? Error)` ### ConfirmUploadCommand - **Input**: `string UserId, string ObjectKey, string FileName, string ContentType, long FileSizeBytes, FileAccessLevel AccessLevel, string? TenantId` - **Logic**: Validate object key ownership (userId in path) -> Verify file exists in storage -> Idempotency check (existing file by objectKey) -> Double-check quota -> Create StorageFile entity -> Update quota -> Invalidate caches - **Validator**: UserId (required, max 128), ObjectKey (required, max 1024), FileName (required, max 255), ContentType (required, max 100), FileSizeBytes (> 0) - **Result**: `ConfirmUploadResult(bool Success, Guid? FileId, FileDto? Metadata, string? Error)` ### DeleteFileCommand - **Input**: `Guid FileId, string UserId` - **Logic**: Get file -> Check ownership -> Delete from storage provider (best-effort) -> Soft delete record -> Update quota -> Invalidate caches - **Validator**: FileId (required), UserId (required, max 128) - **Result**: `DeleteFileResult(bool Success, string? Error)` ### CreateFileShareCommand - **Input**: `Guid FileId, string UserId, SharePermission Permission, string? SharedWith, string? Password, DateTime? ExpiresAt, int? MaxDownloads` - **Logic**: Verify file exists + ownership -> Create FileShare entity (generates random token, hashes password with PBKDF2) -> Save -> Return share URL - **Validator**: FileId (required), UserId (required, max 128), SharedWith (max 256), Password (min 4, max 100), MaxDownloads (> 0, <= 10000), ExpiresAt (must be future) - **Result**: `CreateFileShareResult(bool Success, Guid? ShareId, string? ShareToken, string? ShareUrl, string? Error)` ### RevokeFileShareCommand - **Input**: `Guid ShareId, string UserId` - **Logic**: Get share -> Verify ownership -> Revoke (idempotent) -> Save - **Result**: `RevokeFileShareResult(bool Success, string? Error)` ### InitiateMultipartUploadCommand - **Input**: `string FileName, long FileSizeBytes, string ContentType, string UserId, int? ChunkSizeBytes, FileAccessLevel AccessLevel, string? TenantId` - **Logic**: Validate file size -> Check quota -> Determine chunk size (default 5MB, min 5MB, max 100MB) -> Generate object key -> Initiate multipart upload at provider -> Create MultipartUpload entity (expiration: 24h) -> Save - **Result**: `InitiateMultipartUploadResult(... UploadId, ProviderUploadId, ObjectKey, BucketName, TotalChunks, ChunkSizeBytes, ExpiresAt ...)` ### UploadPartCommand - **Input**: `Guid UploadId, int PartNumber, Stream DataStream, string UserId` - **Logic**: Get upload with parts -> Verify ownership -> Check status (InProgress) -> Check expiration -> Upload part to provider -> Add part to domain entity -> Save - **Result**: `UploadPartResult(bool Success, string? ETag, string? Error)` ### CompleteMultipartUploadCommand - **Input**: `Guid UploadId, string UserId` - **Logic**: Get upload with parts -> Verify ownership -> Domain validation (all parts uploaded) -> Complete at provider -> Create StorageFile entity -> Update quota -> Save - **Result**: `CompleteMultipartUploadResult(bool Success, Guid? FileId, string? ObjectKey, string? Error)` ### AbortMultipartUploadCommand - **Input**: `Guid UploadId, string UserId` - **Logic**: Get upload -> Verify ownership -> Abort at provider (best-effort) -> Mark as aborted in DB -> Save - **Result**: `AbortMultipartUploadResult(bool Success, string? Error)` ### AdminDeleteFileCommand - **Input**: `Guid FileId, string AdminUserId, string Reason` - **Logic**: Same as DeleteFileCommand but bypasses ownership check. Logs admin action with reason. - **Result**: `AdminDeleteFileResult(bool Success, string? DeletedFileName, string? FileOwnerUserId, string? Error)` ### AdminRevokeShareCommand - **Input**: `Guid ShareId, string AdminUserId, string Reason` - **Logic**: Get share -> Revoke -> Log admin action with reason -> Save - **Result**: `AdminRevokeShareResult(bool Success, Guid? FileId, string? ShareOwnerUserId, string? Error)` ### UpdateUserQuotaCommand - **Input**: `string TargetUserId, long MaxStorageBytes, int MaxFileCount, string? QuotaTier` - **Logic**: Get or create quota -> Update limits (validates max >= current usage) -> Invalidate cache -> Save - **Validator**: TargetUserId (required, max 128), MaxStorageBytes (> 0), MaxFileCount (> 0), QuotaTier (max 50) - **Result**: `UpdateUserQuotaResult(bool Success, QuotaUpdatedDto? Data, string? Error)` --- ## Queries ### GetFileQuery - **Input**: `Guid FileId, string UserId` - **Logic**: Cache-aside pattern (10min TTL) -> Get from DB -> Check ownership - **Result**: `FileDto?` ### GetUserFilesQuery - **Input**: `string UserId, int Skip, int Take, string? SearchTerm` - **Logic**: If search term: search by filename; otherwise: list by userId -> Count total -> Map to DTOs - **Result**: `UserFilesResult(IReadOnlyList Files, int TotalCount)` ### GetUserQuotaQuery - **Input**: `string UserId` - **Logic**: Cache-aside pattern (5min TTL) -> GetOrCreate quota -> Map to DTO - **Result**: `QuotaDto?` ### GetDownloadUrlQuery - **Input**: `Guid FileId, string UserId, int ExpirationSeconds` - **Logic**: Get file -> Check access (private files: owner only) -> Mark accessed -> Generate pre-signed download URL - **Result**: `DownloadUrlResult(bool Success, string? Url, int? ExpiresInSeconds, string? Error)` ### GetFileSharesQuery - **Input**: `Guid FileId, string UserId` - **Logic**: Verify file ownership -> Get all shares for file -> Map to DTOs - **Result**: `IEnumerable` ### AccessSharedFileQuery - **Input**: `string Token, string? Password` - **Logic**: Get share by token -> Validate (not expired/revoked/limit reached) -> Validate password (PBKDF2) -> Get file -> Generate pre-signed download URL -> Increment download count -> Save - **Result**: `SharedFileAccessResult(bool Success, string? DownloadUrl, string? FileName, ...)` ### GetMultipartUploadProgressQuery - **Input**: `Guid UploadId, string UserId` - **Logic**: Get upload with parts -> Verify ownership -> Calculate progress - **Result**: `MultipartUploadProgressResult?` ### AdminGetFilesQuery - **Input**: `int PageNumber, int PageSize, string? UserId, string? AccessLevel, string? ContentType, DateTime? UploadedAfter/Before, string? SortBy, bool Descending` - **Logic**: IgnoreQueryFilters (includes soft-deleted) -> Apply filters -> Sort -> Paginate - **Result**: `AdminFilesResult(IReadOnlyList Items, int TotalCount, int PageNumber, int PageSize, int TotalPages)` ### GetAllUsersQuotaQuery - **Input**: `int PageNumber, int PageSize, string? QuotaTier, double? MinUsagePercentage, string? SortBy, bool Descending` - **Logic**: Filter by tier, min usage% -> Sort -> Paginate - **Result**: `AllUsersQuotaResult(IReadOnlyList Items, ...)` ### GetStorageStatisticsQuery - **Input**: (none) - **Logic**: Aggregate stats: total users, total storage used/allocated, total files, avg usage%, users by tier, users near limit (>80%), users over limit (>=100%) - **Result**: `StorageStatisticsDto` ### AdminGetSharesQuery - **Input**: `int PageNumber, int PageSize, string? Status, string? SharedBy` - **Logic**: Filter by status, sharedBy -> Paginate -> Order by CreatedAt desc - **Result**: `AdminSharesResult(IReadOnlyList Items, ...)` --- ## Domain Model ### StorageFile (Aggregate Root) - **Table**: `storage_files` - **Fields**: Id, FileName, BucketName, ObjectKey, ContentType, FileSizeBytes, UserId, TenantId, Provider (enum->string), AccessLevel (enum->string), UploadedAt, LastAccessedAt, ExpiresAt, Checksum, IsDeleted, DeletedAt - **Behavior Methods**: `MarkAccessed()`, `UpdateAccessLevel(level)`, `Delete()` (soft), `SetExpiration(dt)`, `UpdateFromVersion(objectKey, size, contentType)` - **Domain Events**: `FileUploadedDomainEvent` (on create), `FileDeletedDomainEvent` (on delete) - **Query Filter**: `!IsDeleted` (global) ### UserStorageQuota (Aggregate Root) - **Table**: `user_storage_quotas` - **Fields**: Id, UserId, MaxStorageBytes (default 1GB), UsedStorageBytes, MaxFileCount (default 1000), CurrentFileCount, QuotaTier (default "free"), LastUpdatedAt, CreatedAt - **Computed**: `RemainingStorageBytes`, `RemainingFileCount`, `UsagePercentage` - **Behavior Methods**: `CanUpload(size)`, `AddUsage(size, count)`, `RemoveUsage(size, count)`, `UpdateLimits(max, maxFiles, tier)`, `RecalculateUsage(totalBytes, totalFiles)` - **Domain Events**: `UserQuotaUpdatedDomainEvent` (on AddUsage/RemoveUsage) - **Validation**: `UpdateLimits` throws if new max < current usage ### FileShare (Aggregate Root) - **Table**: `file_shares` - **Fields**: Id, FileId (FK -> storage_files), SharedBy, SharedWith, Permission (enum->string), ShareToken (unique, 32 random bytes base64), PasswordHash (PBKDF2-SHA256, 100K iterations), ExpiresAt, MaxDownloads, DownloadCount, Status (enum->string), CreatedAt, RevokedAt - **Behavior Methods**: `IsValid()` (checks status, expiry, download limit), `ValidatePassword(pwd)`, `IncrementDownloadCount()`, `Revoke()` - **Enums**: `SharePermission` (View=0, Download=1, Edit=2, Admin=3), `FileShareStatus` (Active=0, Expired=1, Revoked=2, LimitReached=3) ### FileVersion (Entity) - **Table**: `file_versions` - **Fields**: Id, FileId (FK -> storage_files), VersionNumber, ObjectKey, SizeBytes, ContentType, Checksum, CreatedAt, CreatedBy, Comment, IsCurrent - **Behavior Methods**: `MarkAsCurrent()`, `UnmarkAsCurrent()` ### Folder (Aggregate Root) - **Table**: `folders` - **Fields**: Id, UserId, ParentId (FK self-referencing, restrict delete), Name, Path (materialized path e.g. `/parent/child/`), Level (0=root), CreatedAt, UpdatedAt, IsDeleted, DeletedAt - **Behavior Methods**: `CreateRoot(userId, name)` (static), `CreateChild(name)`, `Rename(newName)`, `MoveTo(newParent)`, `Delete()` (soft) - **Design Note**: Folders are LOGICAL only. Storage uses flat UUID keys. Rename/move is O(1) -- database-only. - **Query Filter**: `!IsDeleted` (global) ### MultipartUpload (Aggregate Root) - **Table**: `multipart_uploads` - **Fields**: Id, UserId, FileName, TotalSizeBytes, ChunkSizeBytes, TotalChunks (computed), UploadedChunks, ProviderUploadId, BucketName, ObjectKey, ContentType, Status (enum->string), CreatedAt, CompletedAt, ExpiresAt - **Behavior Methods**: `AddPart(partNum, etag, size)`, `Complete()` (validates all parts uploaded), `Abort()`, `MarkFailed()`, `IsExpired()`, `GetProgressPercentage()` - **Domain Events**: `MultipartUploadInitiatedDomainEvent`, `MultipartUploadCompletedDomainEvent`, `MultipartUploadAbortedDomainEvent` - **Child Entity**: `MultipartUploadPart` (Id, MultipartUploadId, PartNumber, ETag, SizeBytes, UploadedAt) - **Enums**: `MultipartUploadStatus` (InProgress=0, Completed=1, Aborted=2, Failed=3, Expired=4) --- ## Database Schema ### Table: `storage_files` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | file_name | varchar(255) | NOT NULL | | bucket_name | varchar(100) | NOT NULL | | object_key | varchar(500) | NOT NULL, UNIQUE | | content_type | varchar(100) | NOT NULL | | file_size_bytes | bigint | NOT NULL | | user_id | varchar(100) | NOT NULL | | tenant_id | varchar(100) | nullable | | provider | varchar (string enum) | NOT NULL | | access_level | varchar (string enum) | NOT NULL | | uploaded_at | timestamp | NOT NULL | | last_accessed_at | timestamp | nullable | | expires_at | timestamp | nullable | | checksum | varchar(100) | nullable | | is_deleted | boolean | NOT NULL, default false | | deleted_at | timestamp | nullable | **Indexes**: object_key (unique), user_id, tenant_id, uploaded_at, is_deleted **Query Filter**: `is_deleted = false` ### Table: `user_storage_quotas` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | user_id | varchar(100) | NOT NULL, UNIQUE | | max_storage_bytes | bigint | NOT NULL | | used_storage_bytes | bigint | NOT NULL | | max_file_count | int | NOT NULL | | current_file_count | int | NOT NULL | | quota_tier | varchar(50) | nullable | | created_at | timestamp | NOT NULL | | last_updated_at | timestamp | NOT NULL | **Indexes**: user_id (unique) ### Table: `file_shares` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | file_id | uuid | NOT NULL, FK -> storage_files (CASCADE) | | shared_by | varchar(255) | NOT NULL | | shared_with | varchar(255) | nullable | | permission | varchar(50) (string enum) | NOT NULL | | share_token | varchar(255) | NOT NULL, UNIQUE | | password_hash | varchar(255) | nullable | | expires_at | timestamp | nullable | | max_downloads | int | nullable | | download_count | int | NOT NULL, default 0 | | status | varchar(50) (string enum) | NOT NULL | | created_at | timestamp | NOT NULL | | revoked_at | timestamp | nullable | **Indexes**: file_id, share_token (unique), shared_with, shared_by ### Table: `file_versions` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | file_id | uuid | NOT NULL, FK -> storage_files (CASCADE) | | version_number | int | NOT NULL | | object_key | varchar(500) | NOT NULL | | size_bytes | bigint | NOT NULL | | content_type | varchar(100) | NOT NULL | | checksum | varchar(100) | nullable | | created_at | timestamp | NOT NULL | | created_by | varchar(255) | NOT NULL | | comment | varchar(500) | nullable | | is_current | boolean | default false | **Indexes**: file_id, (file_id + version_number) unique, (file_id + is_current) ### Table: `folders` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | user_id | varchar(255) | NOT NULL | | parent_id | uuid | nullable, FK self (RESTRICT) | | name | varchar(255) | NOT NULL | | path | varchar(1000) | NOT NULL | | level | int | NOT NULL | | created_at | timestamp | NOT NULL | | updated_at | timestamp | NOT NULL | | is_deleted | boolean | NOT NULL, default false | | deleted_at | timestamp | nullable | **Indexes**: user_id, parent_id, (user_id + parent_id + name) unique filtered `is_deleted = false`, path **Query Filter**: `is_deleted = false` ### Table: `multipart_uploads` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | user_id | varchar(255) | NOT NULL | | file_name | varchar(255) | NOT NULL | | total_size_bytes | bigint | NOT NULL | | chunk_size_bytes | int | NOT NULL | | total_chunks | int | NOT NULL | | uploaded_chunks | int | NOT NULL | | upload_id | varchar(255) | NOT NULL (provider upload ID) | | bucket_name | varchar(255) | NOT NULL | | object_key | varchar(500) | NOT NULL | | content_type | varchar(100) | NOT NULL | | status | varchar(50) (string enum) | NOT NULL | | created_at | timestamp | NOT NULL | | completed_at | timestamp | nullable | | expires_at | timestamp | NOT NULL | **Indexes**: (user_id + status), upload_id, created_at ### Table: `multipart_upload_parts` | Column | Type | Constraints | |--------|------|-------------| | id | uuid | PK | | multipart_upload_id | uuid | NOT NULL, FK -> multipart_uploads (CASCADE) | | part_number | int | NOT NULL | | etag | varchar(255) | NOT NULL | | size_bytes | bigint | NOT NULL | | uploaded_at | timestamp | NOT NULL | **Indexes**: (multipart_upload_id + part_number) unique ### Migrations 1. `20260112185402_InitialCreate` - storage_files, user_storage_quotas 2. `20260113155939_AddMultipartUploadTables` - multipart_uploads, multipart_upload_parts 3. `20260113170635_AddFileSharing` - file_shares 4. `20260113171238_AddFileVersioning` - file_versions 5. `20260113171612_AddFolders` - folders --- ## Dependencies ### NuGet Packages **API Layer:** - MediatR 12.4.1 - FluentValidation 11.11.0 - Microsoft.AspNetCore.Authentication.JwtBearer 10.0.1 - Swashbuckle.AspNetCore 7.2.0 - Asp.Versioning.Mvc 8.1.0 - AspNetCore.HealthChecks.NpgSql 8.0.2 - AspNetCore.HealthChecks.Redis 8.0.1 - Hellang.Middleware.ProblemDetails 6.5.1 - Serilog.AspNetCore 8.0.3 **Domain Layer:** - MediatR.Contracts 2.0.1 **Infrastructure Layer:** - Microsoft.EntityFrameworkCore 10.0.0 - Npgsql.EntityFrameworkCore.PostgreSQL 10.0.0 - MediatR 12.4.1 - Dapper 2.1.35 - Polly 8.5.0 - StackExchange.Redis 2.8.16 - Minio 6.0.4 - AWSSDK.S3 3.7.400 (for low-level multipart upload) - Aliyun.OSS.SDK.NetCore 2.14.1 ### External Services - **IAM Service** (`http://localhost:5001`) - JWT token validation - **MinIO** (`167.114.174.113:9000`) - Object storage backend - **Redis** (`167.114.174.113:6379`) - Caching (quota, file metadata, user files) - **PostgreSQL** (Neon) - Metadata persistence --- ## Configuration ### appsettings.json | Section | Key | Default | Description | |---------|-----|---------|-------------| | ConnectionStrings | DefaultConnection | Neon PostgreSQL | Database connection | | Storage | Provider | `minio` | Active provider (`minio` or `aliyun`) | | Storage | DefaultBucket | `goodgo` | Default S3 bucket | | Storage | PreSignedUrlExpirationSeconds | 3600 | Pre-signed URL TTL | | Storage | MaxFileSizeBytes | 104857600 (100MB) | Max upload size | | Storage:MinIO | Endpoint | `167.114.174.113:9000` | MinIO endpoint | | Storage:MinIO | AccessKey | minioadmin | MinIO access key | | Storage:MinIO | UseSSL | false | SSL for MinIO | | Storage:MinIO | Region | us-east-1 | MinIO region | | Storage:AliyunOSS | Endpoint | (empty) | Aliyun OSS endpoint | | Redis | Host | `167.114.174.113` | Redis host | | Redis | Port | 6379 | Redis port | | Redis | Database | 0 | Redis database index | | Jwt | Secret | goodgo-iam-service-secret-key-32chars! | JWT secret | | Jwt | Issuer | goodgo-platform | JWT issuer | | Jwt | AccessTokenExpiryMinutes | 15 | Token TTL | | CDN | Enabled | false | CDN integration toggle | | CDN | BaseUrl | (empty) | CDN base URL | | IamService | BaseUrl | `http://localhost:5001` | IAM service URL | | IamService | TimeoutSeconds | 30 | HTTP timeout | --- ## Architecture Notes ### Upload Patterns 1. **Server-Side Upload** (`POST /files/upload`): File goes through the API, uploaded to MinIO by the handler. Simple but doesn't scale for millions of concurrent uploads. 2. **Direct Client Upload** (`POST /storage/sign-upload` + `POST /storage/confirm-upload`): Client gets pre-signed PUT URL, uploads directly to MinIO, then confirms. Bypasses backend for file transfer -- highly scalable. 3. **Multipart Upload** (`POST /files/multipart/initiate` -> upload parts -> `POST /files/multipart/complete`): For large files, uploaded in 5MB chunks. 24-hour expiration. ### Caching Strategy - **File metadata**: Redis cache-aside, 10min TTL - **User quota**: Redis cache-aside, 5min TTL - Cache invalidation on: file delete, upload confirm, quota update ### Security - All user-facing endpoints require JWT Bearer auth - File ownership validation on all operations (except admin endpoints) - Share passwords hashed with PBKDF2-SHA256 (100K iterations, 16-byte salt) - Share tokens: 32 random bytes, base64url-encoded - Admin endpoints require `Admin` or `SuperAdmin` role - Object key ownership validation via path structure (`{access}/{userId}/...`) ### Object Key Structure ``` {accessLevel}/{userId}/{yyyyMMdd}/{8-char-uuid}_{sanitized-filename} ``` - accessLevel: `public`, `private`, or `shared` - File names sanitized (invalid chars removed, max 100 chars + extension) ### MediatR Pipeline `LoggingBehavior` -> `ValidatorBehavior` -> `TransactionBehavior` -> Handler ### Resilience - PostgreSQL: retry on failure (5 retries, 30s max delay) - IAM HTTP client: Polly retry (3 retries, exponential backoff) + circuit breaker (5 failures, 30s break) - Storage delete: best-effort (continues with soft delete even if provider delete fails) - Multipart upload abort: best-effort provider cleanup ### Tests - **Unit Tests**: StorageFile, FileShare, Folder domain entity tests; UploadFileCommandHandler, DeleteFileCommandHandler, SignUploadCommandHandler, FileShareCommandHandler tests - **Functional Tests**: FilesApi, FileSharingApi, SignedUrlApi tests with CustomWebApplicationFactory (InMemory DB)