Storage

Storage API

Multi-tenant MinIO storage management with bucket initialization, file upload/download, usage analytics, and quota management. Organize files by bucket types (primary, documents, archive) with comprehensive tracking.

15 EndpointsMinIO/S3Multi-TenantQuota Management

Overview

Storage Architecture

• Primary Bucket: General file storage
• Documents Bucket: Document management
• Archive Bucket: Long-term storage
• Auto-Routing: Category-based organization

Features

• Bucket initialization per organization
• File upload with validation
• Usage analytics and quotas
• Hierarchical file management
• Orphan file cleanup (admin)

Bucket Management

POST/v1/storage/buckets/initializeAuth Required

Initialize Organization Buckets

Initialize all required buckets for the current organization based on subscription plan. Creates buckets, configures versioning/object lock, and sets up lifecycle policies.

Response

Responsejson

{
  "message": "Bucket initialization completed",
  "org_id": "org_abc123",
  "plan": "professional",
  "buckets": {
    "org-org_abc123-primary": true,
    "org-org_abc123-documents": true,
    "org-org_abc123-archive": true
  },
  "created_database_records": [
    "org-org_abc123-primary",
    "org-org_abc123-documents"
  ],
  "success_count": 3,
  "total_count": 3
}

Try it out

cURLbash

curl -X POST "https://sapienstream.com/api/v1/storage/buckets/initialize" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

GET/v1/storage/bucketsAuth Required

List Organization Buckets

List all buckets for the current organization with usage statistics and quota information.

Response

Responsejson

{
  "org_id": "org_abc123",
  "total_storage_bytes": 536870912,
  "total_storage_gb": 0.5,
  "total_objects": 125,
  "buckets": [
    {
      "bucket_name": "org-org_abc123-primary",
      "bucket_type": "primary",
      "storage_used_bytes": 268435456,
      "object_count": 75,
      "versioning_enabled": true,
      "object_lock_enabled": true,
      "last_activity_at": "2024-08-26T15:30:00Z",
      "created_at": "2024-08-01T10:00:00Z"
    },
    {
      "bucket_name": "org-org_abc123-documents",
      "bucket_type": "documents",
      "storage_used_bytes": 268435456,
      "object_count": 50,
      "versioning_enabled": true,
      "object_lock_enabled": false,
      "last_activity_at": "2024-08-25T12:00:00Z",
      "created_at": "2024-08-01T10:00:00Z"
    }
  ],
  "quota": {
    "max_storage_bytes": 10737418240,
    "usage_percentage": 5.0,
    "is_quota_exceeded": false
  },
  "last_updated": "2024-08-26T15:30:00Z"
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/buckets" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

File Upload

POST/v1/storage/uploadAuth Required

Upload File

Upload file with automatic bucket routing and comprehensive validation. Validates file size and type against subscription limits, routes to appropriate bucket, creates database record with metadata.

Request Body

Requestjson

// multipart/form-data
file: [binary data]              // Required: The file to upload
machine_id: "mach_001"           // Required: Associated machine ID
bucket_type: "primary"           // Optional: Bucket type (default: primary)
category: "documents"            // Optional: File category
subcategory: "manuals"           // Optional: Subcategory
description: "Machine manual"    // Optional: File description
document_kind: "manual"          // Optional: Document type (default: other)

Response

Responsejson

{
  "message": "File uploaded successfully",
  "document_id": "doc_abc123",
  "bucket_name": "org-org_abc123-primary",
  "object_key": "2024/08/documents/manuals/machine_manual.pdf",
  "file_size": 1234567,
  "etag": "d41d8cd98f00b204e9800998ecf8427e",
  "upload_timestamp": "2024-08-26T15:30:00Z",
  "storage_usage": {
    "total_bytes": 536870912,
    "usage_percentage": 5.0
  }
}

Try it out

cURLbash

curl -X POST "https://sapienstream.com/api/v1/storage/upload" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

POST/v1/storage/files/uploadAuth Required

Upload Files to Path

Upload multiple files to a specific path within the organization's bucket. Creates both MinIO objects and database Document records for tracking.

Request Body

Requestjson

// multipart/form-data
path: "my-folder/subfolder"      // Optional: Target path
files: [binary data]             // Required: One or more files
bucket_type: "primary"           // Optional: Bucket type (default: primary)

Response

Responsejson

{
  "message": "Uploaded 3 files successfully",
  "files": [
    {
      "filename": "document1.pdf",
      "document_id": "doc_abc123",
      "object_key": "my-folder/subfolder/document1.pdf",
      "size": 1234567,
      "etag": "d41d8cd98f00b204e9800998ecf8427e"
    },
    {
      "filename": "image.jpg",
      "document_id": "doc_def456",
      "object_key": "my-folder/subfolder/image.jpg",
      "size": 234567,
      "etag": "e10adc3949ba59abbe56e057f20f883e"
    }
  ],
  "bucket_name": "org-org_abc123-primary",
  "path": "my-folder/subfolder"
}

Try it out

cURLbash

curl -X POST "https://sapienstream.com/api/v1/storage/files/upload" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

POST/v1/storage/files/create-folderAuth Required

Create Folder

Create a new folder by uploading an empty object with a trailing slash.

Request Body

Requestjson

// multipart/form-data
path: "existing-folder"          // Optional: Parent path
name: "new-folder"               // Required: Folder name
bucket_type: "primary"           // Optional: Bucket type (default: primary)

Response

Responsejson

{
  "message": "Folder created successfully",
  "folder_path": "existing-folder/new-folder/",
  "bucket_name": "org-org_abc123-primary"
}

Try it out

cURLbash

curl -X POST "https://sapienstream.com/api/v1/storage/files/create-folder" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

File Browsing

GET/v1/storage/files/listAuth Required

List Files and Folders

List files and folders in a specific path within the organization's bucket. System folders (components, machines, pdfs, .system) are hidden at root level.

Parameters

pathstring

Path within the bucket to list (default: root)

bucket_typestring

Bucket type to browse (default: primary)

Response

Responsejson

{
  "path": "documents",
  "bucket_name": "org-org_abc123-primary",
  "files": [
    {
      "name": "manuals",
      "type": "folder"
    },
    {
      "name": "reports",
      "type": "folder"
    },
    {
      "name": "readme.txt",
      "type": "file",
      "size": 1024,
      "lastModified": "2024-08-26T15:30:00Z",
      "etag": "d41d8cd98f00b204e9800998ecf8427e"
    }
  ]
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/files/list" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

GET/v1/storage/files/list-allAuth Required

List All Files in All Buckets

List ALL files across all organization buckets for diagnostics. Returns every physical file in MinIO regardless of database tracking.

Response

Responsejson

{
  "org_id": "org_abc123",
  "total_files": 125,
  "files": [
    {
      "filename": "machine_manual.pdf",
      "bucket_name": "org-org_abc123-primary",
      "object_key": "2024/08/documents/machine_manual.pdf",
      "file_size": 1234567,
      "last_modified": "2024-08-26T15:30:00Z",
      "etag": "d41d8cd98f00b204e9800998ecf8427e"
    },
    {
      "filename": "component_spec.pdf",
      "bucket_name": "org-org_abc123-documents",
      "object_key": "components/comp_001/spec.pdf",
      "file_size": 234567,
      "last_modified": "2024-08-25T12:00:00Z",
      "etag": "e10adc3949ba59abbe56e057f20f883e"
    }
  ]
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/files/list-all" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

GET/v1/storage/documentsAuth Required

List Organization Documents

List documents stored in organization buckets with filtering and pagination.

Parameters

bucket_typestring

Filter by bucket type

categorystring

Filter by file category (machine, component, independent)

limitinteger

Items per page (1-100, default: 50)

offsetinteger

Pagination offset (default: 0)

Response

Responsejson

{
  "org_id": "org_abc123",
  "total_count": 125,
  "documents": [
    {
      "id": "doc_abc123",
      "filename": "machine_manual.pdf",
      "title": "Machine Manual",
      "kind": "manual",
      "bucket_name": "org-org_abc123-primary",
      "object_key": "2024/08/documents/machine_manual.pdf",
      "file_size": 1234567,
      "mime_type": "application/pdf",
      "version": "1.0",
      "is_current_version": true,
      "download_count": 5,
      "created_at": "2024-08-26T15:30:00Z",
      "last_accessed_at": "2024-08-26T16:00:00Z",
      "vector_indexed": true,
      "collection_name": "org_abc123_docs",
      "chunk_count": 45,
      "file_category": "machine"
    }
  ],
  "pagination": {
    "limit": 50,
    "offset": 0,
    "has_more": true
  }
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/documents" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

File Download

GET/v1/storage/files/downloadAuth Required

Download File

Download a file from the organization's bucket. Returns the file as a streaming response.

Parameters

pathstringRequired

Full path to the file

bucket_typestring

Bucket type (default: primary)

Response

Responsejson

// Binary file response
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=machine_manual.pdf

[Binary file data]

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/files/download" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

File Deletion

DELETE/v1/storage/files/deleteAuth Required

Delete File or Folder

Delete a file or folder from the organization's bucket AND database. For folders, recursively deletes all contents.

Parameters

pathstringRequired

Path to file or folder

typestring

Type: 'file' or 'folder' (default: file)

bucket_typestring

Bucket type (default: primary)

Response

Responsejson

// File deletion
{
  "message": "File deleted successfully",
  "path": "documents/old_manual.pdf",
  "database_records_deleted": 1
}

// Folder deletion
{
  "message": "Folder deleted successfully",
  "path": "documents/old-folder",
  "deleted_objects": 15
}

Try it out

cURLbash

curl -X DELETE "https://sapienstream.com/api/v1/storage/files/delete" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

Usage Analytics

GET/v1/storage/usageAuth Required

Get Storage Usage Analytics

Get detailed storage usage analytics including current usage, quota information, and recent activity.

Response

Responsejson

{
  "org_id": "org_abc123",
  "plan_name": "professional",
  "current_usage": {
    "storage_bytes": 536870912,
    "storage_gb": 0.5,
    "object_count": 125,
    "buckets": {
      "org-org_abc123-primary": {
        "size_bytes": 268435456,
        "object_count": 75
      },
      "org-org_abc123-documents": {
        "size_bytes": 268435456,
        "object_count": 50
      }
    }
  },
  "quota": {
    "max_storage_bytes": 10737418240,
    "max_objects": 10000,
    "usage_percentage": 5.0,
    "is_quota_exceeded": false,
    "alert_threshold_percent": 80
  },
  "recent_activity": [
    {
      "event_type": "upload",
      "object_key": "2024/08/documents/new_file.pdf",
      "size_bytes": 1234567,
      "timestamp": "2024-08-26T15:30:00Z",
      "user_id": "user_001"
    }
  ],
  "last_updated": "2024-08-26T15:30:00Z"
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/usage" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

GET/v1/storage/statsAuth Required

Get Storage Statistics for Dashboard

Get simplified storage statistics optimized for dashboard display.

Response

Responsejson

{
  "total_files": 125,
  "total_size_bytes": 536870912,
  "total_size_formatted": "0.5 GB",
  "files_this_month": 25,
  "storage_used_gb": 0.5,
  "storage_limit_gb": 10.0,
  "utilization_percentage": 5.0,
  "org_id": "org_abc123",
  "last_updated": "2024-08-26T15:30:00Z"
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/stats" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

Health Check

GET/v1/storage/health

Storage System Health Check

Check the health of the storage system including MinIO connection status.

Response

Responsejson

{
  "status": "healthy",
  "minio_connection": "ok",
  "minio_accessible": true,
  "bucket_count": 25,
  "endpoint": "minio.sapienstream.com:9000",
  "timestamp": "2024-08-26T15:30:00Z"
}

Try it out

cURLbash

curl -X GET "https://sapienstream.com/api/v1/storage/health" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

Admin Operations

System Admin Only: These endpoints require system administrator privileges.

POST/v1/storage/cleanup-orphanedAuth Required

Clean Up Orphaned Files

Clean up orphaned files in MinIO that don't have corresponding database records. Supports dry-run mode to preview what would be deleted.

Parameters

dry_runboolean

If true, only report what would be deleted (default: true)

org_idstring

Organization ID to cleanup (admin can specify, otherwise uses own org)

Response

Responsejson

{
  "org_id": "org_abc123",
  "dry_run": true,
  "orphaned_files": [
    {
      "bucket_name": "org-org_abc123-primary",
      "object_key": "orphan_file.pdf",
      "size_bytes": 1234567
    }
  ],
  "total_orphaned": 1,
  "total_size_bytes": 1234567
}

Try it out

cURLbash

curl -X POST "https://sapienstream.com/api/v1/storage/cleanup-orphaned" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

DELETE/v1/storage/cleanup-orphaned-filesAuth Required

Delete Orphaned Files

Permanently delete orphaned files that don't have database records. System admin only.

Parameters

dry_runboolean

If true, only report what would be deleted

org_idstring

Organization ID to cleanup

Response

Responsejson

{
  "org_id": "org_abc123",
  "deleted_files": 5,
  "freed_bytes": 5368709,
  "message": "Cleanup completed successfully"
}

Try it out

cURLbash

curl -X DELETE "https://sapienstream.com/api/v1/storage/cleanup-orphaned-files" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json"

Complete Upload Example

// 1. Initialize buckets for organization (first time only)
const initResponse = await fetch('/v1/storage/buckets/initialize', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer ' + token
  }
});

const initResult = await initResponse.json();
console.log('Buckets initialized:', initResult.buckets);

// 2. Upload file to specific path
const formData = new FormData();
formData.append('file', fileBlob);
formData.append('machine_id', 'mach_001');
formData.append('category', 'documents');
formData.append('subcategory', 'manuals');
formData.append('description', 'Machine operation manual');
formData.append('document_kind', 'manual');

const uploadResponse = await fetch('/v1/storage/upload', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer ' + token
  },
  body: formData
});

const uploadResult = await uploadResponse.json();
console.log('File uploaded:', uploadResult.document_id);

// 3. Check storage usage
const usageResponse = await fetch('/v1/storage/usage', {
  headers: {
    'Authorization': 'Bearer ' + token
  }
});

const usage = await usageResponse.json();
console.log('Storage used:', usage.current_usage.storage_gb, 'GB');
console.log('Quota:', usage.quota.usage_percentage + '%');

File Browser Example

// List files in a folder
const listResponse = await fetch('/v1/storage/files/list?path=documents/manuals&bucket_type=primary', {
  headers: {
    'Authorization': 'Bearer ' + token
  }
});

const { files } = await listResponse.json();

// Separate folders and files
const folders = files.filter(f => f.type === 'folder');
const docs = files.filter(f => f.type === 'file');

// Create a new folder
const folderFormData = new FormData();
folderFormData.append('path', 'documents');
folderFormData.append('name', 'new-folder');

await fetch('/v1/storage/files/create-folder', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer ' + token
  },
  body: folderFormData
});

// Upload files to the new folder
const uploadFormData = new FormData();
uploadFormData.append('path', 'documents/new-folder');
uploadFormData.append('files', file1);
uploadFormData.append('files', file2);

await fetch('/v1/storage/files/upload', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer ' + token
  },
  body: uploadFormData
});

// Download a file
const downloadResponse = await fetch(
  '/v1/storage/files/download?path=documents/new-folder/file1.pdf',
  {
    headers: {
      'Authorization': 'Bearer ' + token
    }
  }
);

const blob = await downloadResponse.blob();