Skip to main content

Getting Started with Lioran S3

Lioran S3 (engine name: Lioran Bastion) is a high-performance, single-node object storage server and media processing engine engineered in Rust. It provides scalable bucket and object semantics, bounded-memory streaming I/O, RocksDB-backed metadata persistence, resumable multipart uploads, expiring HMAC-SHA256 signed URLs, quota enforcement, and an official TypeScript driver and CLI.

:::info Pre-Alpha Status Lioran S3 is currently in active pre-alpha development (v0.1.0-prealpha). While the storage engine, metadata architecture, CLI, and TypeScript driver are fully implemented and verified, interfaces may evolve between releases. :::

What is Object Storage?​

Unlike traditional hierarchical file systems (POSIX directories and files) or block storage (raw disk blocks), object storage treats data as discrete, self-contained units known as objects.

Each object consists of:

  • Payload Data: The raw file bytes (images, videos, archives, datasets, documents).
  • Object Key: A unique string identifier within a bucket (e.g., uploads/2026/report.pdf).
  • System Metadata: Size in bytes, content MIME type, creation/modification timestamps, ETags, and cryptographic SHA-256 checksums.
  • Custom Metadata: Optional user-defined key-value attributes.

Objects reside within flat namespaces called Buckets. Buckets can have storage quotas and versioning configurations, but objects cannot be nested inside physical subfolders—slashes (/) in object keys are purely conceptual prefixes used for filtering and organization.

Architectural Planes​

Lioran S3 strictly decouples the Metadata Plane from the Object Data Plane:

               Client Applications / Web Apps / Tools
                                 │
                   ┌─────────────┴─────────────┐
                   │                           │
          @liorans3/driver              @liorans3/cli
                   │                           │
                   └─────────────┬─────────────┘
                                 │ HTTP / REST API (Port 27118)
                                 ▼
                    ┌─────────────────────────┐
                    │     Lioran Bastion      │
                    │   Axum HTTP Router      │
                    │   Auth & Middleware     │
                    └────────────┬────────────┘
                                 │
           ┌─────────────────────┴─────────────────────┐
           ▼                                           ▼
┌───────────────────────────┐               ┌───────────────────────────┐
│      Metadata Plane       │               │      Data / I/O Plane     │
│    (bastion-metadata)     │               │     (bastion-object)      │
├───────────────────────────┤               ├───────────────────────────┤
│ • Embedded RocksDB Store  │               │ • Bounded Streaming I/O   │
│ • Bucket Configurations   │               │ • Temporary Staging Space │
│ • Object Metadata & ETags │               │ • Committed Payload Blobs │
│ • Quotas & Secondary Index│               │ • Atomic File Promotion   │
│ • Multipart Upload State  │               │ • Byte-Range Slicing      │
└───────────────────────────┘               └───────────────────────────┘

1. Metadata Plane (RocksDB)​

Metadata records—including bucket definitions, quota usage, object metadata, SHA-256 hashes, and multipart upload sessions—are managed by an embedded RocksDB key-value store.

Strict Invariant: Object payload bytes are never stored within the RocksDB metadata database.

2. Data Plane (Storage Volumes)​

Physical file payloads are streamed directly between network sockets and storage volumes on disk. All mutating writes follow a staging and promotion protocol:

  1. Bytes are written to an isolated temporary staging file designated by an internal UUID v7.
  2. Checksums (SHA-256) are calculated incrementally as chunks stream.
  3. The staged file is flushed and synced to disk storage (fsync in strict durability mode).
  4. Metadata is atomically committed to RocksDB.
  5. The temporary file is atomically renamed/promoted into the active object storage volume.

Connection URI Format​

Lioran S3 components (CLI and Driver) communicate with the server using standard HTTP/HTTPS URLs or native Bastion connection URIs:

bastion://[identity]:[secret]@[host]:[port]

Examples:​

  • Local Basic Authentication (Dev):

    bastion://admin:YOUR_PASSWORD@127.0.0.1:27118

  • Programmatic Access Key:

    bastion://bk_live_0192a7b8:sk_live_9f83a21b4c7e6d5a@127.0.0.1:27118

  • Production HTTPS Endpoint (Behind Reverse Proxy):

    https://s3.example.com

Ecosystem Packages​

ComponentPackage / BinaryDescription
Serverbastion-serverHigh-performance Rust server binary.
CLI@liorans3/cliOfficial command-line interface (liorans3).
Driver@liorans3/driverOfficial TypeScript & JavaScript client SDK for Node.js / Bun.

Next Steps​