Skip to main content

Performance & Benchmarks

Lioran S3 is engineered in Rust for high-throughput streaming with bounded memory overhead.

Benchmark Results​

In controlled load testing on modern NVMe storage and dedicated gigabit/10-gigabit networking, Lioran S3 achieved the following benchmark throughput metrics:

┌───────────────────────────┬───────────────────────────┬───────────────────────────┐
│        300+ MB/s          │        300+ MB/s          │          100 GB           │
│     Write Throughput      │      Read Throughput      │      Tested Workload      │
└───────────────────────────┴───────────────────────────┴───────────────────────────┘
  • Write Throughput: Sustained 300+ MB/s streaming write throughput.
  • Read Throughput: Sustained 300+ MB/s streaming read throughput.
  • Tested Datasets: Validated under continuous 100 GB soak testing workloads.

:::note Benchmark Disclaimer Benchmark results vary based on hardware specifications, storage media (local NVMe vs SATA SSD vs cloud block volumes), object size distribution, concurrency, network topology, reverse proxy configuration, and configured write durability settings (strict vs balanced). These metrics represent tested results under benchmark workloads and are not universal guarantees. :::

Key Tuning Factors​

1. Durability Modes (BASTION_DURABILITY)​

  • In strict mode, every chunk and completed object write executes a synchronous fsync (sync_all) before committing metadata. This guarantees crash consistency on sudden power loss, with throughput bound by storage drive sync latency.
  • In balanced mode, synchronous fsync calls are omitted, allowing the OS to manage writeback caching. This mode yields peak throughput on enterprise NVMe drives or battery-backed controllers.

2. Streaming Buffer Chunk Size (BASTION_STREAM_CHUNK_KIB)​

  • Default is 256 KiB (valid range: 64 to 4096 KiB).
  • Larger chunk sizes (e.g. 1024 or 2048 KiB) can increase sequential throughput on high-bandwidth networks, while smaller chunk sizes reduce per-connection memory footprint under extreme concurrency.

3. Multipart Concurrency​

When uploading large files via @liorans3/driver or liorans3 multipart upload:

  • Setting --concurrency 4 or 8 parallelizes chunk ingestion across available CPU cores and network streams.
  • Recommended part size: 32 MiB (or 64 MiB for files > 10 GB).

4. Reverse Proxy Unbuffered Streaming​

Ensure Caddy or your reverse proxy uses unbuffered chunk forwarding (flush_interval -1). Proxy response buffering will introduce latency and inflate proxy memory usage during large transfers.