Choosing a Machine to Run iconik Storage Gateway with LucidLink Choosing a Machine to Run iconik Storage Gateway with LucidLink
  1. iconik
  2. iconik Storage Gateway (ISG) & Edge Transcoder
  3. iconik Storage Gateway (ISG)

Choosing a Machine to Run iconik Storage Gateway with LucidLink

iconik Storage Gateway (ISG)

See all 19 articles

Running iconik Storage Gateway (ISG) on top of LucidLink lets you treat cloud object storage as a fast, local filesystem while ISG handles ingest, checksums, and transcodes. To avoid bottlenecks, you need to size the machine for both ISG and LucidLink, and configure ISG to understand LucidLink.

This guide covers:

  • OS and hardware sizing
  • How to calculate a sensible LucidLink cache
  • Steps to enable LucidLink’s upload API in ISG

Operating System

Choose an OS that is supported by both ISG and LucidLink:

Windows: Windows 10,11, or Windows Server 2019/2022

macOS: Monterey, Ventura, Sonoma, or later within ISG’s “latest three macOS versions”

Linux: Ubuntu 22.04/24.04 or a current Enterprise Linux 8/9
 

For a dedicated ISG + LucidLink host, Linux or Windows Server are usually simplest for always-on operation.

CPU Sizing

4 CPU cores per HD transcode stream are required for the best performance. Heavier codecs or UHD will need more. LucidLink itself is relatively light on CPU. If you size correctly for ISG transcodes and checksums, LucidLink will be fine.

Use this formula to calculate the best value:

max-transcoding-jobs = number of CPU cores / 4

For example, a machine with 16 cores can handle 4 HD transcode streams (16/4=4), so max-transcoding-jobs should be set to 4.

Memory Sizing

The ISG requires at least the following:

  • 4 GB of RAM for the first stream
  • 2 GB of RAM for each additional stream
     

Then, add 8–12 GB for OS, LucidLink, and other services.

Here is an example on calculating memory for an ISG with 4 transcode streams:

  • 4 Streams: 4GB + (3 × 2GB) = 10 GB
  • Overhead: +8–12 GB
  • 10GB + 8-12GB  = 18–22 GB → choose 32 GB as a practical minimum
     

For heavier workloads (more streams, UHD, complex workflows), 64 GB is safer. 

Storage and LucidLink Cache

You have three main storage needs:

  1. OS + applications: Around 80 GB or more for OS, ISG, and tools
  2. ISG scratch / temp: For transcodes, thumbnails, and any intermediate files
  3. LucidLink cache: Default (around 25 GiB) is often too small for transcode-heavy setups. See below to calculate recommended cache

Use this to estimate a sensible cache size:

Ideal cache size ≈
(number of simultaneous transcodes) × (maximum data rate of media) × ( longest single asset duration) × 1.2

Where:

  • Simultaneous transcodes = your peak number of local transcodes
  • Maximum data rate = in MB/s (e.g., ~25 MB/s for 200 Mb/s media)
  • Maximum length = in seconds
  • 1.2 = 20% safety margin
     

For example:

  • Simultaneous transcodes = 4 
  • Maximum data rate = 25 MB/s
  • Maximum length = 3600 seconds (1 hour)
     

Cache ≈ 4 × 25 × 3600 × 1.2 ≈ 432,000 MB ≈ 422 GiB

In practice, you’d round up to a 500 GB–1 TB SSD for LucidLink cache for this kind of workload. Smaller workloads scale down proportionally, but serious transcode environments should expect significantly more than the default.

Network and Bandwidth

For the ISG to scan, checksum, and index files stored in LucidLink’s cloud backend, those files first have to be pulled down into the LucidLink cache over the host machine’s network connection. The speed at which new or uncached media becomes visible and usable to ISG is ultimately limited by the host machine’s WAN bandwidth.

Rough calculation for required bandwidth:

Total bitrate = sum of bitrates for all simultaneous streams

Example: 4 × 500 Mb/s = 2 Gb/s → can saturate 1 GbE, so 10 GbE and a robust, symmetrical connection are recommended.

Enabling LucidLink Upload API in ISG

ISG can use LucidLink’s upload indicator API to confirm when a file is fully uploaded via LucidLink before ingesting it. It is highly recommended that users enable this setting to avoid partially ingested files.

Steps to Enable:

  1. Stop the ISG service.

  2. Edit the config.ini file and add this line:

    enable-lucidlink-upload-api=true

    Config file locations (config.ini):

    • Linux: /etc/iconik/iconik_storage_gateway/config.ini
    • Windows: C:\ProgramData\Cantemo\IconikStorageGateway\config.ini
    • macOS: /Library/Application Support/Cantemo/IconikStorageGateway/config.ini
  3. Restart the ISG service

Example Host Profiles

Light transcode / small team:

  • 1–2 HD transcodes
  • CPU: 8 cores
  • RAM: 16–32 GB
  • Storage:
    • 256–512 GB SSD for OS/apps
    • 250–500 GB SSD for LucidLink cache (per formula)
  • Network: 1–10 GbE (10 GbE preferred), solid business internet

Heavy transcode / long-form

  • 4–6 HD or 2–3 UHD transcodes
  • CPU: 16–24 cores
  • RAM: 64 GB+
  • Storage:
    • 512 GB SSD for OS/apps
    • 1 TB+ NVMe for LucidLink cache
  • Network: 10 GbE+; high-quality fibre