deltaforge-compute

The headless compute node. Deploy as a native binary on any cloud VM via systemd, or as a container in Docker or Kubernetes. The control plane requires a Postgres instance. No JVM, no Spark cluster to manage.

Signed binaries for every platform: deltaforge-org/delta-forge on GitHub

Install

The standard production deployment. Install the binary (detects x64 or ARM64), drop a systemd unit, and start the service.

install
curl -fsSL https://deltaforge.org/install.sh | sh -s -- --pkg deltaforge-compute
systemd unit
sudo tee /etc/systemd/system/deltaforge-compute.service >/dev/null <<EOF
[Unit]
Description=DeltaForge Compute Node
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/local/bin/deltaforge-compute
Restart=on-failure
RestartSec=5
Environment=DELTAFORGE_LICENSE_KEY=your-license-key
Environment=DELTAFORGE_CONTROL_PLANE=https://console.deltaforge.org

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now deltaforge-compute

Once running, the node registers itself with your control plane workspace. Add more nodes at any time to scale throughput. See the autoscaling guide.

Useful for local development and testing compute node behaviour before deploying to a cloud VM.

curl -fsSL https://deltaforge.org/install.sh | sh -s -- --pkg deltaforge-compute

Or via Homebrew:

# Add the tap (one-time)
brew tap deltaforge-org/tap
brew install deltaforge-compute

Useful for local development on Windows workstations. For production, use the Linux systemd path.

PowerShell
iwr -useb https://deltaforge.org/install.ps1 | iex; Install-DeltaForge -Pkg deltaforge-compute

Or via winget:

winget install DeltaForge.Compute

How it fits together

Compute nodes run inside your cloud environment and read Delta Lake and Iceberg tables directly from object storage. They report license and metering data to the DeltaForge control plane over outbound HTTPS.

%%{init: {'theme': 'dark', 'themeVariables': {'primaryColor': '#1e1e2e', 'primaryBorderColor': '#b95731', 'lineColor': '#b95731', 'secondaryColor': '#13131a', 'tertiaryColor': '#0d0d14', 'clusterBkg': '#1a1b23', 'clusterBorder': '#444', 'edgeLabelBackground': '#13131a', 'nodeTextColor': '#e0e0e0'}}}%%
flowchart TB
    subgraph clients["Client Tools"]
        pbi["Power BI / Excel / Tableau"]
        db["DBeaver / JDBC Applications"]
        ai["Claude / Cursor via MCP"]
        dfd["DeltaForge Desktop"]
    end

    subgraph cp["Control Plane (self-hosted or cloud)"]
        cpsvc["deltaforge-control-plane :3000"]
        pg[("PostgreSQL")]
        cpsvc --- pg
    end

    subgraph cloud["Your Cloud: AWS, Azure, On-Premises"]
        lb["Load Balancer / ASG / VMSS"]

        subgraph nodes["Compute Fleet - horizontal scale"]
            n1["deltaforge-compute"]
            n2["deltaforge-compute"]
            n3["deltaforge-compute ... +N"]
        end

        subgraph store["Object Storage: S3 / ADLS Gen2"]
            dl["Delta Lake Tables"]
            ice["Iceberg Tables"]
        end
    end

    pbi & db --"ODBC / ADBC"--> cpsvc
    ai --"MCP"--> cpsvc
    dfd --"HTTP API"--> cpsvc
    cpsvc --"query routing"--> lb
    lb --> n1 & n2 & n3
    n1 & n2 & n3 --"Parquet I/O"--> store

AWS EC2

Install on any EC2 instance (x86-64 or ARM64), point table LOCATION paths at s3:// URIs, and IAM role credentials are picked up automatically from the instance profile.

Azure VM

Deploy on any Azure VM or VMSS. Table LOCATION paths use abfss:// (ADLS Gen2) or wasbs:// (Blob Storage) URIs. Managed Identity credentials are picked up automatically.

Container or native binary

Deploy as a Docker container, in Kubernetes via Helm, or as a standalone binary on a VM fleet managed by an Auto Scaling Group or VMSS. The control plane also runs containerised and requires Postgres.

Need more nodes? Scale horizontally.

Add compute nodes from the Console and let the autoscaler handle fleet sizing for you.

Add nodes from the Console Autoscaling guide