No GPU? Rent One by the Minute

I built modl on a machine with a 24GB GPU, but I don’t always have it — and a lot of people who want to train a LoRA or run a big overnight batch never had one to begin with. The usual answer is a hosted service: upload your photos to someone’s cloud, pay their margin, get a file back. Pods are the other answer: rent a GPU directly from the Vast.ai marketplace with your own API key, and let modl treat it like your own hardware for as long as you keep it.

One command rents and bootstraps the machine. After that, any job takes a --pod flag — generate, edit, train, run — and executes there instead of locally. Every terminal transcript in this guide is from real sessions this week, costs included.

The idea

modl pod up searches the marketplace, rents the best-value instance, and installs modl itself on it — the same binary, the same Python runtime, the same content-addressed model store you run locally. Then your laptop drives it over plain SSH:

• Your key, your machine. The pod is rented on your Vast.ai account. modl never proxies your traffic or your billing — there’s no hosted middleman and no cloud storage. Dataset up, artifacts down, all directly between your laptop and the pod.
• Artifacts land in your library. A LoRA trained on a pod is registered into your local ~/modl/store when it finishes. Images from a pod run are imported into ~/.modl/outputs, where modl outputs and the web UI already look.
• Jobs are fire-and-forget. The run executes on the pod under its own process. Close the laptop; the run finishes without you and syncs home when you reconnect.

The pod is disposable. The YAML files, LoRAs, and images that come back are the durable part.

Setup

Two things, once:

1. A Vast.ai account with credit and an SSH key added at cloud.vast.ai/account (modl connects over SSH; a missing key is the most common first-run failure).
2. Your API key where modl looks for it:

You need modl v0.2.15 or later — the pod installs a matching copy of itself from the release, so older versions can’t bootstrap.

Your first pod

Ask for a GPU by name. modl quotes the all-in price before renting — Vast bills GPU time and disk as separate line items, and dashboards that show only the GPU rate undersell what you’ll actually pay. modl prices the storage in up front:

Yes, that’s a real transcript, dud hosts and all — more on that in the reliability section. The part that matters: every dead host was destroyed automatically before moving on, so you never pay for a machine that didn’t work. The one that boots gets modl installed, plus a managed PyTorch runtime, and stays up for reuse.

Check what’s running — and what it has cost so far:

Generate on the pod

With a pod up, --pod sends any generation there. modl installs the model on the pod first (the pod has its own store, and its download pipe is usually much fatter than your home connection):

The image is on your laptop when the command returns — imported into your outputs library with its sidecar metadata, not sitting in some bucket.

Workflows run the same way. Write the YAML exactly as you would locally — multi-step, per-step models, seed sweeps — and add --pod:

name: scene-batch
model: flux2-klein-9b

images:
  alice: "data:image/png;base64,iVBORw0KGgo..."   # character reference

steps:
  - id: portraits
    generate: "alice in a sunlit study, oil painting style"
    seeds: [42, 7, 99]

  - id: scene
    edit: "$alice"
    prompt: "alice reading under a tree at golden hour"

Train a LoRA

This is the flow pods were built for. modl train --pod ships your dataset over SSH, trains on the rented GPU, and registers the finished LoRA into your local store — indistinguishable from a LoRA you trained on your own hardware:

That last line is the actual bill for that run: five minutes, three cents. A real character or style LoRA (1500–3000 steps on a bigger base model) runs longer, but the arithmetic stays friendly — an hour of 3090 is about $0.21, and you only pay for minutes you use.

Notice train --pod destroyed the pod when it finished — training rents are torn down by default so a forgotten session can’t bill overnight. If you’re iterating on training runs, keep the machine warm instead:

The next --pod job reuses the warm pod automatically — models installed, runtime ready, no re-bootstrap. Pass --fresh to pod up when you want to swap the machine out (the replacement is provisioned before the old pod is destroyed, so you’re never left podless).

LoRAs and Lightning on pods

LoRAs work in both directions:

• Registry LoRAs (Lightning, anything in the modl registry) are pulled pod-side on first use, exactly like models.
• Your local LoRAs — including ones you just trained — are pushed automatically: modl syncs the file into the pod’s content-addressed store and registers it there. --lora my-style works on a pod with zero extra steps.

fast: is a first-class workflow field, so Lightning fast mode travels too:

name: quick-drafts
model: flux2-klein-9b
steps:
  - id: drafts
    generate: "storyboard sketch, a lighthouse in a storm"
    fast: 4          # auto-applies the Lightning LoRA + scheduler overrides
    seeds: [1, 2, 3, 4]

Four-step drafts on a rented 3090 cost fractions of a cent each — a good way to explore compositions before spending real steps on the keeper.

Close the laptop

Pod runs are fire-and-forget by design. The run executes under its own process on the pod — your SSH connection dropping doesn’t touch it. Submit a batch, close the lid, come back later:

Three tools for re-attaching:

• modl status <run-id> --pod — aggregate status straight from the pod’s own job database.
• modl pod logs <run-id> -f — tail the run’s log live, like the job was running in your terminal all along.
• modl pod pull <run-id> — fetch a finished run’s artifacts if the automatic sync-home didn’t get a chance to run (laptop died mid-run, for example). Runs finish on the pod regardless; pulling is idempotent.

Give your agent a GPU

The part I use most. modl’s MCP server exposes the whole pod lifecycle as tools, which means Claude Code on a GPU-less laptop can rent a GPU, run a workflow on it, collect the images, and give the machine back — while you watch or don’t.

The MCP config (~/.claude/mcp_servers.json — local modl mcp, or over SSH to any machine that has modl and your Vast key):

{
  "modl": {
    "command": "modl",
    "args": ["mcp"]
  }
}

The agent flow, as tool calls:

pod_up returns immediately; the agent polls pod_ls until ready: true, submits work, polls job_status until synced_home: true — at which point the images are already in your local library — and destroys the pod. Reference images travel as base64 in the workflow’s images: map, same rule as the CLI.

For the fuller submit-from-anywhere pattern — same MCP tools, your own workstation instead of a rented pod — see the remote Mac workflow guide.

The cost math

Real numbers from this week’s sessions, all-in (GPU + storage — modl quotes both before you confirm):

The 24GB cards are the sweet spot: at fp8 they run every model in the registry except the two 20B+ giants, and the 3090 is reliably the value pick — same VRAM as the 4090 at roughly 60% of the price. Ask for bigger iron only when the model demands it; modl pod up a100-80gb quotes the all-in price and waits for your confirmation before renting.

What sessions actually cost me this week: a 100-step training smoke, $0.03. A full demo session — provision, install a model from scratch, generate, sync home, destroy — about $0.30. Rent, do the work, destroy. The meter only runs between pod up and pod rm.

Reliability, honestly

Vast.ai is a marketplace of other people’s machines, and some of them are duds — containers that fail to start, SSH that never comes up, hosts that hang in loading forever. The transcript in your first pod is genuinely representative: one evening this week it took four rents to get one working pod. Another attempt that same evening burned through every plausible offer and gave up cleanly:

modl’s job is to make that churn boring rather than pretend it doesn’t exist:

• Dud hosts are destroyed automatically the moment they fail — boot timeout, container error, or dead SSH. You don’t pay for machines that never worked, and you don’t have to notice them.
• Failover is price-capped. Fallback offers more than 25% over the price you were quoted are skipped, so a bad marketplace evening fails cleanly instead of quietly renting you something expensive.
• Nothing is left dangling. Every failure path ends with the instance destroyed and says so. modl pod ls is always the ground truth for what’s billing.

Most evenings, pod up succeeds on the first or second offer in two or three minutes. On a bad one, expect a few visible retries — or a clean error asking you to try again shortly. What you should never see is a silent zombie instance running up your bill.

When to use what

Three ways to run modl, one product:

They compose. I generate drafts locally on small models, rent a pod for training runs and anything my card can’t hold, and drive my workstation over SSH from a Mac when I’m away. Same YAML, same commands, same output library — the --pod flag is the only thing that changes.

Quick reference

The habit that matters: pod ls when you sit down, pod rm when you stand up. Everything in between is ordinary modl.