Why Metrale
You invested in the datacenter. Now get the most out of it.
Metrale models your inference economics first, then runs, governs and measures every workload against that model directly. A fundamentally different approach to getting more inference, and more accountability, from silicon you already own.
Speed
Compiled per hardware, model and quantization. More tokens per GPU, and the curve keeps climbing where agent fleets run.
Security
One signed Rust binary, no interpreter in the request path, nothing leaves your perimeter.
Governance
Every token has a receipt. Every GPU hour has an owner. Finance can sign it.
01 · Speed
The same GPUs produce more inference, and the curve holds where it matters.
Generic runtimes ship one code path for every accelerator and tune it in Python. Metrale compiles the path for the silicon in front of it.
What it is
A pure Rust and CUDA inference engine with hand tuned kernels per hardware, model and quantization target, NVFP4 and FP8 quantization, speculative decoding with MTP draft heads and DFlash block diffusion, radix tree prefix caching and expert parallelism across nodes.
How it is built
Every kernel target has a compatibility manifest and a gate. A release ships only after the serve matrix passes on the real box, and a release that ships slower than its committed baseline fails the gate. Numbers on this site are generated from those records, never typed.
Why it changes the outcome
Agentic work arrives as fleets of tool calling agents sharing a system prompt, not one conversation at a time. On the published GB10 ladder Metrale wins every rung from C=1 to C=128 and keeps scaling at the top while the matched vLLM configuration flattens. That gap is how many agents one box can run, and it compounds across a rack.
Why it is hard to build
It is a decision made at the start, not a feature added later. Leaving the Python and PyTorch ecosystem meant rewriting the serving stack from scratch in Rust and running an AI first repository where every kernel change must carry a certified benchmark before it merges.
Ask for the throughput curve at C=128 on your workload, not a single stream number on theirs.
Read the ladder and every gate record →02 · Security
Nothing leaves your perimeter, and there is nothing in the request path you did not sign.
A serving stack that pulls two hundred dependencies at startup is a supply chain you did not audit. Metrale is one binary.
What it is
One signed binary of about 75 MB with no Python, no PyTorch and no runtime compilation. Recipes, models and kernels arrive as signed artifacts. Node to node links are designed for a hostile network, so the mesh can span racks, sites and edges you already have with no private backbone.
How it is built
Cargo deny audits every dependency on every pull request. Kernel targets are content hashed down to their transitive include closure, so a build can prove what it was compiled from. Release images promote by digest from staging to canary to production, and are never rebuilt between environments.
Why it changes the outcome
Regulated buyers cannot move at hyperscaler speed because every deployment re proves its own compliance. Pinned, signed, repeatable deployment primitives turn that overhead from a blocker into the reason to buy. Prompts, weights and telemetry stay where the policy says they stay.
Why it is hard to build
Security is a property of the request path, not a feature beside it. It is only cheap when the engine, the control plane and the deployment primitives were designed by the same people at the same time, which is the team Metrale started with.
Ask what is in the request path, and who audits the two hundred dependencies behind it.
Read the security posture →03 · Governance
Every token has a receipt. Every GPU hour has an owner.
The industry measures inference in tokens per second. Enterprises pay for it in dollars per workload. Metrale owns the layer between the two.
What it is
A correlation layer that ties each unit of work to the workload, model, runtime, configuration, GPU and cluster that produced it, then a ledger that reports cost per million tokens, cost per successful workload at SLO, productive GPU hours, stranded capacity and savings against the baseline, by business unit.
How it is built
The engine owns the request path, so utilization, cache behaviour and queue pressure are measured at the source rather than inferred from a proxy. The control plane records every rollout, canary, rollback and policy decision with a signed recipe and a gate record attached.
Why it changes the outcome
The decision maker walks to the CFO with a payback period rather than a percentage. Everything after payback is upside, and every month it is recomputed from production data rather than a vendor estimate.
Why it is hard to build
You cannot attribute cost to a workload you did not serve. Platforms that sit beside the engine can only estimate. Metrale is under the workload, which is the only place the ledger can be exact.
Ask what a workload cost last Tuesday, by business unit. A tokens per second chart is not an answer.
See Metrale Economics →Only Metrale
Each layer earns the next.
The engine earns the deployment
Nobody installs a governance layer for its own sake. Metrale gets into the fleet by making the same GPUs produce more inference.
The deployment earns the telemetry
Once the engine owns the request path, every kernel, cache and queue decision is observed at the source instead of inferred from a proxy.
The telemetry earns the economics
With source data in hand, cost per workload, stranded capacity and payback stop being estimates and become a ledger.
One architectural decision, made at the start. Own the request path, then measure what it is worth. Everything on this page is downstream of that.
The value
Faster inference. Stronger governance. Higher savings.
Measured on the box we can put our hands on. Modeled where the box is yours, with the inputs on the page.
the throughput of the matched vLLM configuration at C=128, same GB10, same checkpoint, same client.
Read the ladder →about 75 MB of Rust and CUDA. No Python or PyTorch in the request path. Signed, gated and replayable.
See the posture →to pay back the license on a 256 GPU fleet at a 1.20x uplift. Every month after is upside. Edit the inputs yourself.
Open the model →Getting there
Complement or replace your serving stack, at the pace you choose.
Metrale runs beside vLLM, SGLang or llama.cpp on day one. Nothing is turned off and nothing is migrated until your own numbers say so.
Week one
Side by side on your workload
One binary per node, one signed recipe per model, next to what you run today. A side by side ladder against your current engine, on your hardware, in week one. The economics baseline starts recording the same day.
Months one to six
Traffic moves workload by workload
Production traffic shifts one model family at a time. The control plane takes over rollout, canary, rollback and fleet policy. Economics reports every cluster against what it cost before, by business unit.
At renewal
Your call, on evidence
Some customers keep the old engine for one model family. Others move off it entirely. You make that call having run both, on your own receipts rather than a vendor timeline.
Questions
The questions we actually get asked.
Short answers. Each one is backed by something on this site or in the repository.
What is Metrale?
Metrale is an inference economics platform for GPUs you own or rent by the hour. Metrale Engine runs open models faster on the same silicon, Metrale Control deploys and governs the fleet, and Metrale Economics turns the telemetry into cost per workload, chargeback and payback. The engine is open source under AGPL-3.0. The platform is licensed per GPU.
Do I have to replace vLLM, SGLang or llama.cpp to use it?
No. Metrale deploys beside your current engine and takes traffic one model family at a time. Many teams keep the old engine for a family we do not ship a recipe for yet. You decide what moves, on your own side by side numbers.
How long does a deployment take?
One binary per node and one signed recipe per model. A single box runs in minutes from one install command. A fleet pilot has a side by side ladder against your current engine in week one. Production cutover is workload by workload over the following weeks, at your pace.
Who owns the data?
You do. Prompts, weights, outputs and telemetry stay on hardware you own, in your cloud account, or on an air gapped network. The control plane manages configuration, licensing, versions and aggregate metrics, and it never sits on the request path. In a bring your own cloud deployment no inference request leaves your account.
What does verified mean on this site?
An image ships only after the serve matrix passes. Every model boots, stays coherent under greedy determinism with no token leakage and reliable tool calls, and holds throughput within ten percent of its committed baseline. A release that ships slower than its baseline fails the gate. Every number on the benchmarks page is generated from a record in the repository.
How do you measure savings?
Against your own baseline. Economics records what each cluster cost per workload before Metrale takes traffic, then reports the delta as traffic moves. The pilot ends with a receipt in dollars per million tokens and dollars per successful workload, not a slide.
Ask the rest in a working session, or read the deployment guide ↗.
Next step
Run the tests against your own fleet.
A proof of value runs four weeks against success criteria you set. Not a scripted demo on someone else’s hardware.
- Four weeks, start to finish
- A side by side ladder on your workload in week one
- Your hardware, your models, your criteria
- We supply five criteria if you do not have your own