Why Is There No Spot Market for Inference, When So Much GPU Capacity Sits Idle

What’s the Problem

LLM inference is already priced by the token: so much per million input tokens, so much per million output tokens. But that price is a fixed list rate set by each provider. OpenAI quotes one number, Amazon Bedrock quotes another, and nothing connects them at a live clearing price.

The waste piles up on both sides. On one side there is idle GPU capacity. Self-hosted boxes, rented instances, and cards sitting free after a training run all throw away spare inference every minute. On the other side there are buyers running large batch jobs. Work like document classification, data extraction, and summarization is latency-tolerant and high-volume, and paying full list price makes the unit cost heavy. There clearly exist sellers who would offload spare capacity cheaply and buyers who would pay less, yet no spot market prices the two together. GPU hours have a spot market, but the actual output of inference, the tokens, does not.

Why Now

Three things lined up at once. First, token pricing is already a standard unit. A per-million-token rate is the common yardstick, so you can compare different providers on the same scale. Second, open-weight models mean the same model runs identically across many providers. That creates a fungible commodity: not a brand, but “output tokens of this model.” Third, agents can now shop on their own. In fact, mtok.market shipped this shape first. It is non-custodial and seller-hosted, settles in USDC on the Base chain per chunk, shows a median derived from actually delivered trades as the spot price, and plugs into agents over MCP.

The macro picture points the same way. Venues like the Shanghai Futures Exchange, CME, and ICE are designing GPU compute futures, and there is open talk of trading tokens like gold or oil (TechCrunch, arXiv). Where futures get discussed, a spot market usually stands underneath first.

flowchart LR
  A[Idle inference capacity<br/>spare GPUs, self-hosted] --> B[Spot order book<br/>per-model price]
  C[Batch inference demand<br/>classify, extract, summarize] --> B
  B --> D[Verification and settlement<br/>non-custodial]
  D --> E[Cheaper than list<br/>idle capacity monetized]

How You Could Build It

The key is to start with one narrow, fungible good. For example, the output tokens of a specific open model, and within that only asynchronous batch jobs that tolerate latency. Sellers post spare capacity at a live price, buyers pick the cheapest lot on the order book. Keep settlement non-custodial so the platform never holds funds or keys. Lowering payment friction is what unlocks early liquidity.

The hardest piece is verification. You need to confirm that a seller actually ran the stated model and did not quietly swap in a cheaper one. Add a trust layer by re-running samples to cross-check outputs, or by requiring reputation scores and posted collateral. Revenue is a thin fee on matched volume. Do not chase real-time conversational inference at the start; go after price-sensitive, latency-tolerant batch work first.

What Makes It Work

Two assumptions have to hold. First, buyers must accept “any seller running this model” rather than a specific brand. That holds for batch and non-real-time work, but not for production traffic where latency and reliability are everything. Second, you must be able to prove cheaply that delivered tokens are genuine. If verification costs more than the savings, the market never forms. Clear both and a liquidity flywheel spins: more sellers make it cheaper than list, cheaper draws buyers, and buyers pull in more sellers. Whoever turns that wheel first owns the reference price for inference.