Start with a workload, then use Price Gouge playbooks and official references to choose the lowest-cost practical strategy for routing, batch, caching, subscriptions, fine-tuning, local inference, and GPU hosting.
Explore by format
Browse practical examples, technical notes, and product docs without hunting through provider pages.
Concrete workload economics and before/after cost plans.
2 itemsOutcome-focused examples for teams trying to cut AI spend.
2 itemsGuidance by workload: dev, docs, research, support, local, and GPU.
7 itemsShort technical arguments about routing, caching, and model choice.
2 itemsBreak-even models and source-backed pricing analysis.
2 itemsAPI, pricing engine, local inference, and fine-tuning instructions.
5 itemsThe calculator, router, and provider matrix explained plainly.
3 itemsPopular decisions
Each option opens a practical guide path with pricing assumptions and provider sources.
Find the cheapest production architecture that still keeps a frontier reserve.
Route bulk document work through cache, batch, and open-model lanes first.
Check when steady volume justifies rented GPUs instead of per-token APIs.
Compare prompt routing, retrieval, fine-tuning, and local inference for support work.
Best next reads
Read one workload guide, check the levers that move the bill, then use official references only when you need to verify a provider detail.
Best route: Hybrid router: open-model API or batch for routine work, a small frontier reserve for judgment calls, and GPU hosting only after utilization proves out. Use frontier for: Planning, architecture, evaluator samples, final reviews, low-confidence cases, and high-value decisions. Avoid: Starting with self-hosted GPUs or a frontier-only API plan before volume, quality bar, cache rate, and fallback policy are measured.
Best route: Subscription reserve for humans plus open-model lanes for routine agent loops. Use frontier for: Planning, architecture, difficult bugs, code reviews, and risky changes. Avoid: Paying frontier API rates for repeated scaffolding, summaries, tests, and low-risk rewrites.
Best route: Batch or cached open-model APIs for extraction, classification, and summaries. Use frontier for: Schema design, evaluator samples, disputed outputs, and decision-changing edge cases. Avoid: Sending every document through a premium long-context model before sampling quality.
Best route: Cheap collection and extraction first; frontier synthesis only at the judgment layer. Use frontier for: Source reconciliation, final synthesis, methodology review, and high-stakes reasoning. Avoid: Using frontier models for every source chunk, citation scrape, or first-pass summary.
Best route: Route simple drafts to cheap models, then escalate low-confidence or high-value tickets. Use frontier for: Policy-sensitive tickets, angry customers, refunds, legal risk, and final review samples. Avoid: Fine-tuning before retrieval, routing, cache, and eval failures are measured.
Best route: Benchmark local models against an open-model API baseline before renting GPUs. Use frontier for: Eval generation, regression review, fallback quality, and hard-case audits. Avoid: Buying dedicated GPU capacity before utilization and accepted-output rates are proven.
Best route: Move steady traffic to GPU hosting only after utilization beats API pricing with ops included. Use frontier for: Fallbacks, evaluator samples, and high-risk requests that local models still miss. Avoid: Treating hourly GPU rental as the full bill while ignoring idle time, storage, egress, and operations.
Start with workload type, volume, cache shape, and latency. Then compare open-model API, batch, subscription reserve, frontier escalation, and GPU hosting.
Use frontier systems for planning, architecture, code review, hard bugs, research synthesis, and risky judgment. Keep extraction and drafts on cheaper lanes.
GPU hosting wins only when utilization, reliability, storage, egress, and operations still beat managed API pricing on your real prompts.
Fine-tune only after the prompt, routing, cache, and eval loop prove that repeated behavior is cheaper to teach than to prompt every time.
A practical routing pattern for keeping Codex, Claude, and other frontier models on high-value work while cheaper open-model lanes handle routine volume.
Why subscriptions, usage buckets, overages, cache misses, output tokens, storage, and egress make the real bill larger than the sticker price.
How recommendations are built from official provider prices, workload assumptions, deterministic cost math, and optional LLM help.
A provider-price break-even model comparing dedicated H100 monthly cost against frontier APIs and low-cost open-model API lanes.
How prompt caching changes the economics of document processing, coding agents, support drafts, research workflows, and repeated tool use.
Run open models locally or on a GPU endpoint before committing to paid frontier APIs or reserved infrastructure.
Decide when fine-tuning is cheaper than routing, retrieval, or prompt engineering, and account for build and run costs.
Use subscriptions for humans and open-model APIs for routine loops; reserve frontier models for planning, architecture, reviews, and complex bugs.
Batch non-urgent work, validate cache rates, and compare open-model APIs before committing to dedicated GPUs.
Use cheap collection and extraction lanes, then spend frontier tokens only on synthesis, review, and difficult reasoning.
Process large document volumes with open-model extraction, batch routes, cache discipline, and a small frontier evaluator sample.
Price routine support drafts separately from policy-sensitive escalations, human review, and frontier-model evaluation.
Compare GPU rental, storage, egress, utilization, interruption risk, and operations overhead before self-hosting a workload.
Cost levers
Keep repeated instructions, schemas, tools, examples, and policy text stable so providers can reuse the expensive prefix instead of charging every request like it is new.
OpenAI says prompt caching can reduce latency by up to 80% and input-token costs by up to 90% when repeated prefixes match.
Read guideSeparate work that can wait from work that needs an immediate response, then run document backfills, enrichment, evals, and nightly jobs through batch lanes.
OpenAI documents a 50% batch discount versus synchronous APIs, higher rate-limit headroom, and completion within 24 hours for eligible jobs.
Read guideUse Claude for review, research, and difficult reasoning without re-paying full input price for shared context that stays the same across requests.
Anthropic lists cache read tokens at 0.1x base input price, with 5-minute writes at 1.25x and 1-hour writes at 2x.
Read guideUse a gateway when reliability and routing rules matter, but keep the fallback list ordered by cost and quality so an outage does not silently pick the priciest route.
Vercel AI Gateway tries backup models in order when the primary route fails, and returns the first successful model/provider combination.
Read guideGPU hosting can beat token pricing only when worker start time, execution time, idle timeout, storage, and scale-to-zero behavior still pencil out.
RunPod serverless bills per second from worker start until full stop, and total cost includes both compute time and storage.
Read guideDo not default to the biggest GPU. For many inference jobs, memory bandwidth, batch size, queueing, and model fit matter more than peak arithmetic.
Modal notes that one-prompt-at-a-time language-model runs can be memory-bound, so the fastest GPU may cost more than it saves.
Read guideTreat every AI quote as a workload forecast: owner, use case, volume, quality bar, routing plan, cache assumption, and actual-versus-plan review cadence.
FinOps Foundation recommends granular workload-level AI cost planning so engineers can compare deployment choices and stay within budget constraints.
Read guideOfficial references
Curated docs for the levers that usually decide the bill: caching, batch, routing, provisioned throughput, GPU billing, and forecasting.
How automatic prefix caching affects latency and input-token cost for repeated instructions, tools, examples, and long prompts.
Asynchronous job processing for work that can wait, with separate throughput and lower cost than synchronous calls.
Automatic and explicit cache controls for repeated Claude context, including multi-turn and long-context workloads.
Batch processing for asynchronous Claude workloads where lower cost and higher throughput matter more than instant response.
Gemini batch mode guidance for non-urgent work that can trade latency for lower per-request cost.
Controls for provider filtering, ordering, and fallback behavior when one model can be served by multiple providers.
Fallback routing behavior for moving to another model or provider when the primary route fails.
Fixed-cost capacity planning for Bedrock models when higher guaranteed throughput is worth commitment.
How Microsoft Foundry provisioned throughput billing works across hourly usage and reserved capacity.
Serverless worker billing, flex versus active workers, and where scale-to-zero changes the GPU cost model.
How Modal represents GPU requests and hardware options for Python workloads that need bursty acceleration.
A practitioner guide for estimating AI workload cost from pilot through production across business, finance, and engineering.
Cost metrics for AI services, including tokens, API calls, GPU utilization, anomalies, and time to business value.
Official DeepSeek model pricing with cache-hit and cache-miss prices for low-cost production inference routes.
Serverless model pricing, priority tiers, fine-tuning, and dedicated deployment options for hosted open models.
Official hosted open-model pricing, rate-limit tier context, and low-latency inference options.
Serverless inference, dedicated endpoints, fine-tuning, and GPU cluster pricing from Together AI.
Official Mistral API pricing for production model choices, code models, and smaller open-weight options.
Inference API plans, pay-per-token access, developer tiers, dedicated endpoints, and rate-limit context.
Dedicated deployment pricing by instance, GPU minute, training hardware, and hosted model API prices.
On-demand GPU instances, clusters, reserved capacity, and private-cloud pricing context for dedicated infrastructure.
Billing rules for public models, private models, deployments, hardware time, failed runs, and prepaid credits.
Marketplace GPU pricing mechanics, on-demand versus interruptible instances, bandwidth, and storage considerations.
Model catalog pricing fields, provider routing context, and model-level metadata for gateway comparisons.
Provider price links
OpenAI
Verified Sep 3, 2026
Next review Sep 4, 2026
API token pricing, cached input, batch/flex prices, and Codex API model prices.
OpenAI ChatGPT
Verified Sep 3, 2026
Next review Sep 10, 2026
ChatGPT Plus and Pro subscription pricing used as human-in-the-loop reserve.
Anthropic
Verified Sep 3, 2026
Next review Sep 4, 2026
Claude API token pricing, cache write/read pricing, and dated promotion notes.
Claude Max
Verified Sep 3, 2026
Next review Sep 10, 2026
Claude Max web subscription levels used for planning and review reserve.
xAI
Verified Sep 3, 2026
Next review Sep 4, 2026
Grok Code API, Chat API token pricing, cached input, tool invocation costs, batch discounts, and priority premiums.
Google Gemini API
Verified Sep 3, 2026
Next review Sep 4, 2026
Gemini Flash standard and batch/flex token pricing.
DeepSeek
Verified Sep 3, 2026
Next review Sep 4, 2026
DeepSeek V4 Flash and Pro token pricing with cache economics.
Fireworks AI
Verified Sep 3, 2026
Next review Sep 4, 2026
GLM 5.2 Standard and Priority serverless token pricing.
Fireworks AI Fine-tuning
Verified Sep 3, 2026
Next review Sep 4, 2026
Fireworks supervised and preference fine-tuning prices per 1M training tokens.
Groq
Verified Aug 4, 2026
Next review Sep 4, 2026
GPT OSS hosted open-model token pricing and cache prices.
Together AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Throughput-oriented hosted open-model token pricing.
Mistral AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Mistral API token pricing for Medium, Small, Large, and Devstral model prices.
Baseten
Verified Sep 3, 2026
Next review Sep 4, 2026
GLM 5.2 hosted API pricing and H100 dedicated deployment cost.
AWS EC2
Verified Sep 3, 2026
Next review Sep 4, 2026
P5 Capacity Blocks H100 instance-hour and per-accelerator GPU rental pricing.
Google Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Compute Engine accelerator-optimized A3 H100 VM pricing normalized to per-GPU-hour planning prices.
Azure Virtual Machines
Verified Sep 3, 2026
Next review Sep 4, 2026
Azure ND H100 v5 retail VM pricing normalized to per-GPU-hour planning prices.
AWS Bedrock
Verified Aug 10, 2026
Next review Sep 4, 2026
Model, region, service tier, provisioned throughput, and marketplace product pricing.
Google Vertex AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Vertex model garden, regions, batch, provisioned throughput, and endpoint pricing.
Azure AI Foundry Models
Verified Sep 3, 2026
Next review Sep 4, 2026
Azure hosted model products, managed GPU compute hours, and provisioned throughput.
Vercel AI Gateway
Verified Sep 3, 2026
Next review Sep 10, 2026
Gateway provider list-price pass-through, BYOK no-markup note, fallback credit risk, and provider routing.
OpenRouter
Verified Sep 3, 2026
Next review Sep 4, 2026
Machine-readable OpenRouter model catalog pricing for cheap open-model and gateway-routed options.
Modal
Verified Sep 3, 2026
Next review Sep 4, 2026
Per-second GPU, CPU, memory, storage, plan, region, and non-preemptible uplifts.
RunPod
Verified Sep 3, 2026
Next review Sep 4, 2026
Public serverless H100 rate and storage prices; clusters, reservations, and marketplace availability still need direct quote review.
CoreWeave
Verified Sep 3, 2026
Next review Sep 4, 2026
Classic GPU Cloud H100 PCIe per-GPU-hour pricing for neocloud capacity comparison.
Crusoe Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Crusoe Cloud H100 and H200 on-demand GPU-hour pricing.
Nebius AI Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Nebius H100 NVLink on-demand and preemptible GPU-hour pricing.
TensorDock
Verified Sep 3, 2026
Next review Sep 4, 2026
TensorDock marketplace H100 SXM5 on-demand GPU-hour pricing.
DigitalOcean GPU Droplets
Verified Sep 3, 2026
Next review Sep 4, 2026
DigitalOcean GPU Droplets H100x8 pricing normalized to per-GPU-hour planning prices.
Hyperstack
Verified Sep 3, 2026
Next review Sep 4, 2026
Hyperstack H100, H100 NVLink, H100 SXM, and H200 on-demand GPU-hour pricing.
Vultr
Verified Sep 3, 2026
Next review Sep 4, 2026
Vultr Cloud GPU public plan pricing from the official plans API.
Cerebras
Verified Sep 3, 2026
Next review Sep 4, 2026
Pay-per-token inference, subscription/code plans, dedicated endpoint, and rate-limit tiers.
Lambda
Verified Sep 3, 2026
Next review Sep 4, 2026
Public H100 PCIe/SXM instance prices; clusters, reservations, taxes, and storage/network add-ons need deeper quoting.
Replicate
Verified Sep 3, 2026
Next review Sep 4, 2026
Model-specific hardware billing, training, deployments, and storage-related costs.
Replicate Billing
Verified Sep 3, 2026
Next review Sep 10, 2026
Prepaid credit, auto-reload threshold, minimum reload balance, and zero-balance behavior.
Vast.ai
Verified Sep 3, 2026
Next review Sep 4, 2026
Marketplace GPU rental rates, host reliability, storage, bandwidth, and interruption risk from the public bundle API.