BatchOpenAI Batch API
Batch processing
Tag every workload as interactive, same-day, or overnight before choosing a model, then route overnight jobs away from realtime endpoints.
Make the AI pricing decision without reading every provider page.
Start with a workload, then use Price Gouge playbooks and official references to choose the lowest-cost practical strategy for routing, batch, caching, subscriptions, fine-tuning, local inference, and GPU hosting.
13 Price Gouge playbooks24 official references35 pricing sources
Recommended path
Documents and batch data
Batch or cached open-model APIs for extraction, classification, and summaries.
Best route
Batch or cached open-model APIs for extraction, classification, and summaries.
Use frontier for
Schema design, evaluator samples, disputed outputs, and decision-changing edge cases.
Avoid
Sending every document through a premium long-context model before sampling quality.
Change these first
Change these cost controls before you negotiate with a provider.
CachingOpenAI prompt caching
Prompt caching
Move static system prompts, tool definitions, schemas, and examples ahead of user-specific content, then measure cache-hit rate by route.
CachingAnthropic prompt caching
Claude cache reads
Place cache breakpoints on shared instructions, long reference packs, and tool definitions instead of the per-user request body.
Read first
Useful references
Explore by format
Pick the kind of evidence you need.
Browse practical examples, technical notes, and product docs without hunting through provider pages.
Case studies
Concrete workload economics and before/after cost plans.
2 itemsCustomer stories
Outcome-focused examples for teams trying to cut AI spend.
2 itemsUse cases
Guidance by workload: dev, docs, research, support, local, and GPU.
7 itemsBlog
Short technical arguments about routing, caching, and model choice.
2 itemsResearch
Break-even models and source-backed pricing analysis.
2 itemsDocs
API, pricing engine, local inference, and fine-tuning instructions.
5 itemsProducts
The calculator, router, and provider matrix explained plainly.
3 itemsBest next reads
Start with the shortest path to a decision.
Read one workload guide, check the levers that move the bill, then use official references only when you need to verify a provider detail.
Workload guide
Budget-constrained production architecture
Best route: Hybrid router: open-model API or batch for routine work, a small frontier reserve for judgment calls, and GPU hosting only after utilization proves out. Use frontier for: Planning, architecture, evaluator samples, final reviews, low-confidence cases, and high-value decisions. Avoid: Starting with self-hosted GPUs or a frontier-only API plan before volume, quality bar, cache rate, and fallback policy are measured.
Workload guide
Software development
Best route: Subscription reserve for humans plus open-model lanes for routine agent loops. Use frontier for: Planning, architecture, difficult bugs, code reviews, and risky changes. Avoid: Paying frontier API rates for repeated scaffolding, summaries, tests, and low-risk rewrites.
Workload guide
Documents and batch data
Best route: Batch or cached open-model APIs for extraction, classification, and summaries. Use frontier for: Schema design, evaluator samples, disputed outputs, and decision-changing edge cases. Avoid: Sending every document through a premium long-context model before sampling quality.
Workload guide
Research and analysis
Best route: Cheap collection and extraction first; frontier synthesis only at the judgment layer. Use frontier for: Source reconciliation, final synthesis, methodology review, and high-stakes reasoning. Avoid: Using frontier models for every source chunk, citation scrape, or first-pass summary.
Show more guide links20 more
Workload guide
Support automation
Best route: Route simple drafts to cheap models, then escalate low-confidence or high-value tickets. Use frontier for: Policy-sensitive tickets, angry customers, refunds, legal risk, and final review samples. Avoid: Fine-tuning before retrieval, routing, cache, and eval failures are measured.
Workload guide
Local inference and fine-tuning
Best route: Benchmark local models against an open-model API baseline before renting GPUs. Use frontier for: Eval generation, regression review, fallback quality, and hard-case audits. Avoid: Buying dedicated GPU capacity before utilization and accepted-output rates are proven.
Workload guide
Self-hosted GPU production
Best route: Move steady traffic to GPU hosting only after utilization beats API pricing with ops included. Use frontier for: Fallbacks, evaluator samples, and high-risk requests that local models still miss. Avoid: Treating hourly GPU rental as the full bill while ignoring idle time, storage, egress, and operations.
Decision guideBudget cap / Routing / Break-even
Choose the cheapest practical route
Start with workload type, volume, cache shape, and latency. Then compare open-model API, batch, subscription reserve, frontier escalation, and GPU hosting.
Decision guideSoftware dev / Docs / Support
Move routine work off frontier models
Use frontier systems for planning, architecture, code review, hard bugs, research synthesis, and risky judgment. Keep extraction and drafts on cheaper lanes.
Decision guideH100 / Utilization / Ops risk
Check self-hosting before renting GPUs
GPU hosting wins only when utilization, reliability, storage, egress, and operations still beat managed API pricing on your real prompts.
Decision guideLocal models / Evals / Training cost
Decide whether fine-tuning is worth it
Fine-tune only after the prompt, routing, cache, and eval loop prove that repeated behavior is cheaper to teach than to prompt every time.
BlogFrontier only for judgment
Do not buy frontier tokens for routine inference
A practical routing pattern for keeping Codex, Claude, and other frontier models on high-value work while cheaper open-model lanes handle routine volume.
BlogSticker price is not the bill
AI pricing is moving from flat rates to hybrid bills
Why subscriptions, usage buckets, overages, cache misses, output tokens, storage, and egress make the real bill larger than the sticker price.
DocsEngine rules
How Price Gouge builds estimates
How recommendations are built from official provider prices, workload assumptions, deterministic cost math, and optional LLM help.
ResearchUtilization decides
When self-hosting beats API pricing
A provider-price break-even model comparing dedicated H100 monthly cost against frontier APIs and low-cost open-model API lanes.
ResearchCache hits change the route
The cache-heavy workload model
How prompt caching changes the economics of document processing, coding agents, support drafts, research workflows, and repeated tool use.
DocsOpen models
Local inference setup
Run open models locally or on a GPU endpoint before committing to paid frontier APIs or reserved infrastructure.
DocsFine-tuning
Fine-tuning cost guide
Decide when fine-tuning is cheaper than routing, retrieval, or prompt engineering, and account for build and run costs.
Use caseHybrid coding agents
Software development
Use subscriptions for humans and open-model APIs for routine loops; reserve frontier models for planning, architecture, reviews, and complex bugs.
Use caseBatch and cache
Data-heavy processing
Batch non-urgent work, validate cache rates, and compare open-model APIs before committing to dedicated GPUs.
Use caseSynthesis reserve
Research workflows
Use cheap collection and extraction lanes, then spend frontier tokens only on synthesis, review, and difficult reasoning.
Use caseMillion-doc routing
Document processing
Process large document volumes with open-model extraction, batch routes, cache discipline, and a small frontier evaluator sample.
Use caseAccepted draft cost
Support automation
Price routine support drafts separately from policy-sensitive escalations, human review, and frontier-model evaluation.
Use caseUtilization wins
GPU hosting strategy
Compare GPU rental, storage, egress, utilization, interruption risk, and operations overhead before self-hosting a workload.
Cost levers
Check the levers before choosing a provider.
CachingOpenAI prompt caching
Prompt caching
Keep repeated instructions, schemas, tools, examples, and policy text stable so providers can reuse the expensive prefix instead of charging every request like it is new.
OpenAI says prompt caching can reduce latency by up to 80% and input-token costs by up to 90% when repeated prefixes match.
Read guideBatchOpenAI Batch API
Batch processing
Separate work that can wait from work that needs an immediate response, then run document backfills, enrichment, evals, and nightly jobs through batch lanes.
OpenAI documents a 50% batch discount versus synchronous APIs, higher rate-limit headroom, and completion within 24 hours for eligible jobs.
Read guideCachingAnthropic prompt caching
Claude cache reads
Use Claude for review, research, and difficult reasoning without re-paying full input price for shared context that stays the same across requests.
Anthropic lists cache read tokens at 0.1x base input price, with 5-minute writes at 1.25x and 1-hour writes at 2x.
Read guideShow all cost levers4 more
RoutingVercel AI Gateway fallbacks
Gateway fallbacks
Use a gateway when reliability and routing rules matter, but keep the fallback list ordered by cost and quality so an outage does not silently pick the priciest route.
Vercel AI Gateway tries backup models in order when the primary route fails, and returns the first successful model/provider combination.
Read guideGPU hostingRunPod serverless pricing
Serverless GPU idle time
GPU hosting can beat token pricing only when worker start time, execution time, idle timeout, storage, and scale-to-zero behavior still pencil out.
RunPod serverless bills per second from worker start until full stop, and total cost includes both compute time and storage.
Read guideGPU hostingModal GPU guide
GPU fit before H100
Do not default to the biggest GPU. For many inference jobs, memory bandwidth, batch size, queueing, and model fit matter more than peak arithmetic.
Modal notes that one-prompt-at-a-time language-model runs can be memory-bound, so the fastest GPU may cost more than it saves.
Read guideForecastingFinOps AI workload cost estimation
Forecast loop
Treat every AI quote as a workload forecast: owner, use case, volume, quality bar, routing plan, cache assumption, and actual-versus-plan review cadence.
FinOps Foundation recommends granular workload-level AI cost planning so engineers can compare deployment choices and stay within budget constraints.
Read guideOfficial references
Show all official references
24 links
Official references
Show all official references
Curated docs for the levers that usually decide the bill: caching, batch, routing, provisioned throughput, GPU billing, and forecasting.
CachingOpenAI Developers
OpenAI prompt caching
How automatic prefix caching affects latency and input-token cost for repeated instructions, tools, examples, and long prompts.
BatchOpenAI Developers
OpenAI Batch API
Asynchronous job processing for work that can wait, with separate throughput and lower cost than synchronous calls.
CachingAnthropic Docs
Claude prompt caching
Automatic and explicit cache controls for repeated Claude context, including multi-turn and long-context workloads.
BatchAnthropic Docs
Claude Message Batches
Batch processing for asynchronous Claude workloads where lower cost and higher throughput matter more than instant response.
BatchGoogle AI for Developers
Gemini Batch API
Gemini batch mode guidance for non-urgent work that can trade latency for lower per-request cost.
RoutingVercel Docs
Vercel AI Gateway provider options
Controls for provider filtering, ordering, and fallback behavior when one model can be served by multiple providers.
RoutingVercel Docs
Vercel AI Gateway model fallbacks
Fallback routing behavior for moving to another model or provider when the primary route fails.
Cloud AIAWS Docs
Amazon Bedrock provisioned throughput
Fixed-cost capacity planning for Bedrock models when higher guaranteed throughput is worth commitment.
Cloud AIMicrosoft Learn
Azure provisioned throughput billing
How Microsoft Foundry provisioned throughput billing works across hourly usage and reserved capacity.
GPU hostingRunPod Docs
RunPod serverless pricing
Serverless worker billing, flex versus active workers, and where scale-to-zero changes the GPU cost model.
GPU hostingModal Docs
Modal GPU guide
How Modal represents GPU requests and hardware options for Python workloads that need bursty acceleration.
ForecastingFinOps Foundation
FinOps cost estimation of AI workloads
A practitioner guide for estimating AI workload cost from pilot through production across business, finance, and engineering.
ForecastingFinOps Foundation
FinOps for AI overview
Cost metrics for AI services, including tokens, API calls, GPU utilization, anomalies, and time to business value.
Open-model APIDeepSeek Docs
DeepSeek API pricing
Official DeepSeek model pricing with cache-hit and cache-miss prices for low-cost production inference routes.
Open-model APIFireworks AI
Fireworks AI serverless pricing
Serverless model pricing, priority tiers, fine-tuning, and dedicated deployment options for hosted open models.
Open-model APIGroq Docs
Groq on-demand pricing
Official hosted open-model pricing, rate-limit tier context, and low-latency inference options.
Open-model APITogether AI
Together AI pricing
Serverless inference, dedicated endpoints, fine-tuning, and GPU cluster pricing from Together AI.
Open-model APIMistral AI
Mistral API pricing
Official Mistral API pricing for production model choices, code models, and smaller open-weight options.
Open-model APICerebras
Cerebras inference pricing
Inference API plans, pay-per-token access, developer tiers, dedicated endpoints, and rate-limit context.
GPU hostingBaseten
Baseten cloud pricing
Dedicated deployment pricing by instance, GPU minute, training hardware, and hosted model API prices.
GPU hostingLambda
Lambda AI cloud pricing
On-demand GPU instances, clusters, reserved capacity, and private-cloud pricing context for dedicated infrastructure.
GPU hostingReplicate
Replicate billing
Billing rules for public models, private models, deployments, hardware time, failed runs, and prepaid credits.
GPU hostingVast.ai Docs
Vast.ai instance pricing
Marketplace GPU pricing mechanics, on-demand versus interruptible instances, bandwidth, and storage considerations.
GatewayOpenRouter Docs
OpenRouter model pricing
Model catalog pricing fields, provider routing context, and model-level metadata for gateway comparisons.
Provider price links
35 official provider price pages
Show provider price pages
Provider price links
35 official provider price pages
OpenAI
Verified Sep 3, 2026
Next review Sep 4, 2026
API token pricing, cached input, batch/flex prices, and Codex API model prices.
OpenAI ChatGPT
Verified Sep 3, 2026
Next review Sep 10, 2026
ChatGPT Plus and Pro subscription pricing used as human-in-the-loop reserve.
Anthropic
Verified Sep 3, 2026
Next review Sep 4, 2026
Claude API token pricing, cache write/read pricing, and dated promotion notes.
Claude Max
Verified Sep 3, 2026
Next review Sep 10, 2026
Claude Max web subscription levels used for planning and review reserve.
xAI
Verified Sep 3, 2026
Next review Sep 4, 2026
Grok Code API, Chat API token pricing, cached input, tool invocation costs, batch discounts, and priority premiums.
Google Gemini API
Verified Sep 3, 2026
Next review Sep 4, 2026
Gemini Flash standard and batch/flex token pricing.
DeepSeek
Verified Sep 3, 2026
Next review Sep 4, 2026
DeepSeek V4 Flash and Pro token pricing with cache economics.
Fireworks AI
Verified Sep 3, 2026
Next review Sep 4, 2026
GLM 5.2 Standard and Priority serverless token pricing.
Fireworks AI Fine-tuning
Verified Sep 3, 2026
Next review Sep 4, 2026
Fireworks supervised and preference fine-tuning prices per 1M training tokens.
Groq
Verified Aug 4, 2026
Next review Sep 4, 2026
GPT OSS hosted open-model token pricing and cache prices.
Together AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Throughput-oriented hosted open-model token pricing.
Mistral AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Mistral API token pricing for Medium, Small, Large, and Devstral model prices.
Baseten
Verified Sep 3, 2026
Next review Sep 4, 2026
GLM 5.2 hosted API pricing and H100 dedicated deployment cost.
AWS EC2
Verified Sep 3, 2026
Next review Sep 4, 2026
P5 Capacity Blocks H100 instance-hour and per-accelerator GPU rental pricing.
Google Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Compute Engine accelerator-optimized A3 H100 VM pricing normalized to per-GPU-hour planning prices.
Azure Virtual Machines
Verified Sep 3, 2026
Next review Sep 4, 2026
Azure ND H100 v5 retail VM pricing normalized to per-GPU-hour planning prices.
AWS Bedrock
Verified Aug 10, 2026
Next review Sep 4, 2026
Model, region, service tier, provisioned throughput, and marketplace product pricing.
Google Vertex AI
Verified Sep 3, 2026
Next review Sep 4, 2026
Vertex model garden, regions, batch, provisioned throughput, and endpoint pricing.
Azure AI Foundry Models
Verified Sep 3, 2026
Next review Sep 4, 2026
Azure hosted model products, managed GPU compute hours, and provisioned throughput.
Vercel AI Gateway
Verified Sep 3, 2026
Next review Sep 10, 2026
Gateway provider list-price pass-through, BYOK no-markup note, fallback credit risk, and provider routing.
OpenRouter
Verified Sep 3, 2026
Next review Sep 4, 2026
Machine-readable OpenRouter model catalog pricing for cheap open-model and gateway-routed options.
Modal
Verified Sep 3, 2026
Next review Sep 4, 2026
Per-second GPU, CPU, memory, storage, plan, region, and non-preemptible uplifts.
RunPod
Verified Sep 3, 2026
Next review Sep 4, 2026
Public serverless H100 rate and storage prices; clusters, reservations, and marketplace availability still need direct quote review.
CoreWeave
Verified Sep 3, 2026
Next review Sep 4, 2026
Classic GPU Cloud H100 PCIe per-GPU-hour pricing for neocloud capacity comparison.
Crusoe Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Crusoe Cloud H100 and H200 on-demand GPU-hour pricing.
Nebius AI Cloud
Verified Sep 3, 2026
Next review Sep 4, 2026
Nebius H100 NVLink on-demand and preemptible GPU-hour pricing.
TensorDock
Verified Sep 3, 2026
Next review Sep 4, 2026
TensorDock marketplace H100 SXM5 on-demand GPU-hour pricing.
DigitalOcean GPU Droplets
Verified Sep 3, 2026
Next review Sep 4, 2026
DigitalOcean GPU Droplets H100x8 pricing normalized to per-GPU-hour planning prices.
Hyperstack
Verified Sep 3, 2026
Next review Sep 4, 2026
Hyperstack H100, H100 NVLink, H100 SXM, and H200 on-demand GPU-hour pricing.
Vultr
Verified Sep 3, 2026
Next review Sep 4, 2026
Vultr Cloud GPU public plan pricing from the official plans API.
Cerebras
Verified Sep 3, 2026
Next review Sep 4, 2026
Pay-per-token inference, subscription/code plans, dedicated endpoint, and rate-limit tiers.
Lambda
Verified Sep 3, 2026
Next review Sep 4, 2026
Public H100 PCIe/SXM instance prices; clusters, reservations, taxes, and storage/network add-ons need deeper quoting.
Replicate
Verified Sep 3, 2026
Next review Sep 4, 2026
Model-specific hardware billing, training, deployments, and storage-related costs.
Replicate Billing
Verified Sep 3, 2026
Next review Sep 10, 2026
Prepaid credit, auto-reload threshold, minimum reload balance, and zero-balance behavior.
Vast.ai
Verified Sep 3, 2026
Next review Sep 4, 2026
Marketplace GPU rental rates, host reliability, storage, bandwidth, and interruption risk from the public bundle API.