There's a pattern I see constantly with founders building AI features.
They integrate OpenAI, pick GPT-4 or Claude Opus because those are the "good" models, and then route every single API call through the most expensive tier available.
Six weeks later they're staring at a $3,000 invoice for a product with 200 users.
The product works fine. The spend doesn't.
Here's the thing: most of those calls didn't need the expensive model. A task that costs $0.08 on GPT-4 costs $0.002 on GPT-3.5. For the right task, the cheaper model gets you 95% of the quality at 4% of the price.
The problem isn't that they chose the wrong model. It's that they chose one model for everything.
Why Founders Default to the Expensive Tier
It's not laziness. It's risk aversion.
When you're building fast, you grab the most capable model because you don't want a dumb AI moment to tank your demo or your user's experience. You ship. The quality is good. You move on.
What you don't see is the cost breakdown under the hood. Every summary, every label, every classification task, every "does this content violate our rules?" check is getting billed at premium rates. Small tasks, big model, all the time.
This compounds fast. A product that makes 1,000 AI calls per day at $0.05 average costs $1,500/month. The same product with basic routing might run at $300.
That's $1,200 in margin you're leaving on the table every single month.
The Three-Tier Mental Model
Before you change anything in your code, it helps to think about AI tasks in three buckets.
Tier 1: Reasoning Tasks These actually need the expensive model. Multi-step reasoning, complex code generation, nuanced writing, ambiguous instructions with a lot of context. GPT-4, Claude Opus, Gemini Ultra territory. Worth the price.
Tier 2: Instruction-Following Tasks Clear instructions, defined output format, moderate context window. Summaries, rewrites, structured extraction, moderate code edits. GPT-3.5-turbo, Claude Haiku, Gemini Flash handle these well. A fraction of the cost.
Tier 3: Classification and Routing Tasks Is this spam? What category does this belong to? Does this violate a rule? Is this positive or negative? These tasks are so simple that even a small model gets 99% accuracy. Sometimes a regex is better than a model at all.
Most AI features in most SaaS products are split roughly 10-20% Tier 1, 40-50% Tier 2, 30-40% Tier 3. But the spend is allocated like everything is Tier 1.
What This Looks Like in Practice
Here's a real example from a product I worked on: a B2B tool that used AI to analyze customer support tickets.
The flow was:
- Incoming ticket arrives
- AI classifies urgency (high/medium/low)
- AI extracts key entities (product, error type, customer tier)
- AI drafts a suggested reply
- AI writes a brief internal summary for the support team
All five steps were running through GPT-4. Steps 2 and 3 are pure classification. Step 4 is instruction-following with a defined template. Only step 5 genuinely benefited from the full reasoning capability.
After routing correctly: Steps 2 and 3 moved to GPT-3.5. Step 4 moved to GPT-3.5 with a tighter prompt. Step 5 stayed on GPT-4.
Cost dropped 68%. Quality was indistinguishable to end users.
How to Audit Your Own AI Calls
You can't fix what you can't see. Start with visibility.
Pull your API usage logs and categorize by call type, not just volume. Most founders look at total spend. You need to look at spend by task type.
If you're using OpenAI, the usage dashboard breaks down by model. But it doesn't tell you what each model is doing. You need to add that context yourself.
The simplest approach: add a task_type field to every API call log. Even just noting it in your application logs is enough. Within a week you'll see where your budget is actually going.
Common surprises:
- Realtime validation calls (checking if user input is appropriate) running at premium rates
- Repeated similar prompts that could be cached
- Classification tasks using large context windows they don't need
The Routing Decision Framework
When you're deciding which model to use for a task, ask three questions:
1. Does this task require multi-step reasoning? If the model needs to hold multiple pieces of logic in tension and work through them to reach a conclusion, use Tier 1. If it's following a clear pattern, don't.
2. What's the cost of getting it wrong? A wrong urgency classification in a support ticket is annoying. A wrong summary in a legal document is a liability. Match model quality to failure cost.
3. What's the input/output structure? Free-form input with free-form output: usually Tier 1. Structured input with structured output: usually Tier 2 or 3. Classification with a fixed set of labels: almost always Tier 3.
Caching Is the Other Half of the Problem
While you're in the audit mindset, check for repeated calls.
Every call where the input is the same or nearly the same is a caching opportunity. System prompts, few-shot examples, static reference content, the same FAQ summary you're generating for every new user session; all of these can be cached at the application layer.
Semantic caching is worth looking into for products with high call volume. If 30% of your users are asking variations of the same five questions, you're paying to generate the same answer hundreds of times a day.
What This Actually Means for Your Unit Economics
This isn't just about reducing your cloud bill. It's about whether your product can survive at scale.
If your AI costs are $0.15 per active user per day and you charge $49/month, you have about 10 days before you're underwater on that user. That sounds extreme, but I've seen it happen with products that didn't audit their AI spend until they were already past 1,000 users.
The founders who get this right early are the ones who think about AI costs the same way they think about hosting costs. It's infrastructure. It scales with usage. You need to know your unit economics before you start pushing for growth.
Model routing is how you keep those unit economics in check. It's not premature optimization. It's the difference between a product that works at 100 users and one that works at 10,000.
Where to Start This Week
If you've never audited your AI call types, here's the quickest path to real numbers:
- Pull one week of API logs and tally call count by the task each call serves
- Categorize each task type as reasoning, instruction-following, or classification
- For any classification task running on your most expensive model, test the same prompts on your cheapest model
- If accuracy holds above 95%, route that task down
Most products can cut 40-60% of AI spend with two or three routing changes. The quality tradeoff is negligible. The margin improvement is immediate.
If you're not sure where to start or your AI architecture has grown faster than your cost controls, that's worth a conversation. The technical debt in AI spend tends to compound quietly until it suddenly becomes a real problem.
Book a call and we can look at your current setup together.