Verticals / Vertical guide

What breaks in an AI wrapper app before it scales?

Matt Turley, Continuum. Updated November 5, 2026.

For founders whose product is a thin or thick layer over a model API (chat, summarisation, agents, generation), usually built with AI coding tools.

The short answer

What breaks in an AI wrapper app before it scales?

Cost and trust. An app that calls an AI model for each user turns every request into money. The usual failures are the provider key reachable from the browser, no per-user limit or spend cap, user data stored in prompts and logs, untrusted text treated as instructions, and long responses that time out.

What is really going on

A normal web request costs a fraction of a cent. A model request can cost orders of magnitude more, and the cost varies with input length, so a bug, a bot or a single heavy user is a billing event. Wrapper apps are the category most likely to receive a surprise invoice.

There is also a trust problem that older apps do not have. The model follows instructions in whatever text it is given, including text from a web page, a document or another user. When your app feeds that text to a model that can take actions, the author of the text can steer them.

And the data is often the point. Users paste in contracts, code, customer lists and medical notes. The app's logs, its prompt history and the provider's retention settings become places that data lives.

Where wrapper apps go wrong, most common first

  1. The provider key is reachable from the browser

    How to tell Search the built JavaScript for your key prefix, and inspect network calls. If the browser calls the model provider directly, the key is public. Variables with a public prefix ship to every visitor.

    The fix Call the model only from server code, rotate the exposed key at the provider and put your own authenticated route in front of it.

  2. No per-user limit and no spend cap

    How to tell One account can send unlimited requests, anonymous users can call the endpoint and the provider dashboard has no budget alert.

    The fix Require login, add per-user quotas and rate limits, cap the input and output length and set spend limits and alerts at the provider.

  3. Untrusted text is treated as instructions

    How to tell Your app summarises web pages, emails or uploaded documents, and a hidden line in one of them can change what the model does, such as revealing other content or calling a tool.

    The fix Treat all fetched and uploaded text as untrusted. Give the model the minimum tools and data, require confirmation for consequential actions and never let model output run as code or render as raw HTML.

  4. User data is stored and sent more widely than you think

    How to tell Prompts and responses are saved in your database, in logs and in analytics. The provider retains requests under its default terms. Users never agreed to that, and your privacy policy does not say it.

    The fix Decide what you store and for how long, check the provider's retention and training terms, minimise what goes in prompts and make the privacy policy match reality.

  5. Long responses meet serverless time limits

    How to tell Responses stop partway, users see a timeout or a blank screen and the failures grow with the length of the output. A function that waits for a whole generation can exceed its time limit.

    The fix Stream the response, raise the function's duration where the plan allows or move long jobs to a background process with a status the user can check.

  6. Provider errors and limits are not handled

    How to tell The provider returns 429 or 5xx and the app shows a crash or silently returns nothing. Model names are deprecated and calls begin to fail on a date.

    The fix Retry with backoff, show a clear message, fall back where it makes sense and track the provider's deprecation notices for the model you use.

How we diagnose it

  1. Find where the key lives. Search the repository, the build output and the network calls for the provider key and confirm every model call is made from the server.
  2. Run the worst case through the bill. Work out the cost of the longest allowed input and output, multiplied by the heaviest plausible user and by a bot. Compare with your caps.
  3. Attack with text. Feed the app documents and pages containing instructions aimed at the model and see what it reveals or does. Check what tools and data it can reach.
  4. Trace the data. Follow a user's input into prompts, logs, analytics and the provider, and write down each place it is kept and for how long.
  5. Time the slow path. Measure the longest realistic request on the deployed function and check what happens at the time limit and under provider errors.

What fixing it changes

  • The provider key is server-side only, and any exposed key is rotated.
  • Per-user limits, input and output caps, spend limits and alerts exist.
  • Untrusted text cannot trigger consequential actions, and output is never executed.
  • The data kept and sent is minimised and matches the privacy policy.
  • Responses stream or run in the background, and provider errors are handled.

When you do not need us

If the app is an internal tool used by a handful of people you know, most of the abuse and cost controls matter less. A key kept server-side and a provider spend cap still take ten minutes and are worth doing.

Your next step

The free Leak Check looks for secrets exposed in the client bundle, which is the first thing to rule out. A Ship Check reads the server code for limits, data handling and untrusted-input risks.

free Leak Check: Paste your live URL. It runs seven surface checks (reachability, SSL, auth flow, payment path, exposed secrets in the client bundle, Supabase exposure, broken links and performance) and returns a green, yellow or red for each. It is a surface scan, not a full security audit. Or: Get a Ship Check, $299.

Run the free Leak Check →See all services and prices →
FAQ

Straight answers

Is my OpenAI or Anthropic key safe if it is in an environment variable?
Only if the variable is used in server code and does not carry a public prefix. Environment variables with a public prefix are bundled into the browser JavaScript and visible to every visitor. Check the built output, not just the settings.
What is prompt injection?
It is when text the model reads, such as a web page or document, contains instructions that the model follows as if the developer wrote them. The risk grows with what the model can do: reading private data or calling tools turns a prank into a breach.
How do I cap what I spend?
Use the provider's budget and rate limit settings as a backstop, and enforce your own per-user quotas in the app. Alert on unusual usage so you find a runaway loop in hours rather than at the end of the month.
Keep reading
My app broke right after launch. What do I do first?By stage. The first hours after a launch goes wrong: stop the damage, find the failure, protect users and data, then make it stable. What usually breaks, in order.What should I check before launching my AI-built app?By stage. A launch-week checklist for apps built with AI tools: who can see whose data, secrets, the real payment path, email, backups, monitoring and a rollback plan.Why does my app work locally but break in production?By symptom. Production differs from your laptop in environment variables, database, file system, Node version and network rules. The usual causes and how to check each.What breaks in an AI-built health app before launch?By vertical. Health apps built with AI tools: whether HIPAA applies, vendors that need a BAA, patient data in logs and AI prompts, and access control. Not legal advice.What breaks in an AI-built app after Demo Day?By stage. After demo day or investor attention: a signup spike, technical questions and an app built to be demoed. What breaks first and what to do this week.What breaks in an AI-built B2B SaaS app before launch?By vertical. For B2B SaaS built with AI tools: tenant isolation, roles, invites, billing and the enterprise asks that arrive first. What breaks and how to check.
Sources
  1. 1. OWASP: Top 10 for Large Language Model Applications, OWASP: Prompt injection, insecure output handling and denial-of-wallet risks.
  2. 2. Anthropic: API rate limits, Anthropic: How provider rate limits and spend limits work, an example of the limits to design for.
  3. 3. Vercel: Function duration, Vercel: How long a serverless function may run, and why long generations need streaming or background work.
What breaks in an AI wrapper app before it scales?