Five Levers That Cut AI Running Costs Without Cutting Quality

The security agent that was paying for the same words thousands of times

ProjectDiscovery builds Neo, an autonomous platform that tests software for security weaknesses. A single Neo task — a vulnerability assessment, a code review, a security audit — routinely runs through 20 to 40 or more AI steps, each one planning, probing, reading results and deciding what to do next. Every step sends the model a long set of standing instructions, around 20,000 tokens, before any new information.

The company wanted what every serious AI product wants: thorough results at a cost that makes sense. The external problem was arithmetic. Dozens of steps per task, each carrying a large block of text, multiplied across many tasks, produced a very large bill. The internal problem was sharper. In security testing, thoroughness is the product. Cutting steps or instructions to save money would mean missing vulnerabilities — the one thing the platform exists to find. And underneath sat a principle worth defending: doing the job properly shouldn't be priced out.

The answer was already sitting in the model providers' pricing. Many providers now offer prompt caching, which lets a model reuse text it has recently processed and charges far less for it. But Neo's cache was barely working. According to the company's own engineering write-up, only about 7% of its input was being served from cache, because a section of the standing instructions changed as each task progressed. Change one part of the opening text and everything after it has to be processed again at full price.

The fix was structural, not technical wizardry. The team moved the changing working memory out of the fixed instructions and to the end of each request, leaving the long opening identical from step to step. The cache rate rose to 84%. ProjectDiscovery reported that caching cut its overall model costs by 59%, reaching 70% in the final days of measurement. In one striking comparison, two tasks with almost identical token volumes differed in cost by roughly 60 times, purely because one ran before the change and one after.

Nothing about the security testing changed. Neo ran the same steps, read the same instructions and reached the same conclusions. The only difference was that it stopped paying full price for words it had already read.

That is the standard for every lever in this article. Reducing AI running cost by lowering quality is easy and pointless. The five levers below reduce cost while keeping the output the same, or measurably as good.

Why quality is the constraint, not the casualty

Most cost-cutting advice treats quality as something to trade away. In AI features, that trade is usually false economy. A cheaper answer that is wrong creates a correction, an escalation or a repeat request, and each of those costs more than the tokens saved. The right question is not how to spend less per request, but how to spend less per successful outcome.

That changes the order in which to pull the levers. Start with the ones that cannot affect quality at all, such as caching and batching. Move next to the ones that affect quality only if done carelessly, such as trimming context and limiting output. Change models last, and only after testing.

Lever 1: Stop paying twice for the same words

Caching comes in two forms, and both are worth understanding.

Prompt caching

Prompt caching reuses the unchanging opening of a request — standing instructions, reference material, examples — so the model does not reprocess it every time. Several major providers now charge a small fraction of the normal input price for cached text, in some cases around a tenth. Output is unaffected because the model sees exactly the same input. The design rule is the one ProjectDiscovery learned: keep the stable material at the start and put anything that changes at the end.

Answer caching

Answer caching goes a step further. If the same question, or one that means the same thing, has already been answered, the stored answer is returned without calling a model at all. In customer and internal assistants, a large share of questions repeat.

One caution matters especially in regulated businesses. A cached answer is only as current as the documents behind it. When an SOP, policy or product specification changes, every cached answer based on it must be cleared. Design that link from the start.

Lever 2: Send less

Much of what is sent to a model with each request is unnecessary. Retrieval systems often pass along far more of your documents than a question needs. Conversations carry their entire history, turn after turn, so the input grows with every exchange. Standing instructions accumulate paragraphs nobody has reviewed in months.

Three habits address this without harming answers. Retrieve fewer, better passages: ranking results well and sending the three most relevant usually beats sending twenty. Summarise long conversations rather than resending them in full. And review standing instructions periodically, removing anything that no longer changes the model's behaviour. Each should be tested against real questions before and after, but in most systems the answers improve, because the model is no longer wading through irrelevant material.

Lever 3: Ask for less

Output is typically the more expensive side of an AI bill, often priced several times higher per token than input. Yet many features let the model write as much as it likes.

Set sensible length limits for each task. Ask for structured output — a defined set of fields rather than free prose — where the result feeds another system. And control reasoning. Many current models can be told how much to "think" before answering, and that hidden reasoning is billed as output. Deep reasoning is worth paying for on a difficult analysis. It is waste on extracting a date from an invoice.

How AxON approaches this

These levers are part of how AxON, our AI delivery platform, is built, not an optimisation pass added after launch. Prompts are structured so stable material can be cached, retrieval is limited to what a task needs, output length and reasoning depth are set per task, and work that can wait is batched. AxON is model-agnostic, so each lever is applied using whatever the chosen provider offers rather than assuming one vendor's features. Every change is tested against a quality baseline before it goes live, because the goal is lower cost per successful outcome, not cheaper wrong answers.

Lever 4: Don't pay for urgency you don't need

Not every AI task needs an answer in two seconds. Classifying last month's documents, summarising a backlog of reports, translating a product catalogue or tagging a data archive can all wait hours without anyone noticing.

Major providers offer batch processing for exactly this kind of work: you submit many requests together and accept results within a set window, commonly up to a day, in exchange for a substantial discount — often around half the normal price. The model and its output are the same. Only the timing changes. On some providers, batch and caching discounts can be combined.

The practical step is simply to sort a feature's tasks into those a person is waiting for and those nobody is. The second group almost always belongs in a batch.

Lever 5: Use the right-sized model — last, and with evidence

Sending simple tasks to smaller, cheaper models is one of the largest savings available, and it is covered in depth in our guide to model routing. It comes last in this list for a reason. Unlike the first four levers, changing the model can change the answer. It should only be done after testing on your own tasks, with a quality bar agreed in advance, and with monitoring once live.

Which levers matter most for your workload

The levers do not carry equal weight everywhere. The table below is a starting point for deciding where to look first.

The guardrail: measure before and after

Every lever should be judged on two numbers together: cost per successful outcome, and quality. Build a set of real, representative requests with known good answers before changing anything. Run it after each change. If quality holds and cost falls, keep the change. If quality slips, the saving was an illusion.

Industry guides regularly claim savings of 60–90% from stacking these techniques. Some workloads achieve that. Many will not, because the achievable saving depends on how repetitive your requests are, how much can wait, and how much unnecessary text you were sending to begin with. Measure your own.

Where to start this week

If you already run AI features, you don't need a project to begin. Five small steps usually reveal most of the opportunity within a fortnight.

1. Log tokens and cost per feature, not just a monthly total, so you can see where the money actually goes.
2. Check your cache hit rate. If it is low, look for text that changes near the start of your prompts.
3. List every task nobody is waiting for in real time. Those are your batch candidates.
4. Set explicit output limits and reasoning settings for each task instead of accepting defaults.
5. Assemble fifty real requests with known good answers. That test set is what lets you change anything else safely.

A note on providers

These levers apply across the market, but the details differ. OpenAI, Anthropic, Google, Mistral and others each price caching, batching and reasoning controls differently, and the terms change often. Open-weight models such as Meta's Llama, DeepSeek, Alibaba's Qwen and the UAE's Falcon, when hosted on your own infrastructure, benefit from the same ideas in a different form: reusing processed text and grouping requests raise the use you get from the hardware you are already paying for. Design for the principle, then apply each provider's specific mechanism.

Questions to ask whoever builds it

1. Which of these five levers does the design use, and which were considered and rejected?
2. How is the prompt structured so that caching actually works?
3. Which tasks are batched, and how are cached answers refreshed when our documents change?
4. What quality test set do you run before and after a cost change?

The bottom line

AI running costs can usually be cut substantially without touching quality. Stop paying twice for the same words, send less, ask for less, batch what can wait, and right-size models last and with evidence. Measure cost per successful outcome, and test quality before and after every change. ProjectDiscovery's agent did exactly the same work after its fix; it simply stopped paying full price for it.

CTA: If you'd like to know which of these levers would matter most for an AI feature you run or are planning, we'll review the design and show the likely saving and its effect on quality in writing.

Frequently asked questions

What is prompt caching?

A feature offered by many AI providers that reuses the unchanging opening of a request, such as standing instructions, and charges much less for it. It does not change the model's output.

How much can batch processing save?

Many providers offer a large discount, often around half the normal price, for requests that can be completed within a set window instead of immediately.

Will cutting AI costs reduce answer quality?

It doesn't have to. Caching and batching don't change outputs. Trimming context, limiting output and changing models should be tested against a quality baseline first.