How Northwind cut their LLM bill by 47% in three weeks

Northwind is a 600-person sales platform with an AI copilot embedded in every workflow. Their LLM bill in January 2026 was meaningfully larger than their entire infrastructure budget for the previous year.

Three weeks after switching to ContextAPI in front of their existing LLM, that bill dropped 47%. Here's the short version of how.

The setup

Northwind's copilot makes roughly 14 million LLM calls per month across:

• ›Email drafting
• ›Meeting summarisation
• ›Pipeline analysis
• ›Customer-facing chat

The average prompt before optimisation was 2,800 tokens. After optimisation it was 1,470. The model they're calling charges per input token at the high end of frontier pricing.

The integration

The integration took one engineer two days. The code change is small — they wrap their existing LLM client with a ContextAPI optimize call:

// before
const response = await openai.chat.completions.create({
  messages: [...],
})

// after
const optimised = await cosavu.context.optimize({
  prompt: buildPromptFromMessages(messages),
  budget: 2000,
})

const response = await openai.chat.completions.create({
  messages: rebuildMessages(optimised.optimizedPrompt),
})

They rolled it out behind a feature flag at 5% traffic, monitored downstream task accuracy on their internal eval set for 48 hours, and rolled to 100% when the numbers held.

The results

Three weeks in:

• ›Token volume to LLM: down 47.5%
• ›LLM bill: down 47% (the small gap is because our compression cost gets added on top — net savings still 35-40% depending on tier)
• ›p99 latency: unchanged
• ›Downstream accuracy: within statistical noise of baseline (we measured a 0.3% relative drop, within their eval set's natural variance)

The latency parity surprised them. Adding any layer between an app and an LLM usually adds latency. ContextAPI adds ~14ms median, which is small enough to disappear inside the LLM's own variance.

What they did with the savings

Northwind's CFO had been holding back agentic features because the projected token cost was prohibitive. With the bill cut nearly in half, they shipped a multi-step research agent that had been on the roadmap for two quarters.

The new agent makes more LLM calls than the previous workflow. Their total bill is back near where it was before — but they're now running a meaningfully more capable product on the same budget.

What we learned

A few things from working with Northwind that we've since rolled into product:

• ›Eval parity matters more than savings. The decision to roll to 100% wasn't about the cost number — it was about the accuracy number. We've since made our SDK's eval-mode output more detailed.
• ›Latency is the second concern, every time. No customer has ever said "yes" to compression that adds visible latency, no matter how big the savings. The latency budget is non-negotiable.
• ›Tier-based pricing is the right model. Northwind is on a Frontier model. Our compression cost there is a small fraction of what we save them. On a Lite model, our cost would be pennies. The adaptive rate matches the value delivered.

Northwind's letting us share this in part because they ran the rollout cleanly and they want their integration to be reproducible. If your stack looks similar — a SaaS product with high-volume LLM calls on a premium model — the same migration pattern works.