CosavuCompilers don't optimise source code directly. They lift it into an intermediate representation, transform it there, and lower it back. The IR is where the interesting work happens — it's typed, structured, and easier to reason about than the surface syntax.
We took that idea seriously for prompts.
The problem with treating prompts as strings
Every prompt-optimisation tool we benchmarked treated prompts as opaque strings. "Remove filler words." "Truncate to N tokens." "Run an LLM-as-compressor pass." These are string-level operations. They work, but they're brittle.
A persona instruction (You are an expert tax accountant) and a retrieved-context block (According to IRS Publication 17...) deserve completely different compression strategies. Treating them the same is the source of half the failure modes we see in production.
The PromptIR block types
PromptIR splits an incoming prompt into one or more typed blocks. Each block has a type, a token count, and a set of constraints that determine how aggressively it can be compressed.
- ›IDENTITY. Persona or role assignment. Compressed lightly — these are short, semantically dense, and changing them shifts model behaviour.
- ›INSTRUCTION. What the model should do. Heavy rewriting allowed. We can rephrase, drop hedges, switch passive to active voice.
- ›CONTEXT. Retrieved documents or background information. Aggressive truncation allowed. We rank sentences by relevance and drop tail.
- ›CONSTRAINT. Output format, length, behavioural rules. Almost never touched. These are load-bearing.
- ›EXAMPLE. Few-shot demonstrations. Compressed by deduplicating across examples — if three examples all start with the same template, we keep one.
- ›OUTPUT_FORMAT. Schema or template specification. Never compressed. Touching these breaks downstream parsers.
The parser
The parser is a rule-based lexer with heuristic fallbacks. We tried training a neural parser. It was slower and less accurate than well-written rules.
The rules detect blocks via three signals:
- ›Anchor phrases. "You are", "Given the following", "Respond in the format" — strong indicators of block type.
- ›Structural position. Top of prompt usually contains IDENTITY. Closing lines usually contain OUTPUT_FORMAT.
- ›Lexical patterns. XML-style tags (
<context>), markdown headers, JSON skeletons — all signals we use.
When the rules can't classify a block confidently, it goes into a catch-all UNTYPED bucket that gets the most conservative compression policy.
Why typed blocks unlock better compression
A heuristic system asking "how aggressively should I compress this prompt?" has one knob. The compression ratio is averaged across the whole prompt. That's a bad average — different parts of the prompt have wildly different compressibility.
A PromptIR-aware system has one knob per block type. We can compress CONTEXT at 70% while leaving OUTPUT_FORMAT at 0%. The same overall compression ratio, but the parts that mattered are preserved.
This is the entire reason ContextAPI's downstream task accuracy hasn't dropped while compression has gone up. We're not compressing more aggressively. We're compressing more targetedly.
The lowering pass
Once the optimisation is done, the IR has to lower back to a string the LLM can consume. We considered keeping it in some structured form (JSON, XML), but the latency hit of asking models to parse structured wrappers was higher than the win.
The lowering pass renders each block back to natural language, in the original order, with the compression applied. The output looks like a normal prompt — just smaller.
What's next
Two open research questions we're working on:
- ›Cross-block optimisation. Some compressions are only safe if you can see two blocks together. We're prototyping a second pass that runs after per-block compression.
- ›Learned block types. The current rule-based parser handles 92% of prompts well. The remaining 8% need a smarter classifier. We're experimenting with a small encoder model.
Both will land in a future post.
Asha Krishnan
March 21, 2026 · 10 min