OPEN BENCHMARK METHOD
A transparent benchmark for AI agent memory and token use.
This benchmark explains exactly how memory-hit rate, input share, and leaner context combine to reduce total model tokens. It publishes the scenarios behind Governora's planning language without presenting them as production outcomes.
Total modeled token reduction equals the share of complete model calls avoided by valid memory hits plus the input tokens removed from all remaining calls, divided by the original total input and output tokens.
10% valid hits · 15% leaner input
20% valid hits · 25% leaner input
50% valid hits · 50% leaner input
Observed evidence must replace assumptions
The benchmark equation
Let H be the valid memory-hit rate, I the share of baseline tokens that are input tokens, and C the input-context reduction on model calls that remain. The total token-reduction fraction is H + (1 − H) × I × C. A memory hit avoids both input and output tokens; lean context affects input only.
Three inspectable scenarios
The scenarios use round inputs to reveal sensitivity, not to predict a specific customer result. A workflow with little repetition should stay near the conservative end. The upper case requires half of requests to reuse a valid result and input tokens to represent 80% of baseline volume.
| Scenario | Valid memory hits | Input share | Leaner input on misses | Total reduction |
|---|---|---|---|---|
| Conservative | 10% | 75% | 15% | 20.1% |
| Base | 20% | 75% | 25% | 35.0% |
| High repetition | 50% | 80% | 50% | 70.0% |
How the 70% upper scenario works
Assume 10,000 requests use 4,000 input and 1,000 output tokens each: 50 million baseline tokens. Valid memory answers 5,000 requests without a model call, avoiding 25 million tokens. The remaining 5,000 calls use 50% less input, avoiding another 10 million tokens. Total modeled use falls by 35 million tokens, or 70%.
Quality gates are part of the benchmark
A candidate does not count as a memory hit merely because retrieval found something similar. It must meet the same correctness, scope, freshness, approval, and task-completion standard used for the baseline.
- Valid answer acceptance rate.
- False or stale memory-hit rate.
- Human-review and fallback rate.
- Successfully completed tasks and retries.
Controlled evaluation protocol
Create a representative task set, freeze the model and source snapshot, run a baseline without shared memory, then replay the same tasks with memory-first routing. Report route distribution, input and output tokens, completed-task quality, latency, retries, and non-model cost.
- Publish task-selection criteria and exclusions.
- Use the same quality rubric for both paths.
- Separate memory hits from cache and RAG effects.
- Repeat after source, prompt, model, or memory changes.
What Governora will publish next
Launch100X already records route and token fields. The next evidence milestone is a fixed before-and-after run that replaces modeled hit rates and token counts with observed values. Until then, Governora labels these figures as planning scenarios.
HOW IT WORKS
Four clear steps.
Each step stays connected to the same execution. This makes authority, external work and the final outcome explainable.
- 01
Freeze
Fix the task set, model, source snapshot, prompts, and quality rubric.
- 02
Baseline
Measure tokens, calls, quality, latency, retries, and completed tasks without memory.
- 03
Replay
Run the same tasks with memory-first routing and record every route outcome.
- 04
Publish
Report observed results, rejected hits, overhead, limitations, and the full formula.
QUESTIONS
Simple questions. Direct answers.
What Governora does, what it does not do, and where it fits.
Is 70% a measured Governora customer result?
No. It is a transparent high-repetition planning scenario. Governora does not present it as observed production performance.
Why can the Launch100X range remain lower?
Launch100X uses a different scenario set: 10–30% hits and 15–35% context reduction. Its published 20–48% range remains separate from this general sensitivity benchmark.
Does the equation include provider prompt caching?
No. Prompt-caching price discounts and compute optimizations should be reported separately because they may change cost without reducing the logical token volume.
Can another company reproduce this benchmark?
Yes. The formula, scenario inputs, CSV, quality gates, and evaluation protocol are public. Replace the planning inputs with your own observed workflow data.
Map one governed AI execution.
We will map the actor, decisions, capabilities, evidence, external calls and proof with you.
Plan a pilot review →