Devon

Lead engineer · AI usage, harness practices and where spend could have gone further.

Demonstration report. This engineer and their usage are synthetic.

This page is for the engineer and their manager to find cheaper defaults and harness fixes. It is never used to rank people.

Period

Summary

FY2027 to date

AI spend

$3,203

200 sessions · $16 each

Savings opportunity

$1,038

32% of spend, at list price

Practices on target

2 of 7

Focus: rtk coverage, Context past 100K

Cache hit rate

81%

Team 82% · org 82%

FY2027 to date · estimated at list price

$1,038 of $3,203 spend

Largest saving first

Improve prompt caching$482
Cache hit rate 81% against an 85% target.
Keep stable context at the front of prompts and avoid switching models mid-session, so 85% of input reads from cache.
Keep context lean$349
31% of input tokens were past 100K of context.
Compact or /clear between tasks instead of carrying more than 100K tokens of context.
Right-size models$162
43% of routine spend ran on Opus: code research $237 of $541, documentation $8 of $15, ops $136 of $329, presentation $14 of $30.
Run docs, decks, ops and read-only research on Sonnet instead of Opus.
Route commands through rtk$45
1% of 1,616 Bash calls ran through rtk; 4.2M tokens of command output went into context.
rtk compacts test, build, git and search output before it reaches the model (about 70% smaller).

Compared with Video Rendering and the organization

Practice	Devon	Team	Org	Target	Recommendation
rtk coverage	1%Off target	2%	56%	≥ 80% of Bash calls	Install the rtk hook so test, build, git and search commands return compact output.
Knowledge MCP use 5 findings saved back	28%Needs attention	28%	30%	≥ 40% of sessions	Search the knowledge MCP before re-researching a codebase, and save findings back with remember.
Cache hit rate	81%Needs attention	82%	82%	≥ 85% of input	Keep CLAUDE.md and tool lists stable, avoid mid-session model switches, and resume sessions instead of restarting.
Context past 100K	31%Off target	17%	10%	≤ 10% of input	Compact or /clear between tasks; hand broad searches to subagents so their reads stay out of the main context.
Opus on routine work	43%Needs attention	43%	44%	≤ 25% of routine spend	Default routine sessions to Sonnet and keep Opus for harness, design and hard debugging work.
Sessions using subagents	48%On target	48%	50%	≥ 30% of sessions	On target
Tool error rate	2.7%On target	3.0%	3.0%	≤ 4% of tool calls	On target

Is the engineer moving toward the targets?

Month	rtk	Knowledge	Cache	Context	Model fit	Subagents	Savings opportunity
Feb 2026	–	–	57%Off target	30%Off target	62%Off target	26%Needs attention	$65
Mar 2026	–	–	61%Off target	30%Off target	62%Off target	23%Needs attention	$52
Apr 2026	–	–	64%Off target	32%Off target	62%Off target	36%On target	$75
May 2026	4%Off target	–	68%Off target	29%Off target	55%Off target	43%On target	$145
Jun 2026	5%Off target	11%Off target	72%Needs attention	32%Off target	53%Off target	42%On target	$317
Jul 2026	0%Off target	26%Needs attention	81%Needs attention	29%Off target	40%Needs attention	37%On target	$99
Aug 2026	1%Off target	34%Needs attention	80%Needs attention	29%Off target	38%Needs attention	60%On target	$125
Sep 2026	1%Off target	39%Needs attention	83%Needs attention	33%Off target	30%Needs attention	42%On target	$69
Oct 2026	0%Off target	32%Needs attention	88%On target	31%Off target	27%Needs attention	70%On target	$90

FY2027 to date

By model tier, by month

Routine types (documentation, presentation, ops, code research) rarely need Opus

Cache reads are billed at a fraction of uncached input; a high share is what good caching looks like

Model	Requests	Uncached input	Cache reads	Cache writes	Output	Spend
Opus 5.5	5,915	22.2M	290M	19.9M	7.8M	$403
Opus 5	3,417	26.5M	171.7M	12.6M	5.4M	$432
Opus 4.8	4,937	58.4M	236.4M	18.8M	7.5M	$714
Opus 4.6	3,115	52M	120.9M	11M	4.5M	$502
Opus 4.5	127	3.1M	4.6M	490K	194K	$26
Sonnet 5.5	7,585	26.8M	366.8M	25.1M	9.3M	$246
Sonnet 5	6,025	41.7M	282.1M	20.7M	8.4M	$413
Sonnet 4.6	3,407	47.9M	144M	12.3M	5M	$308
Sonnet 4.5	120	2.7M	4.1M	435K	143K	$13
Haiku 5.5	7,524	26.7M	376.9M	25.8M	11.8M	$16
Haiku 4.5	4,377	41.4M	204M	15.7M	6M	$111
GPT Realtime (meetings)	261	1.7M	11M	812K	311K	$19
Total	46,810	351.1M	2212.5M	163.6M	66.3M	$3,203

Spend and tokens come from LiteLLM gateway logs and imported agent session history, priced at API list rates.
Savings are estimates: routine Opus work repriced at the Sonnet available at the time; uncached input moved to cache reads up to an 85% hit rate; rtk removing 70% of unfiltered command output, which stays in context for about half the session's remaining turns; and 50% of input past 100K context avoided by compacting. Levers overlap, so the total is an upper bound.
rtk and the knowledge MCP are scored from their rollout (rtk May 2026, knowledge MCP Jun 2026).
Generated Oct 20, 2026, 7:46 AM.