🔍 Read the full analysis: Could AI Outcomes Be A Better Measure Than Token Counts? Amazon Weighs In on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Amazon published engineering guidance on October 27 saying AI usage metrics such as token counts can help track adoption but should not be targets for judging engineers. The memo comes as Amazon’s own Nova models and its investment in Anthropic represent contrasting results in the company’s AI strategy; the scale and causes of those outcomes are not fully established by the available information.
Amazon’s engineering leadership said on October 27 that software engineers should be judged by the effects of their work, not by how many AI tokens they consume or how often they use AI tools. The guidance matters because it addresses reported efforts to inflate internal usage rankings and arrives as Amazon’s own AI-model development and its investment in Anthropic present markedly different outcomes.
The memo, published by Amazon and attributed to Terk Terkowitz, a vice president and distinguished engineer in its stores business, introduces a revised role guideline for software engineers. It says the company should focus on impact for customers, the business and teams. The guidance avoids prescribing a lasting AI-specific job definition, saying that one written now could quickly become outdated.
Amazon’s memo calls measures such as token usage, AI-tool adoption, code changes and reviews “deeply flawed” when turned into targets. It warns that optimizing for those inputs can increase costs and complexity rather than improve work. At the same time, it says such measures currently “correlate well” with desired outputs. The distinction is that usage data may help teams understand adoption, but should not become a score for its own sake.
The memo recommends examining both low and high AI adopters: leaders should ask why some teams use the tools little and whether heavy users are delivering more. It also says engineers should document cases where current AI tools do not help, framing that as a finding rather than a failure. Managers are asked to use the tools on their own production work and explain where they help and where they do not.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
Why AI Usage Targets Can Backfire
The guidance addresses a measurement problem facing companies adopting generative AI: activity is easier to count than value. Token totals and tool-use rates can show whether employees are trying a system, but they do not by themselves show whether customers receive better service, work is completed faster, or costs fall. If compensation or rankings depend on those numbers, employees may have reason to generate activity that does not improve the work.
For Amazon, the policy also has implications beyond internal engineering management. The company sells cloud and AI services through AWS and has invested in building and distributing models. Its own guidance suggests that adoption figures alone are an incomplete measure of whether those investments are working. The company’s third-quarter results, due October 29, may provide further financial context, but no results were available in the source material at publication.
The memo does not say AI tools should be abandoned or that usage metrics have no value. It argues for treating them as diagnostic signals, not goals. That distinction leaves room to identify teams needing support while avoiding pressure to use AI for tasks where it adds little.
Products Worth Considering
Dell XPS 13 9350 AI Professional Laptop 13.4 120Hz FHD+ Display
Dell XPS 13 9350 Laptop, 13.4 FHD+ Display, Intel Ultra 7 256V, 16GB DDR5, 1TB SSD
Dell UltraSharp U2725QE 27-Inch 4K UHD IPS Black Monitor
As an affiliate, we earn on qualifying purchases.
From Internal Rankings to Official Guidance
The memo follows earlier reporting about employee use of internal AI tools. In May, the Financial Times reported that some Amazon staff gave an internal agent called MeshClaw unnecessary tasks to increase token counts. That account was based on reporting and should not be treated as a company-confirmed finding. The same reporting described a target for weekly AI use among developers and internal tracking of token activity.
Amazon did confirm to Business Insider that an internal leaderboard called KiroRank had been “deprecated” at the end of May. The company said it was an informal tracker created by a group of employees and was not intended to promote AI use for its own sake. Senior vice president Dave Treadwell also told staff, “Please don’t use AI just for the sake of using AI.” The October memo puts a related principle into formal engineering guidance.
The source article also compares Amazon’s model work with its investment in Anthropic. It cites Business Insider reporting that most flagship Nova models were moved to maintenance, and Amazon’s lifecycle information showing that Nova Canvas, Nova Reel and the first Nova Sonic had been retired. It reports Amazon invested about $18 billion in Anthropic and that a June 30 filing valued its holdings at about $190 billion. Those figures are reported valuations, not cash proceeds, and do not establish the full performance of either business.
“Tokenmaxxing is at odds with frugality.”
— Amazon engineering memo
What the Guidance Leaves Open
The memo does not specify a replacement formula for measuring engineering performance or define how managers should weigh customer outcomes against slower-to-measure business results. It acknowledges that long-term outcomes can be difficult to assess, leaving judgment and local evaluation central to how the guidance is applied.
It is also unclear how widely token-focused rankings were used, how many employees took part in the reported behavior, or whether the new guidance will change evaluation practices across all teams. The May accounts about unnecessary tasks came from media reporting; Amazon’s confirmation that KiroRank was discontinued does not independently verify every detail of those reports.
The Nova and Anthropic figures also require care. A reported portfolio valuation is not the same as realized profit, and the available material does not provide a comparable measure of the models’ commercial returns. Amazon’s scheduled earnings report may add financial information, but it cannot alone establish whether the engineering policy improves productivity.
Amazon’s Earnings and Policy Tests
Amazon is scheduled to report third-quarter results on October 29, one day after the memo’s publication. Those results may offer updated financial information about the company, but the source material does not provide forecasts or confirmed figures, and the earnings release will not necessarily resolve how effective its AI tools are for engineers.
The practical test for the guidance will be how managers use it: whether they review AI adoption as a clue about obstacles and outcomes, rather than as a target employees must meet. The memo also calls for leaders to try the tools on real production tasks and explain where they help. How Amazon implements that approach, and whether other companies adopt similar outcome-based measures, remain developments to watch.
Key Questions
What did Amazon announce?
Amazon published revised engineering guidance on October 27 saying software engineers should be evaluated by the impact of their work, not by token counts or AI-tool usage alone.
Does Amazon say AI usage metrics are useless?
No. The memo says usage measures currently correlate with important outputs, but calls them flawed when used as targets. It recommends using them to understand adoption and investigate problems.
What was KiroRank?
KiroRank was an internal leaderboard tracking AI activity. Amazon told Business Insider it was an informal employee-created tracker and said it had been discontinued at the end of May.
What does the memo say employees should do if AI does not help?
It says they should document and share that finding. The memo describes lack of benefit from current tools as a finding, not a failing.
When could more information about Amazon’s performance become available?
Amazon was scheduled to report third-quarter results on October 29, 2026. Those results could provide financial updates, though they may not answer how the new engineering guidance affects productivity.
Source: ThorstenMeyerAI.com
