🔍 Read the full analysis: A Better Way To Track AI Progress, According To Amazon’s Bets on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Amazon published engineering guidance on October 27 saying AI usage measures such as token counts should not become performance targets. The memo says these measures can still help identify patterns, but engineers should be judged by outcomes for customers, the business and their teams.
Amazon published revised guidance for its software engineers on October 27, saying measures such as AI token usage and tool adoption should not be used as performance targets. The memo from senior engineering leaders argues that customer, business and team outcomes matter more than how much AI a worker uses, a distinction that comes as Amazon’s own AI projects and its investment in Anthropic show different results.
The memo, written by Terk Terkowitz, a vice president and distinguished engineer in Amazon’s stores business, introduces a revised role guideline for software engineers. It says the guidance avoids prescribing specific AI practices because tools are changing quickly. Instead, it centers the job on impact and asks leaders to assess whether work produces meaningful results.
Amazon’s memo calls input measures such as token usage, AI adoption, code changes and code reviews “deeply flawed” when used as targets. These measures can be useful indicators, it says, but turning them into goals may increase costs and system complexity without improving the outcome. The memo also says those inputs currently “correlate well” with results that matter, so it does not call for abandoning measurement altogether.
The guidance asks managers to examine both low and high AI adoption. They should investigate why teams near the 10th percentile are using the tools less, and whether teams near the 90th percentile are delivering better results. Engineers whose work does not benefit from current tools should document why, rather than be treated as failing to adopt AI. Leaders are also told to use the tools on production work and explain where they help and where they do not.
Six months at Amazon
From token leaderboards to an outcomes memo
Amazon spent 2026 learning the same lesson twice: once with how its engineers use AI, once with its own AI models.
- May 2026Token leaderboards backfireFinancial Times (via The Decoder): staff give an internal agent tool pointless tasks to inflate token counts. Reported, anonymous sources.
- Late May 2026KiroRank “deprecated”Amazon spokesperson to Business Insider. SVP Dave Treadwell: “Please don’t use AI just for the sake of using AI.”
- 24 Jul 2026San Francisco AGI Lab site closesConfirmed by Amazon to GeekWire. Frontier model research continues under Pieter Abbeel.
- 28 Jul 2026Most flagship Nova models wound downBusiness Insider, people familiar: Premier, Omni, Reel, Canvas to “keep the lights on”. Amazon: models “remain one of the most important things we’re working on”.
- 14 / 30 Sep 2026Nova Sonic v1, Nova Reel, Nova Canvas reach end of lifeAWS Bedrock model lifecycle page. Requests to them now fail.
- 26 Oct 2026Anthropic files its S-1Amazon notes: $6.2 billion principal, 12% ownership cap that Amazon can waive. Anthropic commits at least $110 billion to Amazon through April 2036.
- 27 Oct 2026The memo: outcomes, not tokenmaxxingUsage and throughput metrics are “deeply flawed” as targets. “Tokenmaxxing is at odds with frugality.”
- 29 Oct 2026Amazon’s third-quarter resultsAnnounced by Amazon. Not yet published when this article went live.
Sources: Amazon engineering memo (27 Oct 2026); Business Insider (29 May and 28 Jul 2026); GeekWire (24 Jul 2026); The Decoder (12 May 2026, summarising the Financial Times); AWS Bedrock model lifecycle (read 28 Oct 2026); Anthropic S-1 (26 Oct 2026).
Powered by Thorsten Meyer AI
Why Amazon Is Changing Its AI Scorecard
The memo addresses a growing management problem: usage is easy to count, while the value of AI-assisted work can be harder to measure. If workers are rewarded for generating tokens or adopting a tool, they may have an incentive to use it even when it adds little value. Amazon’s guidance explicitly warns that target-setting can raise costs and complexity.
For employees, the policy makes clear that AI use is not the desired outcome. Engineers are expected to exercise judgment about when tools help, and explain cases where they do not. For companies evaluating AI investments, the distinction matters because adoption figures alone do not show whether customers receive better service, work gets completed faster or costs fall.
The memo does not establish that Amazon has found a definitive way to quantify AI’s impact. Instead, it sets a direction: use adoption data to spot questions and differences among teams, then assess the work itself. That is a more cautious standard than treating higher usage as proof of productivity.
Products Worth Considering
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
Logitech MX Keys S Wireless Keyboard, Graphite
Samsung 32-Inch Odyssey OLED G8 (G80SH) 4K Gaming Monitor
As an affiliate, we earn on qualifying purchases.
From Internal Leaderboards to Formal Guidance
The guidance follows reports of internal efforts to rank employees by AI use. In May, the Financial Times reported that some Amazon staff gave an internal agent tool, MeshClaw, unnecessary tasks to increase token counts. The account was based on reported, anonymous employee descriptions and should be treated as reporting, not as a finding established by Amazon’s memo.
At the end of May, an Amazon spokesperson confirmed to Business Insider that a leaderboard called KiroRank had been deprecated. The company described it as an informal tracker created by a group of employees and said it was not intended to encourage AI use for its own sake. Senior vice president Dave Treadwell also told staff not to use AI just for the sake of using it.
The source report connects the new guidance to Amazon’s wider AI portfolio. Business Insider reported in July that most flagship Nova models had been moved to “keep the lights on,” while Amazon Web Services’ lifecycle information lists Nova Canvas, Nova Reel and the first Nova Sonic as retired. In contrast, the report says Amazon invested about $18 billion in Anthropic and valued its holdings at about $190 billion in a June 30 filing. Those figures concern different kinds of measures and do not by themselves establish the commercial performance of either AI effort.
““Tokenmaxxing is at odds with frugality.””
— Amazon’s revised engineering guidance
What the Guidance Does Not Measure
The memo does not specify a replacement formula for measuring AI’s contribution, nor does it set out detailed standards for how managers should compare results across different roles or teams. It says long-term outcomes can be slow and difficult to measure, while offering judgment and examination of high- and low-adoption groups as practical steps.
It also remains unclear how consistently the revised guidance will be applied across Amazon or whether it will change individual performance reviews. The source material does not provide company-wide data showing how much AI use has affected output, costs or customer outcomes. Amazon’s third-quarter results, due October 29, may offer financial context, but they would not by themselves isolate the effect of its AI tools or investments.
Amazon’s Results Are Due October 29
Amazon is scheduled to report its third-quarter results on October 29, one day after publishing the engineering guidance. Those results will provide an update on the company’s overall business, but the source material does not establish that Amazon will disclose a direct measure of AI productivity or the performance of individual models.
The next test for the memo will be how managers put it into practice: whether they use adoption figures as diagnostic signals rather than scorecards, and whether they can connect AI-assisted work to outcomes for customers and teams. Any assessment of Amazon’s broader AI bets will also need to distinguish its in-house Nova products from its investment in Anthropic, and avoid treating usage or valuation figures as proof of operating success.
Key Questions
What did Amazon’s new engineering memo say?
It says engineers should be evaluated by outcomes for customers, the business and their teams, not by token counts or AI usage alone. Usage metrics may help identify patterns, but the memo warns against making them targets.
Is Amazon telling employees to stop using AI?
No. The guidance does not reject AI tools. It asks employees and managers to judge where the tools help and where they do not, rather than treating more usage as automatically better.
What was KiroRank?
KiroRank was an internal leaderboard tracking AI use. Amazon said, as reported by Business Insider, that it was an informal employee-created tracker and that it had been deprecated at the end of May.
What does Amazon’s guidance say about employees who use AI less?
Managers are asked to investigate why low-adoption teams use the tools less. The memo says work that does not benefit from current AI tools is a finding, not a failing, and employees should explain the reasons.
Will Amazon’s October 29 earnings report show whether AI is paying off?
The report will provide Amazon’s third-quarter financial results, but it is not clear whether it will include a direct measure of AI’s effect on productivity or separate results for Nova and Anthropic-related activity.
Source: ThorstenMeyerAI.com
