TL;DR: What’s This Blog Post About?
“Tokenmaxxing” is a term that may seem strange on the surface, but really is ultimately related to achieving efficiencies—or, rather, lack of efficiencies—in the use of AI tokens.
What AI tokenmaxxing refers to is using up AI tokens, usually in a work context. The more the better is a common view among tokenmaxxers, seeing a link between pushing AI to the limits and getting better results.
However, “max”-xing out the tokens is not a good long-term strategy. This key finding was discovered in a series of expensive learning lessons by enterprise corporations. Uber, Microsoft, and others that saw costs rack up by providing AI tools with lax limits on token usage, while not seeing the returns desired.
Much of this boils down to failures in two areas: strategizing and governance. Believing that employees will magically become more productive if you just give them access to AI tools is a fallacy many are becoming wise to. Employees need strategic guidance grounded in governance plans founded on preserving safety and efficiency in AI usage.
Five Key Takeaways from This Substack Post
- Uber had a reckoning with tokenmaxxing when it burned through its entire AI budget within a mere four months. What made it especially painful was that Uber did not see that much productivity gains despite all the spending.
- In some companies, especially in AI-forward tech companies, an irrational competition broke out where the use of tokens was more or less equated with productivity. Some companies even had unauthorized employee-built dashboards tracking AI usage to rank employees by how many tokens each user is using up.
- What are AI tokens, you may be wondering? Tokens are units of measurement for how much “compute” an AI model is using. With LLMs like ChatGPT and Claude, one token is equal to just a small bit of text, usually about four characters or three quarters of a word. So, for those who are generating large responses, you can see how token usage can rack up.
- Measuring AI by productivity it ends with rather than mere use is a start for businesses—don’t just assume that AI use translates to improved productivity.
- Having a reliable way to track AI usage for a given AI tool is one way of making sure that usage stays efficient within your budget.
Why This Matters for Business Owners
In addition to governance and strategizing to limit excessive tokenmaxxing, make sure you are setting actual limits for employees. (You can set such limits in many AI platforms, although beware that limits are not always on by default.)
Do not view your employees as just one mass of AI users. Instead, consider how many approaches to AI usage there may be.
One problem is that some users may end up being “power users” who tokenmaxx. This can be all and well unless these supposed power users are simply using AI a lot without much gain in productivity.
Then there are users who are not so much power users as just inefficient or uncreative users. Examples would be employees having AI conversations that do not have any translatable productivity gain, such as asking the AI for a weather report.
Business owners need to reckon with this reality, and realize that divvying up access for certain employees or divisions within a company can be wise. For instance, a maintenance team likely needs much less AI than an IT department, so a higher allowance to IT makes sense.
A good rule of thumb is to figure out approximately how much token usage a given employee or company division will need, and then provide that amount. There really is no need to allow for more than is necessary, because that could lead to expensive waste.
Is AI “Valuemaxxing” the Next Step in AI Strategies for Business?
If tokenmaxxing can get wasteful because it falsely equates more tokens with more productivity, then valuemaxxing may be the best way forward.
This comes from identifying what is the most valuable way to spend tokens.
That can involve judicious model selection according to a given task’s demands. (Not everything needs to be given to the best available model.)
Another strategy is good old prompt engineering, of which many employees may yet be ignorant.
A guiding principle for valuemaxxing that perhaps this whole blog post is about is that with AI, more is not always more. Sometimes a shorter prompt, or less context (e.g., upload the only relevant page from a document, rather than the full document), or less AI power, is perfectly adequate to get the response you want, and at lower costs from less tokens.
Signing Off
The perils of tokenmaxxing is indicative of a trend that not even Microsoft is safe from, which is being unable to discern between excessive AI use and efficient AI use. The way out is through valuemaxxing, getting the most out of AI systems without accruing undue expenses.
