TL;DR: What’s This Blog Post About?
“Tokenmaxxing” is a term that may seem strange on the surface, but really is ultimately related to achieving efficiencies—or, rather, lack of efficiencies—in the use of A.I. tokens.
What A.I. tokenmaxxing refers to is using up A.I. tokens, usually in a work context. The more the better is a common view among tokenmaxxers, seeing a link between pushing A.I. to the limits and getting better results.
However, “max”-xing out the tokens is not a good long-term strategy. This key finding was discovered in a series of expensive learning lessons by enterprise corporations. Uber, Microsoft, and others that saw costs rack up by providing A.I. tools with lax limits on token usage, while not seeing the returns desired.
Much of this boils down to failures in two areas: strategizing and governance. Believing that employees will magically become more productive if you just give them access to A.I. tools is a fallacy many are becoming wise to. Employees need strategic guidance grounded in governance plans founded on preserving safety and efficiency in A.I. usage.
Five Key Takeaways from This Substack Post
- Uber had a reckoning with tokenmaxxing when it burned through its entire A.I. budget within a mere four months. What made it especially painful was that Uber did not see that much productivity gains despite all the spending.
- In some companies, especially in A.I.-forward tech companies, an irrational competition broke out where the use of tokens was more or less equated with productivity. Some companies even had unauthorized employee-built dashboards tracking A.I. usage to rank employees by how many tokens each user is using up.
- What are A.I. tokens, you may be wondering? Tokens are units of measurement for how much “compute” an A.I. model is using. With LLMs like ChatGPT and Claude, one token is equal to just a small bit of text, usually about four characters or three quarters of a word. So, for those who are generating large responses, you can see how token usage can rack up.
- Measuring A.I. by productivity it ends with rather than mere use is a start for businesses—don’t just assume that A.I. use translates to improved productivity.
- Having a reliable way to track A.I. usage for a given A.I. tool is one way of making sure that usage stays efficient within your budget.
Why This Matters for Business Owners
In addition to governance and strategizing to limit excessive tokenmaxxing, make sure you are setting actual limits for employees. (You can set such limits in many A.I. platforms, although beware that limits are not always on by default.)
Do not view your employees as just one mass of A.I. users. Instead, consider how many approaches to A.I. usage there may be.
One problem is that some users may end up being “power users” who tokenmaxx. This can be all and well unless these supposed power users are simply using A.I. a lot without much gain in productivity.
Then there are users who are not so much power users as just inefficient or uncreative users. Examples would be employees having A.I. conversations that do not have any translatable productivity gain, such as asking the A.I. for a weather report.
Business owners need to reckon with this reality, and realize that divvying up access for certain employees or divisions within a company can be wise. For instance, a maintenance team likely needs much less A.I. than an IT department, so a higher allowance to IT makes sense.
A good rule of thumb is to figure out approximately how much token usage a given employee or company division will need, and then provide that amount. There really is no need to allow for more than is necessary, because that could lead to expensive waste.
Is A.I. “Valuemaxxing” the Next Step in A.I. Strategies for Business?
If tokenmaxxing can get wasteful because it falsely equates more tokens with more productivity, then valuemaxxing may be the best way forward.
This comes from identifying what is the most valuable way to spend tokens.
That can involve judicious model selection according to a given task’s demands. (Not everything needs to be given to the best available model.)
Another strategy is good old prompt engineering, of which many employees may yet be ignorant.
A guiding principle for valuemaxxing that perhaps this whole blog post is about is that with A.I., more is not always more. Sometimes a shorter prompt, or less context (e.g., upload the only relevant page from a document, rather than the full document), or less A.I. power, is perfectly adequate to get the response you want, and at lower costs from less tokens.
Signing Off
The perils of tokenmaxxing is indicative of a trend that not even Microsoft is safe from, which is being unable to discern between excessive A.I. use and efficient A.I. use. The way out is through valuemaxxing, getting the most out of A.I. systems without accruing undue expenses.
