AI's new metric: Intelligence per dollar, not just smarts

For years, the AI industry measured itself by a single question: which model is the smartest. That race hasn't ended, but a second metric is gaining ground just as fast, how much useful intelligence a company can extract per dollar spent.

One such case is that of the overhaul of Amazon’s Alexa+. According to internal documents seen by reporters, the tech giant is routing more traffic through its weaker models than to Anthropic's better but more expensive models.

While it’s not about avoiding expensive intelligence, the aim is to use it only when necessary and as part of an effort to bring down the cloud computing costs of Alexa+ as the platform grows in scale.

According to Kylan Gibbs, CEO of Inworld, which makes voice AI software, it is simply a reflection of what is happening in the industry at large.

With more models reaching a level where they can be considered good enough for daily business purposes, there is no need to chase perfection anymore.

Inworld has responded by building separate research teams focused specifically on making its models cheaper and faster, treating that as a distinct engineering goal from raw intelligence gains.

According to Peter Gostev, AI Capability Lead at Arena AI, comparing the models by their effectiveness becomes more difficult than comparing their performance.

There are four metrics that should be considered: reliability with which the model accomplishes the task, fees charged by the model to understand the request and answer it, how often the model reuses already processed information to reduce the cost, and the number of additional operations needed for accomplishing the task.

The inexpensive model per token might become expensive because of the necessity to accomplish the task several times.