"Why Falling AI Token Prices Have Not Cut Enterprise Spending"
Token
the smallest unit of text, roughly a word or part of a word, that an AI language model processes
Inference
the process of running a trained AI model to answer a query or complete a task
Artificial Intelligence systems called Large Language Models (LLMs) work by breaking text into small units called tokens. A token is roughly a word or part of a word. Running a trained model to answer a query is called inference. This is different from training, which is the one-time process of teaching the model using large datasets.
AI models fall into two broad groups. Closed or proprietary models, such as OpenAI's GPT, Anthropic's Claude and Google's Gemini, keep their trained parameters under the developer's control. Users reach them through a paid hosted service or an API (a channel that lets one software system use another's service). Open-weight models, such as Meta's Llama, Google's Gemma and Mistral's open releases, let organisations download the trained parameters and run the model themselves, under a licence. Being open-weight does not mean the model is fully open-source. Its training data and training method can still stay secret.
Since AI services became commercial, the per-token cost of inference has fallen sharply. Better chips, more efficient model designs and growing competition among providers have driven this fall. Bodies such as the OECD and reports like the Stanford AI Index track this trend closely. They do this because it shapes how affordable AI becomes for businesses and governments.
Falling per-token prices would normally mean falling total costs. But the nature of AI work is changing at the same time. A simple chatbot takes one question and gives one answer. An AI agent, by contrast, breaks a task into many steps. It can call outside tools, check results, revise its approach, and repeat this cycle until the task is done. Each extra step needs more model calls and more tokens.
Researchers have found that agentic tasks can use vastly more tokens than simple tasks. A 2026 study found that agentic coding tasks used about a thousand times more tokens than basic code-chat tasks. Gartner calls this the 'Inference Paradox'. Routing work to an agentic model can raise a provider's inference cost several times over compared with a basic chatbot call. …
Consider the following statements regarding Artificial Intelligence (AI) models:
1.Closed or proprietary AI models make their trained parameters available for download and independent operation by any organisation.
2.Open-weight AI models allow organisations to download and run the trained model themselves, subject to a licence.
3.Being 'open-weight' necessarily means that a model's training data and training process are also fully disclosed.
How many of the above statements are correct?
Which of the following best describes an 'AI agent' as distinguished from a conventional AI chatbot?
Match List I (AI Model) with List II (Developer) and select the correct answer using the codes given below:
List I:
A.GPT
B.Llama
C.Gemini
D.Claude
List II:
1.Meta
2.Google
3.OpenAI
4.Anthropic
In the context of Large Language Models (LLMs), a 'token' refers to which of the following?
Want the full note, Prelims Facts & MCQ answers?
Create a free account — takes less than a minute.
Create free account