The AI ecosystem: from electricity to the customer's benefit

AI is not one thing but a stack of layers, and in each of them a different job is done, different money is spent and different value is created. This piece walks the whole stack without the marketing: from electricity and chips to models, applications, and who finally earns what.

It helps to think of the AI ecosystem in layers. At the bottom are electricity, data centres and chips. In the middle are data, software and models. At the top are applications, companies' processes and the benefit that reaches a customer.

1. What are AI, machine learning and language models?

These terms get used interchangeably even though they are different levels.

• Artificial intelligence, or AI, is the umbrella term for systems that do things once assumed to require human intelligence: recognising, predicting, conversing, planning or deciding.
• Machine learning is a way of building AI using data. Rather than writing out every rule by hand, the program learns statistical relationships from examples.
• Deep learning is the part of machine learning that uses large, many-layered neural networks.
• Generative AI produces new content: text, images, audio, video or source code.
• A language model processes and produces language.
• A large language model, or LLM, is a neural network trained on an extremely large quantity of text.
• A multimodal model handles several kinds of content, such as text, images and audio.
• An agent is not usually a wholly new kind of model. It is a system in which a model has been connected to tools, memory, instructions and a loop.

So not all AI is language models. Fraud detection, demand forecasting and industrial quality control, for example, may use much smaller machine learning models.

2. The physical foundation: from electricity to compute

Electricity and cooling

AI computation consumes electricity and produces heat. A large data centre therefore needs:

• a reliable grid connection
• backup power
• cooling
• a physical building and security
• fast network links.

Electricity does not make anything intelligent by itself, but without it no layer above can be used. In large projects the availability of power can be as significant a constraint as the availability of chips.

Chips

The most important are:

• CPU: the ordinary general-purpose processor
• GPU: originally developed for graphics, but well suited to the parallel arithmetic of neural networks
• AI accelerator: a chip designed specifically for AI computation
• memory chips: hold the model's weights and intermediate results during computation
• networking chips: move data between thousands of processors.

A powerful GPU alone is not enough. If memory or the network link is too slow, an expensive processor sits waiting.

What does compute mean?

Compute means how many mathematical operations a system can perform in a given time.

In AI, what matters is not only the speed of a single chip but the whole:

• how many chips are in use
• how fast they talk to each other
• how much fast memory they have
• how efficiently the software uses them
• how much of the time the hardware does useful work.

What is a supercomputer?

An AI supercomputer is not usually one enormous computer. It is a large group of servers that may contain thousands or tens of thousands of processors.

They are joined by a very fast network so as to work as a single compute cluster. By analogy it is thousands of workers who have to solve the same task in a coordinated way. If the flow of information between the workers stalls, the whole thing slows down.

3. Cloud services

A cloud company builds the data centres and rents their capacity to others.

A customer can buy, for example:

• GPU capacity by the hour
• storage
• databases
• ready-made models through an interface
• tools for training and monitoring models.

The business purpose of the cloud is to turn an enormous up-front investment into a rentable service. An application company does not have to build its own data centre in order to use AI.

Large players can operate in several layers at once: they may own the data centre, design their own chips, provide cloud services, train models and sell end-user applications.

4. Data: what does a model learn from?

A machine learning model learns from examples. The data can be:

• text
• images and video
• audio
• source code
• sensor measurements
• a company's transaction and customer records
• question-and-answer examples written by people
• simulated or AI-generated synthetic data.

Volume alone does not settle it. Quality, coverage, accuracy, rights of use and whether the data represents the real situation all matter too.

In companies the hardest work is often organising their own data:

• the records sit in different systems
• terms do not mean the same thing everywhere
• access rights are unclear
• the data is out of date or wrong
• sensitive information may not be shown to everyone.

That is why a company's own data can be worth more commercially than the chance to train a slightly larger model.

5. Training a model

Training a model means adjusting the numbers inside it, its weights.

A language model is shown an enormous quantity of text and asked to predict the next piece of it. When the prediction is wrong, the weights are changed slightly. This is repeated a great many times.

Think of the model as containing billions of adjustable dials. Training searches for the settings that let the model produce plausible continuations to a wide range of inputs.

Pre-training

In pre-training the model learns general structure:

• language and styles of writing
• connections between concepts
• statistical regularities about the world
• the structure of source code
• some amount of problem-solving.

What results is a base model, or foundation model.

A base model is not an ordinary database. What it learned is spread across its weights. That is why it cannot reliably say where any single piece of information came from.

Post-training

A base model is made more usable by post-training. That can include:

• examples of good answers written by people
• rating and comparing answers
• safety training
• teaching the use of tools
• practice at a particular field or task type
• teaching the model to follow instructions.

Pre-training gives the model general capabilities. Post-training shapes how and for what those capabilities are used.

6. Inference: running the model

Inference happens whenever a trained model is given a new input and asked for a result.

In a chat service, for example:

1. The user's text is chopped into tokens, small pieces of text.
2. The tokens are turned into numbers.
3. On that basis the model computes the probabilities for the next token.
4. The system picks the next token.
5. The same repeats until the answer is finished.

So the model builds the answer piece by piece. It does not generally look a finished answer up in some internal table.

The cost of inference depends, among other things, on:

• the size of the model
• the length of the input
• the length of the answer
• the number of reasoning rounds
• the numerical precision used
• how many users are served at once
• how well queries can be handled in batches
• how busy the hardware is kept.

Training and inference are not the same thing

For a model company, training can be an enormous one-off outlay. Once a service has many users, however, inference can become the larger total cost over time.

7. What gets built around the model

A language model on its own is not usually a finished business product.

The prompt

The prompt is the instruction and context handed to the model. Besides the user's question it can carry the application's rules, examples, documents and descriptions of tools.

Retrieval (RAG)

Retrieval means the system looks something up in an external source before answering (the company's own documents, say) and hands the passages it found to the model.

Retrieval:

• does not usually change the model's weights
• helps use current or company-specific information
• can make it possible to show sources
• requires a good search and proper access control.

Fine-tuning

In fine-tuning, the model's weights are adjusted for a particular purpose. It can teach a house style, a classification task or the conventions of an industry.

Put simply:

• Prompt: say now what needs doing.
• Retrieval: bring in the facts needed.
• Fine-tuning: change how the model behaves by default.

The agent

An agent is usually a system of action built around a language model. It can:

1. size up the task
2. pick a tool
3. look something up or use a piece of software
4. examine the result
5. revise the plan
6. carry on until the task is done.

A tool might be a web search, a calendar, email, a database or a company's own system.

An agent's "intelligence" comes, beyond the model, from its tools, instructions, permissions, memory and checks. An agent does not automatically get to act freely: company use generally needs limits, approvals and logs.

8. The application layer proper

The application turns a general-purpose model into a product suited to a particular job. It includes things like:

• the user interface
• authentication and permissions
• the company's own information
• workflows
• integrations with other systems
• error handling
• quality measurement
• security
• billing and customer support.

This layer is where it is decided whether AI produces any real benefit.

Even a top-tier model does not tell you:

• what the user is actually trying to achieve
• where the correct information is
• who is entitled to use it
• when a person has to approve the result
• how the result moves to the next stage of work
• who answers for a mistake.

For that reason the best model does not always mean the best product.

9. The business roles in the ecosystem

10. Where does the money come from?

The most common revenue models are:

• selling chips and servers
• renting data centre capacity
• hourly pricing for cloud compute
• charging for a model interface by usage or by tokens
• a monthly subscription per user
• an enterprise licence
• maintenance and hosting sold around an open model
• consulting and systems integration
• outcome-based pricing, for example per case handled or task resolved
• advertising, or increasing the value of some other core product.

The same euro can pass through several players in the chain. An application company is paid by its customer and pays a model provider, which pays for cloud capacity, which in turn spends on chips, electricity and the data centre.

11. How do the "shares" of cost and value divide?

There is no single universal percentage split. Technical effort, cost and the value a customer feels are three different things.

A useful principle: the technologically most important layer does not necessarily get the largest share of the business value.

The player who captures most value may be the one holding something scarce or hard to copy:

• the best or cheapest compute
• a model that is hard to build
• unique data
• access to customers
• a strong distribution channel
• deep integration into how work is done
• trust, security or regulatory approvals.

When a layer becomes commonplace and easily swapped out, its prices and margins usually come under pressure. Value can then move to the next bottleneck.

12. Why can the cost of an AI service catch you out?

Costs fall into two main groups.

Investment:

• data centres
• chips and servers
• training the model
• acquiring data
• product development.

Recurring costs:

• inference
• electricity and cooling
• cloud services
• storage and data transfer
• staff
• monitoring, security and customer support.

A cheap demo can turn into an expensive production service if:

• there are many users
• every input carries a lot of documents
• the answers are long
• an agent makes several model calls for one task
• the system ends up using the largest model for everything
• more capacity is reserved than is actually used.

Costs are brought down by, for example, using smaller models for easy tasks, shortening the context, handling queries in batches and storing reusable results.

13. The key terms, briefly

• Parameter or weight: a number inside the model learned during training.
• Token: a piece of text that the model handles; one word can be one token or several.
• Context window: the amount of information the model can take into account in a single run. Not the same thing as permanent memory.
• Embedding: a numerical representation of content that makes it possible to search for things close in meaning.
• Vector database: a system for storing embeddings and searching them by similarity.
• API or interface: the way an application sends requests to a model service.
• Latency: how long it takes to get one answer.
• Throughput: how many requests a system handles in a given time.
• Hallucination: the model produces a convincing but false statement.
• Evaluation, or eval: a systematic test of the quality of a model or an application.
• Quantisation: storing the model's numbers at lower precision so it runs more lightly.
• Distillation: transferring a large model's abilities into a smaller one.
• Open weights: the model's weights can be downloaded. This does not necessarily mean the training data or the whole development process is open.
• MLOps: the practices around releasing, monitoring, versioning and maintaining models.
• Guardrails: the rules and checks that limit dangerous or incorrect actions.

The most important idea in all of it

In the ecosystem, an AI model is a little like a car engine. Chips and data centres make it possible to build the engine and run it. The cloud rents out the infrastructure needed. The model lab manufactures the engine. The application company builds the car, the controls and the safety systems around it.

The customer, though, does not generally want to buy an engine, or even a car, for the sake of the technology. They want to get somewhere: process an application faster, find a piece of information, make fewer errors, serve a customer, or sell more.

That is why the final value of an AI business appears only once the compute, the model, the company's own information, the interface and the way work is done all function together.