Skip to content
Tekoälli

Layer 1

Infrastructure

Electricity, chips and very large buildings. Everything starts here.

On this page

AI computation consumes electricity and produces heat. Electricity does not make anything intelligent by itself, but nothing in the layers above runs without it. In a large project a grid connection can be as hard a constraint as the supply of chips, and in Europe it is often the harder of the two.

What happens in this layer

  1. 01

    A data centre is more than servers

    It needs a dependable grid connection, backup power, cooling, physical security and fast network links to the outside world.

  2. 02

    Chips are not only graphics cards

    Ordinary processors, GPUs or other accelerators, memory chips to hold the model's numbers, and networking silicon to move data between machines.

  3. 03

    The bottleneck is rarely the arithmetic

    If memory or the network is too slow, an expensive chip sits and waits. How much of the time the hardware does useful work is what sets the price.

  4. 04

    A supercomputer is a large group of machines

    Thousands of chips joined by a very fast network so that they behave as one. When the traffic between them stalls, all of it stalls.

A common misconception

Data centre
The building the servers are in: power, cooling, network and security.
CPU
A computer's general-purpose processor, which does a bit of everything.
GPU
A chip originally built for graphics that suits the parallel arithmetic of neural networks.
AI accelerator
A chip designed specifically for AI computation.
Compute cluster
A large group of servers working as a single whole.
Latency
How long it takes to get one answer.
Throughput
How many requests a system handles in a given time.

Articles on this topic

What one AI question actually uses

The bottle-of-water claim comes from a real study whose result changed meaning as it travelled. What it measured, what production measurements now show, and why two honest numbers can differ by a factor of a hundred.

12 min