Technology
What is an NPU?
Apple put one into the iPhone in 2017, laptops now advertise them on the box, and robots use them to see. A neural processing unit, or NPU, is the part of a chip built for one job: the arithmetic of neural networks.
In short. Neural networks spend most of their time multiplying and adding large tables of numbers. An NPU is built for exactly that, so it does more of it for each unit of energy than a general-purpose processor. Makers rate NPUs in TOPS, trillions of operations per second, but chip researchers warn that such a figure says little on its own about how a real model will run. What an NPU makes possible is AI that runs on the device itself.
CPU, GPU, NPU
- The CPU runs everything, one flexible step after another.
- The GPU was built for graphics: thousands of simple calculations at once, which also suits AI.
- The NPU is narrower still. It is built around the multiply-and-add arithmetic of neural networks, often with lower-precision numbers, to do more of it for less energy.
Apple’s A11 chip of 2017 had a “neural engine”, purpose-built for machine learning and rated at up to 600 billion operations per second; it made Face ID and Animoji possible. In 2024 Microsoft launched its Copilot+ PCs around NPUs capable of more than 40 trillion operations per second, alongside the CPU and the GPU.
What TOPS does not tell you
TOPS is the number on the box, and it is a peak. How fast a real model runs also depends on how quickly data reaches the chip, how well the software uses it, and what precision the model needs. Researchers who design these chips titled a paper “TOPS/W (alone) considered harmful”, and describe what else to measure when comparing them, including the flexibility to run a wide range of networks.
Why it matters for privacy
An NPU is what lets a phone or a laptop run AI features on the device itself. Work done on the device can stay on it, and it keeps working without a connection. The trade-off is size: the largest models still run in data centres. More on that choice: at home or in the cloud?
Where we use one
Alkyon’s dock is being designed around a chip with an NPU, so the robot’s camera images can be processed at home rather than in a data centre: a camera in your home. Lychnos Ring works the other way: it sends your voice to our EU cloud, where speech recognition and the language model run. Both are in development.
Sources, read 4 November 2026: Apple Newsroom, the future is here: iPhone X, 12 September 2017; Microsoft, introducing Copilot+ PCs, 20 May 2024; Sze, Chen, Yang and Emer, efficient processing of deep neural networks: a tutorial and survey, Proceedings of the IEEE, 2017, and how to evaluate deep neural network processors: TOPS/W (alone) considered harmful, IEEE Solid-State Circuits Magazine, 2020.
Questions
What does an NPU do?
It runs the arithmetic of neural networks, the kind of AI behind face recognition, speech and image features, on hardware built for it, so the device can do that work quickly and with less energy than on its main processor.
Is a higher TOPS number always better?
Not on its own. TOPS is a peak figure; real speed depends on the model, the memory and the software. Researchers who design these chips argue that TOPS per watt alone is a poor way to compare them.