The hardware that can break AI’s memory wall
The hardware that can break AI’s memory wall
Publish Date: 2026-05-26 04:45:00
Source Domain: www.weforum.org
- Artificial intelligence (AI) is hitting a “memory wall” that slows performance and drives up costs, ultimately limiting progress.
- Standard computer architecture, in which memory and processing are separate, is becoming inefficient in the face of AI’s rapid growth.
- Alternative models to combat the AI bottleneck could include compute-in-memory systems, brain-inspired spiking neural networks, event-based sensors or using lower-precision or approximate computing.
Artificial intelligence (AI) is running into a physical bottleneck. As models grow larger and more complex, an increasing share of time and energy is spent moving data between memory, where information is stored, and processors, where calculations are performed.
The memory wall problem matters for two reasons.
First, the energy demand of large-scale AI systems is increasing rapidly, driving up costs and the infrastructure needed to train and run them.
Second, many valuable uses of AI depend on fast decisions made locally on edge devices rather than in the cloud, in settings where power, size, connectivity and delay all matter.
Medical devices, autonomous vehicles or rescue drones cannot always rely on sending information to a distant data centre for processing and waiting for a response; instead, they often need hardware that makes on-device AI more practical.
As we described in Frontiers in Science, there are three main ways to ease this bottleneck: move computation closer to the data, draw on the brain’s event-driven information-processing method and use lower-precision or stochastic computing where exact arithmetic is unnecessary.
Together, these approaches could support a new generation of AI hardware that is faster, more efficient and better suited to large-scale infrastructure and edge applications.
3 new approaches to break the AI memory wall
1. Bring computation to memory
Most computers still use the long-established von Neumann architecture, in which memory and processing are physically separate. To…
