HBM supply, hybrid bonding, and glass cores fight over how dies are stacked and fed. Processing-in-memory (PIM) and near-memory accelerators attack a different tax: moving weights and activations across the memory wall for recommendation, retrieval, and edge inference. Memory vendors and AI chip startups are shipping PIM-capable DRAM and logic-near-DRAM blocks so bandwidth stays local to the array.
The industrial point is joules per token and rack density—not a new solder pitch. If software stacks cannot target PIM kernels, the silicon stays a lab demo.
What PIM changes
- Bandwidth locality — Compute where the bits already sit for memory-bound layers.
- Power envelope — Less shuttle energy for embedding-heavy and sparse workloads.
- System design — New programming models beside GPU-centric CUDA paths.
What still limits adoption
Toolchains, numerical precision quirks, and which layers actually benefit. Roadmaps that confuse PIM with CXL pooling or HBM capacity races miss the point. This is not hybrid bonding yield, not CoPoS panels, and not LPO/CPO optics.
What to watch next
- Which memory OEMs publish production PIM SKUs with supported frameworks.
- Measured energy-per-inference deltas on real retrieval/embedding services.
- Whether AI servers reserve sockets for PIM DIMMs beside HBM GPUs.
Packaging moves dies closer. PIM tries to stop moving the data at all.