New Research Challenges GPU Utilization as Measure of Inference Infrastructure
Résumé détaillé
New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference…
New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference infrastructure by the useful work it delivers, as applied in an eight-hour benchmark with Nebius on NVIDIA HGX B300 GPUs and WEKA NeuralMesh.
CHICAGO, Nov. 16, 2026 /PRNewswire/ -- From Supercomputing 2026: WEKA, the AI data and memory infrastructure company, in collaboration with market intelligence firm NAND Research, today introduced the Inference Economics Scorecard, a new common framework for measuring inference infrastructure productivity and economic value by the useful work it delivers rather than by how busy it keeps GPUs. The scorecard defines nine measures, including tokens per watt, cost per inference, and context retention. It sets four testing rules, including runs of at least eight hours and counting only successful output.
Buyers Are Already Treating Inference Differently, New Research FindsNew research from 451 Research by S&P Global shows organizations are already adapting, projecting that inference will grow from 34% of AI infrastructure spending in 2026 to 46% by 2030 and finding that 41% of organizations running inference buy fundamentally different infrastructure for it than for the rest of their IT stack.
"Inference is where the value of AI is ultimately realized, and our research shows it's reshaping how organizations buy infrastructure," said Greg Macatee, Senior Analyst at 451 Research by S&P Global. "Agents are driving exponential growth in token volume; memory, power, and supply constraints greatly challenge the notion that organizations can keep up simply by adding more GPUs."
For years, training was the dominant AI workload. Because training keeps GPUs busy with large, predictable jobs, GPU utilization became the default measure of AI infrastructure productivity. Inference, especially agentic inference, places very different demands on GPUs.
Début de l'article, extrait de AAP. Pour lire la suite, ouvrez l'article chez le média.
MarketNews présente le titre et le résumé publiés par le média. L'article complet se lit sur le site de l'éditeur. (www.aap.com.au)