New Research Challenges GPU Utilization as Measure of Inference Infrastructure
详细摘要
New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference…
New NAND Research report underscores that busy GPUs aren't an accurate measure of productive output; introduces new Inference Economics Scorecard to measure inference infrastructure by the useful work it delivers, as applied in an eight-hour benchmark with Nebius on NVIDIA HGX B300 GPUs and WEKA NeuralMesh.
CHICAGO, Nov. 16, 2026 /PRNewswire/ -- From Supercomputing 2026: WEKA, the AI data and memory infrastructure company, in collaboration with market intelligence firm NAND Research, today introduced the Inference Economics Scorecard, a new common framework for measuring inference infrastructure productivity and economic value by the useful work it delivers rather than by how busy it keeps GPUs. The scorecard defines nine measures, including tokens per watt, cost per inference, and context retention. It sets four testing rules, including runs of at least eight hours and counting only successful output.
Buyers Are Already Treating Inference Differently, New Research FindsNew research from 451 Research by S&P Global shows organizations are already adapting, projecting that inference will grow from 34% of AI infrastructure spending in 2026 to 46% by 2030 and finding that 41% of organizations running inference buy fundamentally different infrastructure for it than for the rest of their IT stack.
"Inference is where the value of AI is ultimately realized, and our research shows it's reshaping how organizations buy infrastructure," said Greg Macatee, Senior Analyst at 451 Research by S&P Global. "Agents are driving exponential growth in token volume; memory, power, and supply constraints greatly challenge the notion that organizations can keep up simply by adding more GPUs."
For years, training was the dominant AI workload. Because training keeps GPUs busy with large, predictable jobs, GPU utilization became the default measure of AI infrastructure productivity. Inference, especially agentic inference, places very different demands on GPUs.
文章开头,摘自AAP。如需阅读全文,请前往媒体网站。
MarketNews 仅展示媒体发布的标题和摘要,全文请在出版方网站阅读。 (www.aap.com.au)