– PCs made and sold in 2024 and beyond will generally include AI and machine-learning processing capabilities
– Neural Processing Units (NPUs) are specialized blocks on high-end Intel and AMD CPUs that can accelerate generative AI and machine-learning workloads
– Qualcomm’s Windows PCs and Apple’s M-series chips for Macs already include NPUs
– x86 PCs with Intel’s Core Ultra chips, AMD’s Ryzen CPUs, and Ryzen 8000G desktop CPUs will introduce NPUs to mainstream PC users
– Currently, Windows has limited support for NPUs, but that is slowly changing
– Microsoft’s DirectML API is being expanded to support Intel’s Meteor Lake NPUs, allowing third-party apps to take advantage of built-in NPUs
– NPUs offer power efficiency benefits compared to running AI workloads on GPUs or CPUs
– Intel and Microsoft are working towards a model where NPUs are treated like GPUs, with developers targeting DirectX rather than specific hardware
– Intel is offering GPU-style driver updates for its NPUs
– Windows recognizes NPUs as “graphics cards with no rendering capability”
When it announced the new Copilot key for PC keyboards last month, Microsoft declared 2024 “the year of the AI PC.” On one level, this is just an aspirational PR-friendly proclamation, meant to show investors that Microsoft intends to keep pushing the AI hype cycle that has put it in competition with Apple for the title of most valuable publicly traded company.
But on a technical level, it is true that PCs made and sold in 2024 and beyond will generally include AI and machine-learning processing capabilities that older PCs don’t. The main thing is the neural processing unit (NPU), a specialized block on recent high-end Intel and AMD CPUs that can accelerate some kinds of generative AI and machine-learning workloads more quickly (or while using less power) than the CPU or GPU could.
Qualcomm’s Windows PCs were some of the first to include an NPU, since the Arm processors used in most smartphones have included some kind of machine-learning acceleration for a few years now (Apple’s M-series chips for Macs all have them, too, going all the way back to 2020’s M1). But the Arm version of Windows is a insignificantly tiny sliver of the entire PC market; x86 PCs with Intel’s Core Ultra chips, AMD’s Ryzen 7040/8040-series laptop CPUs, or the Ryzen 8000G desktop CPUs will be many mainstream PC users’ first exposure to this kind of hardware.
Right now, even if your PC has an NPU in it, Windows can’t use it for much, aside from webcam background blurring and a handful of other video effects. But that’s slowly going to change, and part of that will be making it relatively easy for developers to create NPU-agnostic apps in the same way that PC game developers currently make GPU-agnostic games.
The gaming example is instructive, because that’s basically how Microsoft is approaching DirectML, its API for machine-learning operations. Though up until now it has mostly been used to run these AI workloads on GPUs, Microsoft announced last week that it was adding DirectML support for Intel’s Meteor Lake NPUs in a developer preview, starting in DirectML 1.13.1 and ONNX Runtime 1.17.
Though it will only run an unspecified “subset of machine learning models that have been targeted for support” and that some “may not run at all or may have high latency or low accuracy,” it opens the door to more third-party apps to start taking advantage of built-in NPUs. Intel says that Samsung is using Intel’s NPU and DirectML for facial recognition features in its photo gallery app, something that Apple also uses its Neural Engine for in macOS and iOS.
The benefits can be substantial, compared to running those workloads on a GPU or CPU.
“The NPU, at least in Intel land, will largely be used for power efficiency reasons,” Intel Senior Director of Technical Marketing Robert Hallock told Ars in an interview about Meteor Lake’s capabilities. “Camera segmentation, this whole background blurring thing… moving that to the NPU saves about 30 to 50 percent power versus running it elsewhere.”
Intel and Microsoft are both working toward a model where NPUs are treated pretty much like GPUs are today: developers generally target DirectX rather than a specific graphics card manufacturer or GPU architecture, and new features, one-off bug fixes, and performance improvements can all be addressed via GPU driver updates. Some GPUs run specific games better than others, and developers can choose to spend more time optimizing for Nvidia cards or AMD cards, but generally the model is hardware agnostic.
Similarly, Intel is already offering GPU-style driver updates for its NPUs. And Hallock says that Windows already essentially recognizes the NPU as “a graphics card with no rendering capability.”
AI Eclipse TLDR:
Intel’s Core Ultra chips, which include built-in neural processing units (NPUs), are among the first x86 PC processors to offer AI and machine-learning processing capabilities. While PCs made and sold in 2024 and beyond will generally include these capabilities, the inclusion of NPUs in mainstream PCs will be new for many users. Currently, software support for NPUs is limited, with Windows only able to utilize NPUs for certain tasks like webcam background blurring. However, Intel and Microsoft are working to make it easier for developers to create NPU-agnostic apps, similar to how PC game developers create GPU-agnostic games. Microsoft’s API for machine-learning operations, DirectML, is being expanded to include support for Intel’s Meteor Lake NPUs, allowing third-party apps to take advantage of NPUs for AI workloads. Intel’s NPU is primarily used for power efficiency, with features like camera segmentation and background blurring consuming 30 to 50 percent less power when offloaded to the NPU. Intel and Microsoft aim to treat NPUs like GPUs, allowing for driver updates and hardware-agnostic development.