Should Microsoft and the PC industry have paid more attention to the GPU during the development of AI and Copilot+ PCs? After a year’s time waiting for Copilot+ PCs (and their newfangled “Neural Processing Units” to take off, I can’t help but wonder.
Microsoft launched the Copilot+ PC initiative on May 20, 2024, and began shipping them on June 18. Since then, Microsoft has supported Copilot+ PCs with a handful of features, rolling them out first for PCs with the Qualcomm Snapdragon chips inside and then later for PCs powered by Intel Core Ultra Series 2 chips and the AMD Ryzen AI 300 processor.
It’s been a sputtery, stuttery start. Qualcomm is essentially blameless, delivering a potent PC processor with most AI capabilities and long battery life. But AMD and Intel shipped processors without NPUs wielding the requisite AI TOPS to qualify as Copilot+ PCs, leaving PC makers to sell the concept of an “AI PC” without much behind it.
Microsoft had a more cohesive plan in place, delivering several Copilot+ features: Live Captions, which can generate closed captions and even translations of pre-recorded video; Paint’s Cocreator AI functionality, Photos’ Super Resolution; semantic Search; Windows Studio Effects and some Click-to-Do features that “know” what you want to do when you right-click a file. Windows Recall, of course, was supposed to be the flagship, then became mired in controversy.
Mark Hachman / IDG
None of those features, with the exception of Recall and Live Captions, require excessive compute power. Windows Studio Effects runs on the NPU, but the features are essentially clones of existing effects like background blurring that can be run on a CPU. Paint’s Cocreator is quite fast, and probably benefits from running on the NPU. But the app is kind of a gimmick, and not all that accurate.
The most powerful AI silicon isn’t the NPU
Yet we’ve known for months that the most powerful AI engine — not the most efficient, but the most powerful — is the GPU…which is why they’re being snapped up in droves. Yes, gamers love them. But it’s clear that a PC or laptop graphics card can outperform an NPU in most AI tasks, at least right now.
I love creator-class notebooks. A laptop like Microsoft’s Surface Laptop Studio 2 might lack the long battery life of a Snapdragon PC — something I certainly missed while on the road! — but it contains a robust Nvidia GeForce RTX 4060 GPU that often sits idle. If I need to edit a photo, I could certainly load up Adobe Photoshop. Otherwise, it sits unused.
Adam Patrick Murray / Foundry
Let’s face it: many of the Copilot+ features feel like small potatoes. Nothing that runs on the PC feels aspirational. There’s no AI equivalent to Crysis that runs on the local PC.
What if Microsoft had simply said, you know, we believe the future is this thing known as an AI PC, which will eventually include NPUs with increasing power. But to give you a taste of what could be, we’re going to allow these AI functions to take advantage of these powerful GPUs already in your laptops and desktop PCs. Yes, it’s going to spin up your GPU, and your PC’s fans; however, future AI PCs will be much more efficient and less power-hungry.
Heck, I could have done the marketing: think of a GPU and an NPU as a gas-guzzling ICE engine, and an NPU as a slick, efficient electric vehicle.
It’s easy for me to suggest that Microsoft should have split its developer resources, coding Copilot+ features for NPUs as well as GPUs. It’s also easy for me to accuse Microsoft of gobbling up that opportunity for itself, routing billions to its own cloud services.
Meanwhile, products like AMD’s Ryzen AI Max are being specifically designed for a new breed of gaming/inferencing PC workstations. Gigabyte describes its AI TOP 500 TRX50 as “purpose-built for local AI development, multimodal fine-tuning, and high-performance gaming,” with an AMD Threadripper Pro and up to 768GB of DRAM. It runs Windows 11. It does not have an NPU.
Can Microsoft save itself from itself?
Like so many others, Microsoft appears to have been caught flat-footed by AI, tip-toeing ahead with incremental improvements while the giants stride by. It may be that Microsoft’s Foundry Local, launched at Build 2025, may be the answer. But it’s absolutely true that if you want to work with AI on a PC or workstation, you choose a third-party solution — not Microsoft.
Microsoft doesn’t have to do everything, of course. Yet while it still myopically focuses in on optimizing for PCs with a few TOPS of AI power, millions more are sitting around waiting for an opportunity to do more.