Microsoft now gives developers direct access to neural processing units (NPUs) on Windows 11, with a focus on Copilot+ PCs. With updates to the DirectML API and the Windows AI platform, developers can build and run AI models efficiently on NPU hardware from partners such as Qualcomm (Snapdragon X Elite), Intel (Core Ultra), and AMD.
Here are the main highlights of this update:
- DirectML NPU support: DirectML, which is part of the DirectX family, now lets apps use NPUs directly. This means AI workloads can shift from the GPU or CPU to the NPU, improving performance and saving battery life.
- Targeting Copilot plus PCs: These improvements are designed for Copilot plus PCs, which have high-performance NPUs (40+ TOPS) needed to run local AI models like Phi Silica.
- Windows AI APIs and Studio Effects: Developers can build apps that use Windows 11’s built-in AI features like background effects, voice focus, and real-time transcription through standard Windows APIs
- ONNX Runtime integration: The ONNX Runtime now supports NPU acceleration, making it easier to migrate existing models from GPU to NPU with only minor changes.
- Microsoft Foundry on Windows: Previously called Windows AI Foundry, this updated platform now supports the full AI lifecycle from selection to optimization and deployment on GPU, CPU, and NPU.
With these improvements, on-device generative AI is now faster, more private, and uses less energy. For example, you can run local language models like Phi 3.5 directly on Snapdragon X-powered devices.
We’re excited to work with Intel, one of our main partners, to launch the first neural processing unit (NPU) powered by DirectML on Windows. AI is changing the world, powering innovation and creating value in many industries. NPUs are vital for delivering great AI experiences to both developers and consumers.
An NPU is a processor designed for machine learning (ML) tasks that require substantial computing power but don’t require graphics. NPUs also use power efficiently. These new devices will change how AI improves our daily lives. Early next year, we’ll release DirectML for support for Intel Core Ultra processors with Intel AI Boost, the new built-in NPU.
DirectML is a basic API that provides direct access to hardware features of modern devices, such as GPUs, for machine learning tasks. It is part of the DirectX family, the Windows graphics and gaming platform, and works with other DirectX components, such as DirectX 12. DirectML also connects with popular machine learning tools such as the ONNX Runtime and Olive, making it easier to develop and deploy AI across Windows.
Adding NPU support to DirectML opens up new possibilities for AI on Windows. DirectML with NPU support will be available as a developer preview in early 2024, along with the latest ONNX Runtime release. We’ll share additional updates shortly about new features, partners, and how to use DirectML for NPUs.
We can’t wait to see the amazing AI experiences you will create on Windows using direct tunnels and Intel Core Ultra processors.
