In brief: Assisting with various tasks is one of the primary proposed functions of generative AIs from companies like Microsoft, OpenAI, Google, and possibly soon Apple. At a recent unveiling of “AI PCs,” Microsoft claimed that its AI, Copilot, can help players in certain video games in real time, but its effectiveness in real-world scenarios remains to be seen.

While revealing its new AI-powered Surface laptops this week, Microsoft shared a demonstration where the device’s Copilot onboard AI offered real-time tips in games like Minecraft and Sea of Thieves. The fully voiced digital assistant appeared to conduct a natural conversation and react to in-game events.

In one video, a Minecraft player asks Copilot how to craft a sword. The AI can see what items are in the player’s inventory and suggests that they still need to collect some wood or stone. Another video shows zombies attacking the player. Copilot quickly shifts to an alarmed tone of voice and advises the player to either find shelter or build a barrier to stay out of their reach.

It isn’t clear which games Copilot will support, but a screenshot from Windows Central suggests it can also help with Sea of Thieves. If the AI must be tailored on a per-game basis, it might debut for specific Microsoft-published titles.

Stressing an emphasis on privacy, Microsoft stated that all of the information Copilot gathers for in-game assistance will remain on the device. The company won’t upload data to the cloud or use it to train AI models.

Real-time guides are somewhat different from other AI-based applications for video games. The most successful gaming-related uses for the technology have been machine-learning-assisted upscaling and frame generation – features that dramatically improve performance. Meanwhile, Nvidia has proposed filling games with AI-controlled characters that can react to a wide range of player input.

Conversely, in-game assistance would presumably be optional, and wouldn’t directly affect a game’s content. However, some might express concern over the feature’s impact on hand-written guides.

Hallucination is another risk to consider. So far, developers have been unsuccessful in stopping AI-based chatbots from confidently stating false information. The recently unveiled AI summaries for Google search were immediately caught hallucinating, so there’s a good chance the same could happen to Copilot’s game assistant.

Microsoft’s Surface Pro Copilot+ PC will begin shipping in mid-June, starting at $999.99. The PC uses Qualcomm’s Snapdragon X Elite processor to run an Arm-based version of Windows. The Hexagon NPU performs AI workloads at 45 TOPs.