Google Breaks AI Out of the Chat Window

The mouse pointer hasn’t changed much since 1984. Google DeepMind thinks that’s the problem.

For most of computing history, the pointer has tracked where users look. Google now wants it to understand what they see. The company calls the concept AI Pointer, a cursor that reads on-screen content and understands why it matters to the user, without any manual prompting. Google has already built experimental demos into Google AI Studio.

The Problem With the Prompt Box

Current artificial intelligence tools require users to do the translation work themselves: open the chatbot, describe the document, paste the relevant section, ask the question. Each step is a break in workflow. DeepMind’s framing is blunt: Today’s AI tools make users drag their world into them. AI Pointer flips the model by capturing visual and semantic context directly from wherever the pointer sits, across documents, websites and applications, so users can point at a table of figures and say “make this a chart” without writing a line of instructions. DeepMind said the goal is to let users work in natural shorthand, the way people speak to each other: “fix this,” “move that,” “what does this mean?” with the AI reading the rest from context.

Where It’s Going in Products

Google is already moving AI Pointer into two products. Gemini in Chrome now lets users point at any part of a webpage and ask questions directly, without switching to a separate AI panel. The company’s new Googlebook laptop will ship a version called Magic Pointer, letting users point and speak to Gemini anywhere on screen.

The Chrome deployment sits on top of a broader enterprise push. TechCrunch reported that Google introduced Auto Browse at Cloud Next 2026, giving enterprise Chrome users an agentic capability that reads open browser tabs and handles tasks like booking travel, inputting CRM data and comparing vendor pricing across tabs. Workflows still require human confirmation before final actions…

Source