Vercel Labs has taken over agent-desktop, a free command-line tool that lets AI agents read and operate desktop apps on macOS, Windows and Linux through the operating system's accessibility tree rather than screenshots. Its author, the developer who goes by Lahfir and now lists Vercel as his employer, introduced it on X at 7:48 p.m. ET on October 30, and the code is under the Apache-2.0 license.
The tool is not new, though MarkTechPost reported on October 31 that Vercel Labs had open-sourced it. The first commit is dated February 19, 2026, the npm package has been public under Apache-2.0 since February 23, and the Wayback Machine captured the repository under Lahfir's personal account on February 21. What changed this week is the owner: a pull request merged late on October 29 prepared the repository for "the org transfer" to vercel-labs, two days after version 1.0.0 shipped on October 27.
Post on X by @mdlahfir
Introducing agent-desktop Computer use for your agents ▲ E2E test desktop apps with agents ▲ Reads apps via the accessibility tree ▲ Drills into part of a window to save tokens ▲ Runs many agents on one app at once ▲ Works on macOS, Windows & Linux npm i -g agent-desktop
What agent-desktop does
An agent runs a command such as a snapshot of Finder and gets back JSON: the window, and a reference for every element it can act on, such as a button or a list row. The agent then clicks, types or toggles by that reference and takes another snapshot to check the result. The project's README says the loop works "with no pixel matching", and that actions go through the accessibility interface "without moving the pointer or taking focus" unless a flag allows physical input.
The README lists 60 command names, 56 of them working today, grouped into observing, acting on elements, keyboard and mouse, apps and windows, notifications, clipboard and sessions. For dense apps such as Slack, an agent can take a shallow "skeleton" snapshot and drill into one region, which the project says keeps the cost of each look down. Each release also ships a C library so a program can call the tool directly. The tool does not include a model: Lahfir wrote on X that it "is a tool to interact with the apps," and that users set guardrails through their own model and the agent software around it.
Chris Tate of Vercel Labs, the top contributor to its browser automation tool agent-browser, framed the new tool as that product's desktop counterpart, and his post drew more than 6,000 likes by Saturday afternoon.
Post on X by @ctatedev
Introducing agent-desktop agent-browser, for your whole computer → Computer Use for any coding agent → macOS, Windows and Linux → Runs in bg while you keep working → Agents use your computer in parallel, each with its own cursor
How it compares
Tool | What it is | Runs on |
|---|---|---|
agent-desktop | CLI reading the accessibility tree | Your own computer |
E2B Desktop | Desktop sandbox for any model | E2B's isolated sandboxes |
Claude computer use | API tool working from screenshots | An environment you supply |
Mage-VL-Cowork | 4B model, screen or shell | Sandboxes, per Microsoft |
Sources: the E2B Desktop repository, Anthropic's computer use documentation and Microsoft's model card, as SaasBuzz reported it on October 31. agent-desktop, E2B Desktop and Mage-VL-Cowork are Apache-2.0; Claude's tool is billed under Anthropic's standard tool-use pricing.
The four are different layers. E2B supplies the computer, Anthropic and Microsoft supply the model that looks at it, and agent-desktop is the hands and eyes for whichever model a developer already uses. Microsoft's Mage-VL-Cowork, released on October 30, picks between clicking the screen and running shell commands.
What it means for teams building agents
The case for the accessibility tree is cost and precision. Anthropic's documentation says each screenshot in a computer-use loop costs roughly 1,000 to 1,800 input tokens and that declaring the toolset adds about 4,500 more to a request. A JSON list of named buttons avoids sending images, though SaasBuzz has not measured agent-desktop's own token use, and the tool still depends on apps exposing a usable accessibility tree.
The trade is where it runs. Anthropic's reference implementation runs inside a Docker container "for security and isolation," and E2B's sandboxes are isolated from one another, while agent-desktop drives the machine it is installed on. Its FAQ warns that snapshots, clipboard reads and screenshots "contain the desktop content you asked to inspect" and should be kept private. Lahfir also wrote on X that agents can run the tool inside a Linux desktop in Vercel Sandbox, the company's own sandbox service.
The commitment is less settled than the code. The README carries a "Labs product" badge and Lahfir called it a Labs experiment, while the Vercel Labs page, which separates maintained products from experiments that may see "breaking changes", did not list agent-desktop when SaasBuzz checked at about 1:15 p.m. ET on October 31. The project has shipped 47 GitHub releases in eight months, close to six a month, and had about 3,750 stars, roughly 2.4 times as many as E2B Desktop, which dates from May 2024. SaasBuzz has not tested agent-desktop.
