Computer-Use & GUI Agents
Parent: AI Agent Ecosystems · researched 2026-05-31T23:48:05.019Z· 9 sources · 8 concepts · skill ai-agent-engineering
AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RA
ai-agent-engineering
- AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RAG, iterative retrieval, vector/graph datastores); ai-llm-model-layer (training, fine-tuning, alignment/RLHF, compression, inference serving, transformer/multimodal architecture, model selection, observability); ai-mcp-sdk-prompting (MCP servers/builder, Anthropic SDK, prompt engineering, context engineering, LLM frameworks, tool-search, prompt lookup). Route to the matching sub-hub. [source]
- This hub routes to on-demand reference files under references/. See each spoke for depth. [source]
Children
- Computer-use agents (Claude computer use, OpenAI Operator/CUA) (frontier)
- Browser agents (Browser Use, Playwright/CDP) (frontier)
- GUI grounding (pixel vs accessibility tree vs set-of-marks) (frontier)
- Screen parsers (OmniParser) (frontier)
- GUI action space (click/type/scroll/key) (frontier)
- Computer-use benchmarks (OSWorld, WebArena, WebVoyager, Mind2Web, AndroidWorld) (frontier)
- GUI agent reliability ceilings (frontier)
- Computer-use safety (prompt injection via screen, destructive clicks, sandboxing) (frontier)
Frontier under this node: Browser agents (Browser Use, Playwright/CDP), Computer-use agents (Claude computer use, OpenAI Operator/CUA), Computer-use benchmarks (OSWorld, WebArena, WebVoyager, Mind2Web, AndroidWorld), Computer-use safety (prompt injection via screen, destructive clicks, sandboxing), GUI action space (click/type/scroll/key), GUI agent reliability ceilings, GUI grounding (pixel vs accessibility tree vs set-of-marks), Screen parsers (OmniParser)