Skip to content

About

Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

Topics

Resources

Contributing

Security policy

Stars

128 stars

Watchers

3 watching

Forks

Repository files navigation

Featured by Awesome CUA

Awesome Computer-Use Agents

A source-backed collection of papers, models, datasets, benchmarks and tools for agents that use computers.

English · 简体中文

Awesome CUA — computer-use agent research, data and tools connected to original sources

Search & filter · Research papers · Browse topics · Data & agents

Read the research survey — Methods, evaluation, human agency, and reusable research assets. English working manuscript with a versioned bibliography and reading register.

Catalog records Source-reviewed records Catalog checks MIT license

Find by task · Explore · Highlights · Recently reviewed · Use the data

Coverage ledger · Contribute · GitHub organization

Discovery is scheduled three times daily at 00:17, 08:17 and 16:17 UTC. Actual scan results · Schedule and safeguards.

For people, RAG systems, and agents — clone the complete catalog:

git clone https://github.com/reacher-z/Awesome-CUA.git

Agent instructions · RAG and agent guide · Complete JSONL · Retrieval documents

The goal is comprehensive coverage of public research and industry resources, with provenance that can be inspected. Papers, models, training data, evaluation environments, frameworks, infrastructure, products, and publishers belong in the same searchable catalog.

Global completeness is a goal, not a verified claim. We publish the source populations, review states, and remaining gaps. Bibliographic metadata verification is different from reviewing a resource's relevance or quality.

Browse by topic · Topic subsets and fingerprints

Find by task

Your task Start here What you can find
Find papers Research library · Topics Reviewed papers by year and topic, with original sources.
Choose a benchmark Benchmark guide Platforms, tasks, evaluators, and versions to compare.
Find training data Training-data guide Supervision, splits, count units, and data licenses.
Build an agent Agents · Frameworks Implementation entry points, supported platforms, and papers.
Use RAG or an agent Agent guide · Data hub Complete records or retrieval documents with review states and sources.
Trace papers to assets Paper–asset connections Code, datasets, and models linked through reviewed primary evidence.
Local examples (after cloning; Python 3.11+, no dependencies required)
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind paper --status curated --limit 0 --json
python3 scripts/catalog.py search --kind benchmark --environment web --status curated --limit 0 --json
python3 scripts/catalog.py search --kind dataset --environment android --status curated --limit 0 --json
python3 scripts/catalog.py search --kind framework --status curated --limit 0 --json
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/catalog.py get arxiv:2404.07972 --related --json

3,260 resource records · 1,671 curated · 2,789 paper and survey records · 54 discovery documents

Evidence through 2026-10-24T04:38:00Z. 1,589 records have bibliographic verification only; 907 discovered identifiers remain unresolved, not counting 136 links reviewed as non-resources or other addresses of indexed resources. 138 scope exclusions retain their original metadata in the archive. Counts overlap as explained below.

Explore

Research and training Engineering and industry
Research by topic · 1,181 reviewed papers · Full bibliography · 2,789 paper and survey records Agents · 45
Surveys · 19 · Learning resources · 2 Frameworks · 79
Datasets · 86 Infrastructure · 14
Benchmarks · 89 Tools · 22
Models and checkpoint releases · 98 Products · 20 · Publishers and brands · 16

By platform: Web 646 · Desktop 334 · Windows 65 · macOS 30 · Linux 54 · Android 323 · iOS 22 · HarmonyOS 3 · Cross-platform 197

Coverage spans web, desktop, Android, iOS, Windows, macOS, Linux, and cross-platform work. Topics include grounding, screen understanding, planning, memory, long-horizon workflows, reinforcement learning, data generation, safety, robustness, and evaluation. Environment facets overlap; framework support is not assumed from a model's benchmark results.

Highlights

The most-mentioned curated resources per category: the count is how many other cataloged resources mention each one in their reviewed first-party source (README, model or dataset card). It reflects connectedness inside this catalog, not quality, popularity, or benchmark performance. Every category page lists all reviewed entries. Surveys come first, newest first.

Start with the newest surveys · all 19 →

Date Survey Environments
2026‑10‑14 Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory unspecified
2026‑09‑02 Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization cross-platform
2026‑08‑30 A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security cross-platform
2026‑04‑17 A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle unspecified
2026‑02‑12 Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward unspecified

Benchmarks · all 89 →

Resource Mentions Environments Paper License Description
OSWorld 21 desktop, linux, windows, macos 2404.07972 (2024) code Apache-2.0 Execution-based evaluation of agents performing tasks in real desktop environments.
WebArena 15 web 2307.13854 (2023) code Apache-2.0 Self-hosted web environments and tasks for evaluating autonomous web agents.
BrowserGym 8 web 2412.05467 (2024) code Apache-2.0 Unified interfaces for browser-agent tasks and benchmark environments.
ScreenSpot Pro 8 desktop, windows, macos, linux — code MIT GUI grounding benchmark focused on professional high-resolution interfaces.
VisualWebArena 7 web 2401.13649 (2024) — Visually grounded web tasks extending the WebArena evaluation setting.
AndroidWorld 6 android 2405.14573 (2024) code Apache-2.0 Android environment with programmatic task evaluation for mobile agents.

Models · all 98 →

Resource Mentions Environments Paper License Description
SeeClick 9 cross-platform 2401.10935 (2024) code Apache-2.0 GUI grounding model trained to locate visual interface elements.
Aguvis 5 cross-platform 2412.04454 (2024) — Vision-based GUI agent models with unified action representations.
UI-TARS 5 cross-platform 2501.12326 (2025) code Apache-2.0 Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use.
OpenCUA 3 desktop 2508.09123 (2025) code MIT Open computer-use research stack including models and training resources.
OS-Atlas 3 cross-platform 2410.23218 (2024) code Apache-2.0 OS-Atlas GUI action models, including grounding bases fine-tuned from InternVL2-4B and Qwen2-VL-7B-Instruct that accept any image size and output 0-1000 relative coordinates.
ShowUI 3 cross-platform 2411.17465 (2024) code Apache-2.0 Lightweight GUI-agent vision-language-action model from Show Lab and Microsoft, released with UI-guided token selection, training code, the ShowUI-web and ShowUI-desktop datasets and vLLM inference.

Datasets · all 86 →

Resource Mentions Environments Paper License Description
ScreenSpot-Pro data 4 desktop, windows, macos, linux 2504.07981 (2025) data mit High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization.
GUI-Odyssey 3 android 2406.08451 (2024) — Human-demonstrated Android episodes for cross-app navigation, spanning six task types, 212 apps and about 1,400 app combinations, for training and evaluating agents.
GUIAct 3 web, android 2406.11317 (2024) — Web and Android GUI action-instruction data, with single-step, multi-step and smartphone partitions.
GUIChat 3 unspecified 2406.11317 (2024) — Conversational GUI supervision with text-rich images and bounding boxes, organized as single-turn questions and multi-turn dialogues.
GUIEnv-local 3 web 2406.11317 (2024) — The released local-supervision portion of GUIEnv, pairing webpage regions and text for OCR and GUI grounding.
OpenMobile-Data-v2 3 android, web 2610.14157 (2026) — Mobile-agent supervision from emulator and simulator trajectories, plus executable tasks with starting states and success checks for reinforcement learning.

Frameworks · all 79 →

Resource Mentions Environments Paper License Description
Browser Use 7 web — code MIT Open-source browser agent library that runs locally with any LLM against a local or cloud browser, and also serves as a CLI for coding agents and as a driver for Claude's browser toolset.
AgentLab 5 web 2412.05467 (2024) code Apache-2.0 Experiment framework for developing and evaluating browser agents.
Seg2Click 5 cross-platform 2610.16234 (2026) code MIT Training and inference implementation that predicts an executable-region mask and derives a GUI click from that mask.
OpenMobile-2 code 3 android, web 2610.14157 (2026) code Apache-2.0 Evaluation adapters and training guidance for OpenMobile-2 across Android emulators and browser-rendered mobile simulation.
UI-Genie 3 android 2505.21496 (2025) — Training and evaluation code for a mobile-agent framework that iteratively generates trajectories using agent and reward-model feedback.
Computer Use OOTB 2 desktop 2411.10323 (2024) code Apache-2.0 Gradio desktop GUI agent for Windows and macOS that needs no Docker, using Claude 3.5 Computer Use over the API or ShowUI and UI-TARS run locally, with multi-display and phone remote control.

Agents · all 45 →

Resource Mentions Environments Paper License Description
Mobile-Agent 9 android, cross-platform 2401.16158 (2024), 2406.01014 (2024), 2508.15144 (2025), 2602.16855 (2026) code MIT Mobile device agent family using visual interface interaction.
Agent S 2 desktop 2410.08164 (2024), 2504.00906 (2025) code Apache-2.0 Simular's open-source framework for computer-use agents that click, type and scroll through desktop and web apps on macOS, Windows and Linux; its third generation, Agent S3, reports 72.6% on OSWorld.
SeeAct 2 web 2401.01614 (2024) — Generalist web agent framework for large multimodal models such as GPT-4V, with a Playwright tool that runs agents on live websites, text-choice and set-of-mark grounding, and a crawler mode.
AppAgent 1 android 2312.13771 (2023) code MIT Multimodal agent that operates Android apps through taps and swipes without back-end access, building a reusable knowledge base from autonomous exploration or from watching human demonstrations.
Auto-GUI / Auto-UI 1 android 2309.11436 (2023) code Apache-2.0 Screenshot-based mobile agent implementation with training and inference code for chain-of-action prediction.
MagenticLite / Magentic UI 1 web — code MIT Microsoft agent application combining orchestration with browser interaction; successor to Magentic UI.

Recently reviewed

Newest editorial approvals of models, datasets, benchmarks, systems, and tools (newest first). New papers appear by date in the topic hubs.

Reviewed Resource Kind Environments Description
2026‑10‑20 Web Browser Automation with Agents (smolagents tutorial) Learning resources web Versioned tutorial building a vision-guided CodeAgent with Selenium/Helium browser tools, screenshot callbacks and example web-navigation tasks.
2026‑10‑20 ORBIT framework framework web, desktop Inspect-based multi-agent evaluation framework with role, scheduling and communication controls, BrowserART accessibility-tree actions, and desktop mouse, keyboard and screenshot tools.
2026‑10‑20 Building AI Browser Agents Learning resources web Browser-agent course with worked lessons on structured web extraction and multi-step browser interaction, followed by AgentQ search and self-correction concepts.
2026‑10‑20 Yunkai agent android Android phone assistant combining model-driven tool use with accessibility-tree reading, screenshots, and tap, swipe and text-entry primitives. Its documented interaction flow presents operation plans for user confirmation.
2026‑10‑20 WinDOM code framework web GUI-grounding training and evaluation code with same-family distillation, GRPO recipes and point-in-box scoring, using a Windows-style web environment.
2026‑10‑20 Web Context Bench (Python) benchmark web Local-page benchmark comparing six browser representations on token use, factual reading and target-element selection, with deterministic answer scoring.
2026‑10‑20 Mobster engine agent ios iOS agent engine organized around an observe, decide, verify and act loop, using WebDriverAgent screen state and phone actions such as taps, typing, swipes and app switches.
2026‑10‑20 GUI Grounding Evaluation Harness tool unspecified ScreenSpot-Pro evaluation harness with explicit coordinate transforms, prompt and crop-based inference comparisons, and per-sample grounding accuracy and cost records.
2026‑10‑20 Activity Recorder tool desktop, windows Windows tray utility that records primary-display screenshots, keyboard key codes and mouse events as local JPEG and JSON files for training-data collection.
2026‑10‑20 MobileForge framework android Android GUI-agent adaptation framework combining app exploration, generated curricula and rollout feedback with hint-conditioned step-level GRPO; the documented public training path targets Qwen3-VL-8B.
2026‑10‑20 MemGUI-Agent framework android MemGUI-Agent training and offline evaluation tools provide trajectory conversion, LoRA SFT templates, and step-level checks of GUI actions, memory updates and context folding.
2026‑10‑18 VideoCAD data dataset web CAD screen recordings paired with action logs and target model images for studying precise interface control and long-horizon visual imitation.

Use the data

Browse the interactive website to search and filter all resources, inspect source evidence, switch between English and Chinese, or download the complete catalog. Website development and publishing.

The English data hub and 中文数据中心 organize public downloads into six groups: complete catalog, RAG documents, citations, topics and comparisons, sources and review status, and schemas. Cloning still gives you the complete source catalog; the hub provides exports from its published snapshot.

The download manifest records each listed export's actual count and unit, scope, byte size, and SHA-256 fingerprint. These are different formats and overlapping views of the catalog, so their counts must not be added together. Compare catalog_sha256 across manifests before combining files from different downloads.

Reviewed research · Computer-use agent guides · Static resource directory · Search and AI discovery

  • Catalog JSONL: one resource per line, with stable IDs, typed fields, provenance, scoped licenses, and review status.
  • Reference graph: outbound references from primary sources. mentions does not mean trained_on or evaluated_on.
  • Discovery registry and candidates: source snapshots and where each identifier was found.
  • Review queue: unresolved evidence, unindexed discoveries, and proposals requiring editorial work. Discovered links reviewed as non-resources or as other addresses of indexed resources stay listed with a resolution and their reason.
  • JSON Schema and coverage JSON: machine-checkable contracts and coverage accounting.
  • BibTeX and CSL JSON: reviewed-paper citations with recorded authors, dates, and versions. Each resource page also provides its own record.json.
# Python 3.11+. Search and export require only the standard library.
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind model
python3 scripts/catalog.py search --kind dataset --environment android
python3 scripts/catalog.py search --kind product --status curated --json
python3 scripts/catalog.py get arxiv:2404.07972 --related --json
python3 scripts/catalog.py build
# Creates dist/catalog.sqlite and dist/catalog.json, with no network calls.
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/export_citations.py --format bibtex --output dist/citations.bib
# Add --include-metadata to include every active record, with its review state intact.

Use SQLite to join resources, environment/topic facets, and reviewed relations. See query examples. data/catalog.jsonl is committed; SQLite and JSON array exports are reproducible local build products.

What “verified” means

State What has been checked What has not been established
curated AI-assisted editorial review against a first-party source snapshot, with name, scope, summary, and taxonomy Independent reproduction, universal availability, or endorsement
metadata_verified Bibliographic identity, title, authors, and recorded dates checked against retained arXiv or publication sources Topic relevance, methodology, findings, or inclusion in the core field
candidate A normalized URL or identifier was discovered Inclusion, correctness, or availability; excluded from the published catalog

Counts refer to resources, not unique projects. A paper, repository, dataset release, model checkpoint, product, and publisher can be separate related resources. Model families and individual checkpoints are explicitly linked where reviewed.

Code, data, weights, and document licenses are separate. Missing values are null. Dataset archive counts are not converted into training-example counts. Scores from different benchmark versions, splits, action budgets, or evaluators are not combined into a ranking.

Maintain and verify

python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-dev.txt
.venv/bin/python scripts/catalog.py validate
.venv/bin/python scripts/catalog.py build --check
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/catalog.py build
node --test tests/web.test.mjs
.venv/bin/python scripts/build_site.py

These commands match the checks that Catalog quality runs on every push and pull request, in the same order. They need Python 3.11+ and Node.js 24+, the engines version in package.json (CI uses Python 3.12 and Node.js 24). The web tests use only the built-in Node.js test runner, so nothing needs to be installed with npm. npm run test:web and npm run build:web are shortcuts for the last two commands.

Network discovery and verification are explicit commands; ordinary builds are offline. Changed primary snapshots require another editorial review before publication. Maintenance guide · Methodology · Data dictionary

Acknowledgments and reuse

The source registry credits the community collections and surveys used for discovery. Bibliographic records use retained arXiv, publisher, and author sources; publication observations preserve edition-specific titles, authors, and date precision. Selected assets are checked against publisher repositories, documentation, and model/dataset cards. Upstream READMEs, papers, model weights, and datasets are not redistributed here.

Original repository code and annotations use the MIT license. Linked resources retain their own terms; this repository does not relicense them.

To cite the catalog, use CITATION.cff or GitHub's "Cite this repository" button. The data changes with every scan, so pin the exact version you used. Each v* tag has a data release with versioned downloads built from the tagged commit: the catalog as JSONL, JSON and SQLite, the coverage ledger, topic and profile exports, and their SHA256SUMS. Cite the release tag, or record the commit from git rev-parse HEAD in your clone, or link files at that commit, such as https://github.com/reacher-z/Awesome-CUA/blob/<commit>/data/catalog.jsonl. Cutting a release.

About

Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

Topics

Resources

Contributing

Security policy

Stars

128 stars

Watchers

3 watching

Forks

Releases

Packages

Contributors

Languages