A source-backed collection of papers, models, datasets, benchmarks and tools for agents that use computers.
English · 简体中文
Search & filter · Research papers · Browse topics · Data & agents
Read the research survey — Methods, evaluation, human agency, and reusable research assets. English working manuscript with a versioned bibliography and reading register.
Find by task · Explore · Highlights · Recently reviewed · Use the data
Coverage ledger · Contribute · GitHub organization
Discovery is scheduled three times daily at 00:17, 08:17 and 16:17 UTC. Actual scan results · Schedule and safeguards.
For people, RAG systems, and agents — clone the complete catalog:
git clone https://github.com/reacher-z/Awesome-CUA.gitAgent instructions · RAG and agent guide · Complete JSONL · Retrieval documents
The goal is comprehensive coverage of public research and industry resources, with provenance that can be inspected. Papers, models, training data, evaluation environments, frameworks, infrastructure, products, and publishers belong in the same searchable catalog.
Global completeness is a goal, not a verified claim. We publish the source populations, review states, and remaining gaps. Bibliographic metadata verification is different from reviewing a resource's relevance or quality.
Browse by topic · Topic subsets and fingerprints
| Your task | Start here | What you can find |
|---|---|---|
| Find papers | Research library · Topics | Reviewed papers by year and topic, with original sources. |
| Choose a benchmark | Benchmark guide | Platforms, tasks, evaluators, and versions to compare. |
| Find training data | Training-data guide | Supervision, splits, count units, and data licenses. |
| Build an agent | Agents · Frameworks | Implementation entry points, supported platforms, and papers. |
| Use RAG or an agent | Agent guide · Data hub | Complete records or retrieval documents with review states and sources. |
| Trace papers to assets | Paper–asset connections | Code, datasets, and models linked through reviewed primary evidence. |
Local examples (after cloning; Python 3.11+, no dependencies required)
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind paper --status curated --limit 0 --json
python3 scripts/catalog.py search --kind benchmark --environment web --status curated --limit 0 --json
python3 scripts/catalog.py search --kind dataset --environment android --status curated --limit 0 --json
python3 scripts/catalog.py search --kind framework --status curated --limit 0 --json
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/catalog.py get arxiv:2404.07972 --related --json3,260 resource records · 1,671 curated · 2,789 paper and survey records · 54 discovery documents
Evidence through 2026-10-24T04:38:00Z. 1,589 records have bibliographic verification only; 907 discovered identifiers remain unresolved, not counting 136 links reviewed as non-resources or other addresses of indexed resources. 138 scope exclusions retain their original metadata in the archive. Counts overlap as explained below.
| Research and training | Engineering and industry |
|---|---|
| Research by topic · 1,181 reviewed papers · Full bibliography · 2,789 paper and survey records | Agents · 45 |
| Surveys · 19 · Learning resources · 2 | Frameworks · 79 |
| Datasets · 86 | Infrastructure · 14 |
| Benchmarks · 89 | Tools · 22 |
| Models and checkpoint releases · 98 | Products · 20 · Publishers and brands · 16 |
By platform: Web 646 · Desktop 334 · Windows 65 · macOS 30 · Linux 54 · Android 323 · iOS 22 · HarmonyOS 3 · Cross-platform 197
Coverage spans web, desktop, Android, iOS, Windows, macOS, Linux, and cross-platform work. Topics include grounding, screen understanding, planning, memory, long-horizon workflows, reinforcement learning, data generation, safety, robustness, and evaluation. Environment facets overlap; framework support is not assumed from a model's benchmark results.
The most-mentioned curated resources per category: the count is how many other cataloged resources mention each one in their reviewed first-party source (README, model or dataset card). It reflects connectedness inside this catalog, not quality, popularity, or benchmark performance. Every category page lists all reviewed entries. Surveys come first, newest first.
Start with the newest surveys · all 19 →
| Date | Survey | Environments |
|---|---|---|
| 2026‑10‑14 | Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory | unspecified |
| 2026‑09‑02 | Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization | cross-platform |
| 2026‑08‑30 | A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security | cross-platform |
| 2026‑04‑17 | A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle | unspecified |
| 2026‑02‑12 | Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward | unspecified |
Benchmarks · all 89 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| OSWorld | 21 | desktop, linux, windows, macos | 2404.07972 (2024) | code Apache-2.0 | Execution-based evaluation of agents performing tasks in real desktop environments. |
| WebArena | 15 | web | 2307.13854 (2023) | code Apache-2.0 | Self-hosted web environments and tasks for evaluating autonomous web agents. |
| BrowserGym | 8 | web | 2412.05467 (2024) | code Apache-2.0 | Unified interfaces for browser-agent tasks and benchmark environments. |
| ScreenSpot Pro | 8 | desktop, windows, macos, linux | — | code MIT | GUI grounding benchmark focused on professional high-resolution interfaces. |
| VisualWebArena | 7 | web | 2401.13649 (2024) | — | Visually grounded web tasks extending the WebArena evaluation setting. |
| AndroidWorld | 6 | android | 2405.14573 (2024) | code Apache-2.0 | Android environment with programmatic task evaluation for mobile agents. |
Models · all 98 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| SeeClick | 9 | cross-platform | 2401.10935 (2024) | code Apache-2.0 | GUI grounding model trained to locate visual interface elements. |
| Aguvis | 5 | cross-platform | 2412.04454 (2024) | — | Vision-based GUI agent models with unified action representations. |
| UI-TARS | 5 | cross-platform | 2501.12326 (2025) | code Apache-2.0 | Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use. |
| OpenCUA | 3 | desktop | 2508.09123 (2025) | code MIT | Open computer-use research stack including models and training resources. |
| OS-Atlas | 3 | cross-platform | 2410.23218 (2024) | code Apache-2.0 | OS-Atlas GUI action models, including grounding bases fine-tuned from InternVL2-4B and Qwen2-VL-7B-Instruct that accept any image size and output 0-1000 relative coordinates. |
| ShowUI | 3 | cross-platform | 2411.17465 (2024) | code Apache-2.0 | Lightweight GUI-agent vision-language-action model from Show Lab and Microsoft, released with UI-guided token selection, training code, the ShowUI-web and ShowUI-desktop datasets and vLLM inference. |
Datasets · all 86 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| ScreenSpot-Pro data | 4 | desktop, windows, macos, linux | 2504.07981 (2025) | data mit | High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization. |
| GUI-Odyssey | 3 | android | 2406.08451 (2024) | — | Human-demonstrated Android episodes for cross-app navigation, spanning six task types, 212 apps and about 1,400 app combinations, for training and evaluating agents. |
| GUIAct | 3 | web, android | 2406.11317 (2024) | — | Web and Android GUI action-instruction data, with single-step, multi-step and smartphone partitions. |
| GUIChat | 3 | unspecified | 2406.11317 (2024) | — | Conversational GUI supervision with text-rich images and bounding boxes, organized as single-turn questions and multi-turn dialogues. |
| GUIEnv-local | 3 | web | 2406.11317 (2024) | — | The released local-supervision portion of GUIEnv, pairing webpage regions and text for OCR and GUI grounding. |
| OpenMobile-Data-v2 | 3 | android, web | 2610.14157 (2026) | — | Mobile-agent supervision from emulator and simulator trajectories, plus executable tasks with starting states and success checks for reinforcement learning. |
Frameworks · all 79 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| Browser Use | 7 | web | — | code MIT | Open-source browser agent library that runs locally with any LLM against a local or cloud browser, and also serves as a CLI for coding agents and as a driver for Claude's browser toolset. |
| AgentLab | 5 | web | 2412.05467 (2024) | code Apache-2.0 | Experiment framework for developing and evaluating browser agents. |
| Seg2Click | 5 | cross-platform | 2610.16234 (2026) | code MIT | Training and inference implementation that predicts an executable-region mask and derives a GUI click from that mask. |
| OpenMobile-2 code | 3 | android, web | 2610.14157 (2026) | code Apache-2.0 | Evaluation adapters and training guidance for OpenMobile-2 across Android emulators and browser-rendered mobile simulation. |
| UI-Genie | 3 | android | 2505.21496 (2025) | — | Training and evaluation code for a mobile-agent framework that iteratively generates trajectories using agent and reward-model feedback. |
| Computer Use OOTB | 2 | desktop | 2411.10323 (2024) | code Apache-2.0 | Gradio desktop GUI agent for Windows and macOS that needs no Docker, using Claude 3.5 Computer Use over the API or ShowUI and UI-TARS run locally, with multi-display and phone remote control. |
Agents · all 45 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| Mobile-Agent | 9 | android, cross-platform | 2401.16158 (2024), 2406.01014 (2024), 2508.15144 (2025), 2602.16855 (2026) | code MIT | Mobile device agent family using visual interface interaction. |
| Agent S | 2 | desktop | 2410.08164 (2024), 2504.00906 (2025) | code Apache-2.0 | Simular's open-source framework for computer-use agents that click, type and scroll through desktop and web apps on macOS, Windows and Linux; its third generation, Agent S3, reports 72.6% on OSWorld. |
| SeeAct | 2 | web | 2401.01614 (2024) | — | Generalist web agent framework for large multimodal models such as GPT-4V, with a Playwright tool that runs agents on live websites, text-choice and set-of-mark grounding, and a crawler mode. |
| AppAgent | 1 | android | 2312.13771 (2023) | code MIT | Multimodal agent that operates Android apps through taps and swipes without back-end access, building a reusable knowledge base from autonomous exploration or from watching human demonstrations. |
| Auto-GUI / Auto-UI | 1 | android | 2309.11436 (2023) | code Apache-2.0 | Screenshot-based mobile agent implementation with training and inference code for chain-of-action prediction. |
| MagenticLite / Magentic UI | 1 | web | — | code MIT | Microsoft agent application combining orchestration with browser interaction; successor to Magentic UI. |
Newest editorial approvals of models, datasets, benchmarks, systems, and tools (newest first). New papers appear by date in the topic hubs.
| Reviewed | Resource | Kind | Environments | Description |
|---|---|---|---|---|
| 2026‑10‑20 | Web Browser Automation with Agents (smolagents tutorial) | Learning resources | web | Versioned tutorial building a vision-guided CodeAgent with Selenium/Helium browser tools, screenshot callbacks and example web-navigation tasks. |
| 2026‑10‑20 | ORBIT framework | framework | web, desktop | Inspect-based multi-agent evaluation framework with role, scheduling and communication controls, BrowserART accessibility-tree actions, and desktop mouse, keyboard and screenshot tools. |
| 2026‑10‑20 | Building AI Browser Agents | Learning resources | web | Browser-agent course with worked lessons on structured web extraction and multi-step browser interaction, followed by AgentQ search and self-correction concepts. |
| 2026‑10‑20 | Yunkai | agent | android | Android phone assistant combining model-driven tool use with accessibility-tree reading, screenshots, and tap, swipe and text-entry primitives. Its documented interaction flow presents operation plans for user confirmation. |
| 2026‑10‑20 | WinDOM code | framework | web | GUI-grounding training and evaluation code with same-family distillation, GRPO recipes and point-in-box scoring, using a Windows-style web environment. |
| 2026‑10‑20 | Web Context Bench (Python) | benchmark | web | Local-page benchmark comparing six browser representations on token use, factual reading and target-element selection, with deterministic answer scoring. |
| 2026‑10‑20 | Mobster engine | agent | ios | iOS agent engine organized around an observe, decide, verify and act loop, using WebDriverAgent screen state and phone actions such as taps, typing, swipes and app switches. |
| 2026‑10‑20 | GUI Grounding Evaluation Harness | tool | unspecified | ScreenSpot-Pro evaluation harness with explicit coordinate transforms, prompt and crop-based inference comparisons, and per-sample grounding accuracy and cost records. |
| 2026‑10‑20 | Activity Recorder | tool | desktop, windows | Windows tray utility that records primary-display screenshots, keyboard key codes and mouse events as local JPEG and JSON files for training-data collection. |
| 2026‑10‑20 | MobileForge | framework | android | Android GUI-agent adaptation framework combining app exploration, generated curricula and rollout feedback with hint-conditioned step-level GRPO; the documented public training path targets Qwen3-VL-8B. |
| 2026‑10‑20 | MemGUI-Agent | framework | android | MemGUI-Agent training and offline evaluation tools provide trajectory conversion, LoRA SFT templates, and step-level checks of GUI actions, memory updates and context folding. |
| 2026‑10‑18 | VideoCAD data | dataset | web | CAD screen recordings paired with action logs and target model images for studying precise interface control and long-horizon visual imitation. |
Browse the interactive website to search and filter all resources, inspect source evidence, switch between English and Chinese, or download the complete catalog. Website development and publishing.
The English data hub and 中文数据中心 organize public downloads into six groups: complete catalog, RAG documents, citations, topics and comparisons, sources and review status, and schemas. Cloning still gives you the complete source catalog; the hub provides exports from its published snapshot.
The download manifest records each listed export's actual count and unit, scope, byte size, and SHA-256 fingerprint. These are different formats and overlapping views of the catalog, so their counts must not be added together. Compare catalog_sha256 across manifests before combining files from different downloads.
Reviewed research · Computer-use agent guides · Static resource directory · Search and AI discovery
- Catalog JSONL: one resource per line, with stable IDs, typed fields, provenance, scoped licenses, and review status.
- Reference graph: outbound references from primary sources.
mentionsdoes not meantrained_onorevaluated_on. - Discovery registry and candidates: source snapshots and where each identifier was found.
- Review queue: unresolved evidence, unindexed discoveries, and proposals requiring editorial work. Discovered links reviewed as non-resources or as other addresses of indexed resources stay listed with a
resolutionand their reason. - JSON Schema and coverage JSON: machine-checkable contracts and coverage accounting.
- BibTeX and CSL JSON: reviewed-paper citations with recorded authors, dates, and versions. Each resource page also provides its own
record.json.
# Python 3.11+. Search and export require only the standard library.
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind model
python3 scripts/catalog.py search --kind dataset --environment android
python3 scripts/catalog.py search --kind product --status curated --json
python3 scripts/catalog.py get arxiv:2404.07972 --related --json
python3 scripts/catalog.py build
# Creates dist/catalog.sqlite and dist/catalog.json, with no network calls.
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/export_citations.py --format bibtex --output dist/citations.bib
# Add --include-metadata to include every active record, with its review state intact.Use SQLite to join resources, environment/topic facets, and reviewed relations. See query examples. data/catalog.jsonl is committed; SQLite and JSON array exports are reproducible local build products.
| State | What has been checked | What has not been established |
|---|---|---|
curated |
AI-assisted editorial review against a first-party source snapshot, with name, scope, summary, and taxonomy | Independent reproduction, universal availability, or endorsement |
metadata_verified |
Bibliographic identity, title, authors, and recorded dates checked against retained arXiv or publication sources | Topic relevance, methodology, findings, or inclusion in the core field |
candidate |
A normalized URL or identifier was discovered | Inclusion, correctness, or availability; excluded from the published catalog |
Counts refer to resources, not unique projects. A paper, repository, dataset release, model checkpoint, product, and publisher can be separate related resources. Model families and individual checkpoints are explicitly linked where reviewed.
Code, data, weights, and document licenses are separate. Missing values are null. Dataset archive counts are not converted into training-example counts. Scores from different benchmark versions, splits, action budgets, or evaluators are not combined into a ranking.
python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-dev.txt
.venv/bin/python scripts/catalog.py validate
.venv/bin/python scripts/catalog.py build --check
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/catalog.py build
node --test tests/web.test.mjs
.venv/bin/python scripts/build_site.pyThese commands match the checks that Catalog quality runs on every push and pull request, in the same order. They need Python 3.11+ and Node.js 24+, the engines version in package.json (CI uses Python 3.12 and Node.js 24). The web tests use only the built-in Node.js test runner, so nothing needs to be installed with npm. npm run test:web and npm run build:web are shortcuts for the last two commands.
Network discovery and verification are explicit commands; ordinary builds are offline. Changed primary snapshots require another editorial review before publication. Maintenance guide · Methodology · Data dictionary
The source registry credits the community collections and surveys used for discovery. Bibliographic records use retained arXiv, publisher, and author sources; publication observations preserve edition-specific titles, authors, and date precision. Selected assets are checked against publisher repositories, documentation, and model/dataset cards. Upstream READMEs, papers, model weights, and datasets are not redistributed here.
Original repository code and annotations use the MIT license. Linked resources retain their own terms; this repository does not relicense them.
To cite the catalog, use CITATION.cff or GitHub's "Cite this repository" button. The data changes with every scan, so pin the exact version you used. Each v* tag has a data release with versioned downloads built from the tagged commit: the catalog as JSONL, JSON and SQLite, the coverage ledger, topic and profile exports, and their SHA256SUMS. Cite the release tag, or record the commit from git rev-parse HEAD in your clone, or link files at that commit, such as https://github.com/reacher-z/Awesome-CUA/blob/<commit>/data/catalog.jsonl. Cutting a release.