The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.

[ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

Code and data for "Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs"

CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

PG-Video-LLaVA: Pixel Grounding in Large Multimodal Video Models

We perform functional grounding of LLMs' knowledge in BabyAI-Text

Official implementation of ICCV19 oral paper Zero-Shot grounding of Objects from Natural Language Queries (https://arxiv.org/abs/1908.07129)

Code and data for "StructLM: Towards Building Generalist Models for Structured Knowledge Grounding" (COLM 2024)

[CVPR20] Video Object Grounding using Semantic Roles in Language Description (https://arxiv.org/abs/2003.10606)

Hierarchical Universal Language Conditioned Policies

[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)

[Paper][AAAI 2023] DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning

[ICRA2023] Grounding Language with Visual Affordances over Unstructured Data

[ACM MM 2024] Hierarchical Multimodal Fine-grained Modulation for Visual Grounding.

Code for CVPR'18 "Grounding Referring Expressions in Images by Variational Context"

Code repo for "Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding"

[CVPR 2024] Code for "Improved Visual Grounding through Self-Consistent Explanations".

A simple demo for utilizing grounding dino and segment anything v2 models together

A python library for design of earthing networks in electrical substations.

Draw relatedness and causal graphs out of PDDL files.