Skip to content
Cornell House Agent Learning Environment
Branch: master
Clone or download
Latest commit aaf68b9 Nov 9, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information.
src Added a draft of CHALET unity3d code. Oct 11, 2018
unit_test Added a draft of CHALET unity3d code. Oct 11, 2018 Update Nov 9, 2018

CHALET: Cornell House Agent Learning Environment

CHALET is a 3D house simulator with support for navigation and manipulation. Unlike existing systems, CHALET supports both a wide range of object manipulation, as well as supporting complex environemnt layouts consisting of multiple rooms. The range of object manipulations includes the ability to pick up and place objects, toggle the state of objects like taps or televesions, open or close containers, and insert or remove objects from these containers. In addition, the simulator comes with 58 rooms that can be combined to create houses, including 10 default house layouts. CHALET is therefore suitable for setting up challenging environments for various AI tasks that require complex language understanding and planning, such as navigation, manipulation, instruction following, and interactive question answering.


CHALET Version 0.1 is available for download at:

An unstable version of the CHALET source code is now available.

Please see the Wiki for details on how to use these builds and the source code.

For support and issues, please contact us through the issues section of the repository.

Source Code

The CHALET source code is released without proprietary assets purcahsed from the Unity Asset Store. These assets are required to work with the original source, but not with the released binaries. We extensively modified the resources for CHALET. If you wish to work with the source code, please contact us ( for instruction on obtaining the required resources. See more details in the wiki.


Environment and Simulator:
CHALET: Cornell House Agent Learning Environment
Claudia Yan, Dipendra Misra, Andrew Bennett, Aaron Walsman, Yonatan Bisk and Yoav Artzi
arXiv report 2018

Instructional Data:
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
Dipendra Misra, Andrew Bennett, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi
EMNLP 2018


Attribution 4.0 International (CC BY 4.0)

Youtube Demos:

Demo 1 Demo 2 Demo 3

You can’t perform that action at this time.