Skip to content
Integration of the VQA game GuessWhat?! on the robot IRL-1 using ROS
Python JavaScript HTML Shell CMake CSS Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github
docker
docs
scripts
src
tests
.eslintrc.json
.gitignore
.gitmodules
.pylintrc
.readthedocs.yml
AUTHORS
LICENSE
README.rst

README.rst

Welcome to the DEVINE Project!

The goal of the DEVINE Project is to serve as a demonstration on how research results in grounded language learning and understanding can be used in a cooperative task between an intelligent agent and a human. The task, undertaken by a robot, is the question answering game GuessWhat?!. The robot interacts with a human player who performs the role of the oracle. The oracle selects an object in a visual scene without disclosing it to the robot. Playing the role of the guesser, the robot asks questions to the person to deduce which object was selected. The guesser’s questions are generated using the player’s previous answers and visual information. Once the robot has a strong belief, it guesses the object chosen by the player and physically points to it in the scene.

The robotic platform for the demonstration is IRL-1. By design, it can operate in a typical human environment. It is equipped with a Kinect, differential elastic actuators for its arms, a microphone array, and powered wheels with omni directional capability. As IRL-1 has compliant joints, players are protected from physical harm. To begin playing GuessWhat?! with IRL-1, a player approaches the robot. Once the player is close enough, IRL-1 asks if they want to play. After explaining the rules, IRL-1 takes an image of the scene comprising different objects. Then, it turns back to the player and asks if they have selected an object. Upon confirmation, IRL-1 begins to ask them questions. After five questions, the robot reveals its guess. The player validates IRL-1’s guess, IRL-1 reacts accordingly and then concludes the game.

Accessing the documentation

https://devine.readthedocs.io/

You can’t perform that action at this time.