ACL 2020 Tutorial Link to slides: https://docs.google.com/presentation/d/11to5fJBltQuQ2E7ShM2XEIPKVjTUCOFgkWC7tXaelD0/edit?usp=sharing
- Part 1
- Introduction
- Fundamental approaches to multimodal grounding
- Grounding in communication: science and models
- Multimodality in human-human conversation
- Part 2
- Grounding with animated agents and robots
- Adapting grounding to modality and culture
- Exploiting language technology for grounding in multimodal contexts
- Learning multimodal grounding: visual dialogue and other datasets
- Applications, conclusions and future work
The tutorial will be self-contained. For further readings, we recommend the following publications that are central to the non-verbal grounding framework as of late 2019:
- Grounding in communication, Herb Clark and Susan Brennan.
- Meaning and demonstration by Una Stojnic and Matthew Stone
- Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game, Ramesh Manuvinakurike, David DeVault and Kallirroi Georgila.
- Language to Action: Towards Interactive Task Learning with Physical Agents, Joyce Y. Chai by Joyce Y. Chai et al.
- It's Not What You Do, It's How You Do It: Grounding Uncertainty for a Simple Robot, Julian Hough and David Schlangen.
- Learning Effective Multimodal Dialogue Strategies from Wizard-of-Oz Data: Bootstrapping and Evaluation rieser-lemon by Verena Rieser and Oliver Lemon.
- A survey of nonverbal signaling methods for non-humanoid robots by Elizabeth Cha et al.
- The Devil is in the Details: A Magnifying Glass for the GuessWhich Visual Dialogue Game by Alberto Testoni et al.