Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
H2SL Natural Language Symbol Grounding Project
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
H2SL (Human to Structured Language) provides a framework for realtime natural langugage symbol grounding in the context of understanding robot instructions. Copyright (C) 2014 by the Massachusetts Institute of Technology Developed by Thomas M. Howard and Matthew R. Walter at the Computer Science and Artificial Intelligence Laboratory, MIT, Cambridge, Massachusetts USA, with partial support from the U.S. Army Research Laboratory under the Robotics Collaborative Technology Alliance, Cooperative Agreement W911NF-10-2-0016 and National Science Foundation National Robotics Initiative Award #1427547. Contributors to this project include: Derya Aksaray Jacob Arkin Rohan Paul Thomas Howard Matthew Walter This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, see <http://www.gnu.org/licenses/gpl-2.0.html> or write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. OVERVIEW H2SL provides a set of base classes for developing a natural language symbol grounding inferface. Example symbols, features, and data is provided, but is not intended to serve as a drop-in natural language interface for robotic system. To adapt to new applications, one can replace the symbols, features, and examples with those that are relevant to your system and domain. The probablistic graphical models included in this project consist of the Distributed Correspondence Graph (DCG), Hierarhical Distributed Correspondence Graph (HDCG), Adaptive Distributed Correspondence Graph (ADCG), and Hierarhical Adaptive Distributed Correspondence Graph (HADCG). For more details about the mathematical basis of each model, please refer to the following papers and articles: T.M Howard, S. Tellex, and N. Roy. A natural language planner interface for mobile manipulators. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 6652–6659. IEEE, May 2014. O. Propp, I. Chung, M.R. Walter, and T.M. Howard. On the performance of hierarchical distributed correspondence graphs for efficient symbol grounding of robot instructions. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2015. R. Paul, J. Arkin, N. Roy, and T.M. Howard. Effcient grounding of abstract spatial concepts for natural language interaction with robot manipulators. In Proceedings of the 2016 Robotics: Science and Systems Conference, June 2016. Examples of applications of this framework to various problems in natural language symbol grounding of robot instructions include inferring LTL from language, assistive robotics, inferring maps and behaviors, expressing homotopy constraints and multi-modal interactions with human-robot teams. For more details of these applications, please refer to the following papers and articles: F. Duvallet, M.R. Walter, T.M. Howard, S. Hemachandra, J. Oh, S. Teller, N. Roy, and A. Stentz. A probabilistic framework for inferring maps and behaviors from natural language. In Proceedings of the 14th International Symposium on Experimental Robotics, July 2014. S. Hemachandra, F. Duvallet, T.M. Howard, N. Roy, A. Stentz, and M.R. Walter. Learning models for following natural language directions in unknown environments. In Proceedings of the IEEE International Conference on Robotics and Automation. IEEE, May 2015. D. Yi, T.M. Howard, K. Seppi, and M. Goodrich. Expressing homotopic requirements for mobile robot navigation through natural language instructions. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2016. J. Oh, T.M. Howard, M. Walter, D. Barber, M. Zhu, S. Park, A. Suppe, L. Navarro-Serment, F. Duvallet, A. Boularias, O. Romero, J. Vinokrov, T. Keegan, R. Dean, C. Lennon, B. Bodt, M. Childers, J. Shi, K. Daniilidis, N. Roy, C. Lebiere, M. Hebert, and A. Stentz. Integrated intelligence for human-robot teams. In Proceedings of the 2016 International Symposium on Experimental Robotics, October 2016. A. Boteanu, J. Arkin, T.M. Howard, and H. Kress-Gazit. A model for verifable grounding and execution of complex language instructions. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2016. A. Broad, J. Arkin, N. Ratliff, T. M. Howard, and B. Argall. Real-time natural language corrections for assistive robotic manipulators. International Journal of Robotics Research, 36(5-7):684–698, May 2017. J. Arkin, M. Walter, A. Boteanu, M. Napoli, H. Biggie, H. Kress-Gazit, and T.M. Howard. Contextual awareness: Understanding monologic natural language instructions for autonomous robots. In Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication, August 2017. TUTORIAL (DCG) To train the model from a corpus of labeled examples, run ... h2sl-llm-train (examples files) --feature_set=(feature_set.xml) --output=(llm.xml) To run the Distributed Correspondence Graph (DCG) demo, run ... h2sl-dcg-demo --world=(world.xml) --llm=(llm.xml) --grammar=(grammar.xml) --command=(command string) -output=(output.xml) To run the Distributed Correspondence Graph (DCG) test, run ... h2sl-dcg-test (example files) --llm=(llm.xml) --grammar=(grammar.xml) To run the Graphical User Interface (GUI), run ... h2sl-gui-demo --world=(world.xml) --llm=(llm.xml) --grammar=(grammar.xml)