Skip to content

Commit

Permalink
Add TBD tags to the taxonomy.md file and remove social_agents.bib file
Browse files Browse the repository at this point in the history
  • Loading branch information
XuhuiZhou committed Apr 10, 2024
1 parent f7623b8 commit 2d00106
Show file tree
Hide file tree
Showing 6 changed files with 173 additions and 137 deletions.
26 changes: 0 additions & 26 deletions bibtex/social_agents.bib

This file was deleted.

16 changes: 14 additions & 2 deletions bibtex_to_table.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ def preprocess_entry(entry: dict, taxonomy:dict[str, list[str]]) -> dict:
authors_processed.append(author)
authors = ", ".join(authors_processed)
entry["author"] = authors
entry["author_et_al"] = authors.split(",")[0] + " et al."

if "eprint" in entry:
entry["month"] = entry["eprint"].split(".")[0][2:]
Expand All @@ -58,6 +59,16 @@ def preprocess_entry(entry: dict, taxonomy:dict[str, list[str]]) -> dict:
else:
raise ValueError("Month field is missing")

# get the venue of the paper
if "journal" in entry:
entry["venue"] = entry["journal"]
elif "booktitle" in entry:
entry["venue"] = entry["booktitle"]
elif "eprint" in entry:
entry["venue"] = "arXiv"
else:
raise ValueError("Venue field is missing")

entry["title_w_url"] = f"[{entry['title']}]({entry['url']})"
entry["date"] = f"{entry['month']}, {entry['year']}" if "month" in entry else entry["year"]

Expand Down Expand Up @@ -86,10 +97,11 @@ def bibtex_to_table(bibtex: str, taxonomy: dict[str, list[str]]) -> str:
entries.append(entry_dict)

# Create a Markdown table
headers = ["Title", "Date"] + list(taxonomy.keys())
headers = ["Title", "Date"] + list(taxonomy.keys()) + ["helper"]
rows = []
for entry in entries:
row = [entry["title_w_url"], entry["date"]]+[entry[category] for category in taxonomy.keys()]
row += [f"[{row[1]}] {row[0]}, {entry['author_et_al']}, {entry['venue']}"]
rows.append(row)

df = pd.DataFrame(rows, columns=headers)
Expand All @@ -101,7 +113,7 @@ def bibtex_to_table(bibtex: str, taxonomy: dict[str, list[str]]) -> str:
taxonomy = parse_markdown_file('./docs/taxonomy.md')

# Read the BibTeX file
bibtex_file = "./bibtex/social_agents.bib"
bibtex_file = "./main.bib"
with open(bibtex_file, "r") as f:
bibtex_string = f.read()

Expand Down
116 changes: 94 additions & 22 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,38 +35,110 @@ Our goal is to offer insights crucial for understanding and harnessing **social
📝 *We are currently working on a survey paper related to content of this repository. Stay tuned for updates!*

## Table of Contents

- [Papers](#papers)
- [Surveys and Overview](#surveys-and-overview)
- [Social Interaction Types](#social-interaction-types)
- [Collaboration](#language-models)
- [Cooperation](#cooperation)
- [Competition](#competition)
- [Exchange](#exchange)
- [Persuasion](#persuasion)
- [Negotiation](#negotiation)
- [Text Simulation](#text-simulation)
- [Environments](#environments)
- [Agents](#agents)
- [Applications](#applications)
- [Embodied and Virtual Simulation](#embodied-and-virtual-simulation)
- [Environments](#environments)
- [Agents](#agents)
- [Applications](#applications)
- [Human-X Interaction](#human-x-interaction)
- [Human-chatbot Interaction](#human-chatbot-interaction)
- [Human-embodied-agent Interaction](#human-embodied-agent-interaction)
- [Human-robot Interaction](#human-robot-interaction)
- [Ethics](#ethics)
- [Environments](#environments)
- [Text and Speech Environments](#text-and-speech-environments)
- [Embodied Environments](#embodied-environments)
- [Virtual Environments](#virtual-environments)
- [Robotics](#robotics)
- [Modeling](#modeling)
- [In-context Learning](#in-context-learning)
- [Finetuning](#finetuning)
- [Reinforcement learning](#reinforcement-learning)
- [Evaluating social agents](#evaluating-social-agents)
- [Evaluating text social agents](#evaluating-text-social-agents)
- [Evaluating embodied social agents](#evaluating-embodied-social-agents)
- [Evaluating virtual social agents](#evaluating-virtual-social-agents)
- [Evaluating robotics in social contexts](#evaluating-robotics-in-social-contexts)
- [Interactions with humans](#interactions-with-humans)
- [Human-Chatbot Interaction](#human-chatbot-interaction)
- [Human-Embodied Agent Interaction](#human-embodied-agent-interaction)
- [Human Robot Interaction](#human-robot-interaction)
- [Human-Human Interaction](#human-human-interaction)
- [Challenges](#challenges)
- [Theory of Mind](#theory-of-mind)
- [Social Learning](#social-learning)
- [Simultaneous Interaction](#simultaneous-interaction)
- [Applications](#applications)
- [Health](#health)
- [Policy](#policy)
- [Education](#education)
- [Concerns](#concerns)
- [Risks](#risks)
- [Safety](#safety)

## Papers
### Surveys and Overview
- [June, 2023] **[Socially intelligent machines that learn from humans and help humans learn](https://royalsocietypublishing.org/doi/10.1098/rsta.2022.0048)**, Gweon et al., arXiv, 2023.

### Environments

#### Text and Speech Environments

#### Embodied Environments

#### Virtual Environments

#### Robotics

### Modeling

#### In-context Learning

#### Finetuning

#### Reinforcement learning

### Evaluating social agents

#### Evaluating text social agents

#### Evaluating embodied social agents

#### Evaluating virtual social agents

#### Evaluating robotics in social contexts

### Interactions with humans

#### Human-Chatbot Interaction

#### Human-Embodied Agent Interaction

#### Human Robot Interaction

#### Human-Human Interaction

### Challenges

#### Theory of Mind

#### Social Learning

#### Simultaneous Interaction

### Applications

#### Health

#### Policy

#### Education

### Concerns

#### Risks

#### Safety



## Papers

### Surveys and Overview

- [June, 2023] **[Socially intelligent machines that learn from humans and help humans learn](https://royalsocietypublishing.org/doi/10.1098/rsta.2022.0048)**, Gweon et al., arXiv, 2023.



### Social Interaction Types
Expand Down
10 changes: 6 additions & 4 deletions docs/paper_table.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
| Title | Date | environment | agents | evaluation | other |
|:-------------------------------------------------------------------------------------------------------------------------|:--------------|:----------------------------------|:----------------------------------|:-----------------------------------------|:---------------------|
| [Communicative Agents for Software Development](https://arxiv.org/abs/2307.07924) | July, 2023 | Collaboration, Embodied AI | Prompting, More than three agents | Rule-based evaluation | No human involvement |
| [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://openreview.net/forum?id=mM7VurbA4r) | October, 2024 | Mixed Objectives, Text and Speech | Prompting, Two agents | Model-based evaluation, Human evaluation | Human-in-loop |
| Title | Date | environment | agents | evaluation | other | helper |
|:----------------------------------------------------------------------------------------------------------------------------|:--------------|:----------------------------------|:----------------------------------|:-----------------------------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Communicative Agents for Software Development](https://arxiv.org/abs/2307.07924) | July, 2023 | Collaboration, Embodied AI | Prompting, More than three agents | Rule-based evaluation | No human involvement | [July, 2023] [Communicative Agents for Software Development](https://arxiv.org/abs/2307.07924), Chen Qian et al., arXiv |
| [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://openreview.net/forum?id=mM7VurbA4r) | October, 2024 | Mixed Objectives, Text and Speech | Prompting, Two agents | Model-based evaluation, Human evaluation | Human-in-loop | [October, 2024] [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://openreview.net/forum?id=mM7VurbA4r), Xuhui Zhou et al., ICLR |
| [Embodied LLM Agents Learn to Cooperate in Organized Teams](https://arxiv.org/abs/2403.12482) | March, 2024 | Collaboration, Embodied AI | Prompting, More than three agents | Model-based evaluation, Human evaluation | Education | [March, 2024] [Embodied LLM Agents Learn to Cooperate in Organized Teams](https://arxiv.org/abs/2403.12482), Xudong Guo et al., arXiv |
| [CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents](https://arxiv.org/abs/2310.17512) | October, 2023 | Pure Competition, Text and Speech | Prompting | Rule-based evaluation | No human involvement | [October, 2023] [CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents](https://arxiv.org/abs/2310.17512), Qinlin Zhao et al., arXiv |
9 changes: 5 additions & 4 deletions docs/taxonomy.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Taxonomy
## 1 Environment
#### TBD
*The objectives of different agents*
### Social interaction types

Expand Down Expand Up @@ -38,7 +39,7 @@

## 2 Agents
*The types of agents involved in the simulation*

#### TBD
### Modeling
*The techniques used to model the agents*

Expand All @@ -62,6 +63,7 @@

## 3 Evaluation

#### TBD
#### Only qualitative evaluation
*You should definitely add this tag if a work is only based on qualitative evaluation*

Expand All @@ -75,6 +77,7 @@


## 4 Other
#### N/A

### Human involvement
#### No human involvement
Expand All @@ -87,6 +90,4 @@
#### Education
#### Training
#### Entertainment
#### Policy
#### Inter personal communication

#### Policy
133 changes: 54 additions & 79 deletions main.bib
Original file line number Diff line number Diff line change
@@ -1,79 +1,54 @@
@article{gweon2023socially,
title={Socially intelligent machines that learn from humans and help humans learn},
author={Gweon, Hyowon and Fan, Judith and Kim, Been},
journal={Philosophical Transactions of the Royal Society A},
volume={381},
number={2251},
pages={20220048},
social_interaction_types = {Irrelevant},
number_of_agents = {Irrelevant},
social_interaction_environments = {Irrelevant},
modeling_techniques = {Irrelevant},
human_involvement = {Irrelevant},
evaluation_protocol = {Irrelevant},
year={2023},
url = {https://doi.org/10.1098/rsta.2022.0048},
publisher={The Royal Society}
}

@article{min2021film,
title={Film: Following instructions in language with modular methods},
author={Min, So Yeon and Chaplot, Devendra Singh and Ravikumar, Pradeep and Bisk, Yonatan and Salakhutdinov, Ruslan},
journal={arXiv preprint arXiv:2110.07342},
year={2021}
}

@article{min2022don,
title={Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue},
author={Min, So Yeon and Zhu, Hao and Salakhutdinov, Ruslan and Bisk, Yonatan},
journal={arXiv preprint arXiv:2210.04443},
year={2022}
}

@article{puig2023habitat,
title={Habitat 3.0: A co-habitat for humans, avatars and robots},
author={Puig, Xavier and Undersander, Eric and Szot, Andrew and Cote, Mikael Dallaire and Yang, Tsung-Yen and Partsey, Ruslan and Desai, Ruta and Clegg, Alexander William and Hlavac, Michal and Min, So Yeon and others},
journal={arXiv preprint arXiv:2310.13724},
year={2023}
}

@inproceedings{min2023self,
title={Self-Supervised Object Goal Navigation with In-Situ Finetuning},
author={Min, So Yeon and Tsai, Yao-Hung Hubert and Ding, Wei and Farhadi, Ali and Salakhutdinov, Ruslan and Bisk, Yonatan and Zhang, Jian},
booktitle={2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages={7119--7126},
year={2023},
organization={IEEE}
}

@inproceedings{zhu2023excalibur,
title={EXCALIBUR: Encouraging and Evaluating Embodied Exploration},
author={Zhu, Hao and Kapoor, Raghav and Min, So Yeon and Han, Winson and Li, Jiatai and Geng, Kaiwen and Neubig, Graham and Bisk, Yonatan and Kembhavi, Aniruddha and Weihs, Luca},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={14931--14942},
year={2023}
}


@article{wu2023plan,
title={Plan, Eliminate, and Track--Language Models are Good Teachers for Embodied Agents},
author={Wu, Yue and Min, So Yeon and Bisk, Yonatan and Salakhutdinov, Ruslan and Azaria, Amos and Li, Yuanzhi and Mitchell, Tom and Prabhumoye, Shrimai},
journal={arXiv preprint arXiv:2305.02412},
year={2023}
}


@article{chang2023goat,
title={Goat: Go to any thing},
author={Chang, Matthew and Gervet, Theophile and Khanna, Mukul and Yenamandra, Sriram and Shah, Dhruv and Min, So Yeon and Shah, Kavit and Paxton, Chris and Gupta, Saurabh and Batra, Dhruv and others},
journal={arXiv preprint arXiv:2311.06430},
year={2023}
}

@inproceedings{shridhar2020alfred,
title={ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks},
author={Shridhar, Mohit and Thomason, Jesse and Gordon, Daniel and Bisk, Yonatan and Han, Winson and Mottaghi, Roozbeh and Zettlemoyer, Luke and Fox, Dieter},
booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
pages={10740--10749},
year={2020}
}
@misc{qian2023communicative,
title={Communicative Agents for Software Development},
author={Chen Qian and Xin Cong and Wei Liu and Cheng Yang and Weize Chen and Yusheng Su and Yufan Dang and Jiahao Li and Juyuan Xu and Dahai Li and Zhiyuan Liu and Maosong Sun},
year={2023},
url={https://arxiv.org/abs/2307.07924},
environment = {Collaboration, Embodied AI},
agents = {Prompting, More than three agents},
evaluation = {Rule-based evaluation},
other = {No human involvement},
eprint={2307.07924},
archivePrefix={arXiv},
primaryClass={cs.SE}
}

@inproceedings{zhou2024sotopia,
title={SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents},
author={Xuhui Zhou and Hao Zhu and Leena Mathur and Ruohong Zhang and Haofei Yu and Zhengyang Qi and Louis-Philippe Morency and Yonatan Bisk and Daniel Fried and Graham Neubig and Maarten Sap},
booktitle={ICLR},
environment = {Mixed Objectives, Text and Speech},
agents = {Prompting, Two agents},
evaluation = {Model-based evaluation, Human evaluation},
other = {Human-in-loop},
year={2024},
month = {10},
url={https://openreview.net/forum?id=mM7VurbA4r}
}

@misc{guo2024embodied,
title={Embodied LLM Agents Learn to Cooperate in Organized Teams},
author={Xudong Guo and Kaixuan Huang and Jiale Liu and Wenhui Fan and Natalia Vélez and Qingyun Wu and Huazheng Wang and Thomas L. Griffiths and Mengdi Wang},
year={2024},
environment = {Collaboration, Embodied AI},
agents = {Prompting, More than three agents},
evaluation = {Model-based evaluation, Human evaluation},
url={https://arxiv.org/abs/2403.12482},
other = {Education},
eprint={2403.12482},
archivePrefix={arXiv},
primaryClass={cs.AI}
}

@misc{zhao2023competeai,
title={CompeteAI: Understanding the Competition Behaviors in Large Language Model-based Agents},
author={Qinlin Zhao and Jindong Wang and Yixuan Zhang and Yiqiao Jin and Kaijie Zhu and Hao Chen and Xing Xie},
environment = {Pure Competition, Text and Speech},
agents = {Prompting},
evaluation = {Rule-based evaluation},
url = {https://arxiv.org/abs/2310.17512},
other = {No human involvement},
year={2023},
eprint={2310.17512},
archivePrefix={arXiv},
primaryClass={cs.AI}
}

0 comments on commit 2d00106

Please sign in to comment.