Switchboard Dialog Act Corpus added under datasets/swda

[gmihaila]

Switchboard Dialog Act Corpus

Intro:
The Switchboard Dialog Act Corpus (SwDA) extends the Switchboard-1 Telephone Speech Corpus, Release 2,
with turn/utterance-level dialog-act tags. The tags summarize syntactic, semantic, and pragmatic information
about the associated turn. The SwDA project was undertaken at UC Boulder in the late 1990s.

Details:
homepage
repo

I believe this is an important dataset to have since there is no dataset related to dialogue act added.

I didn't find any formatting for pull request. I hope all this information is enough.

For any support please contact me.

[lhoestq]

Really cool thank you !

I left a few comments

After changing the feature type to ClassLabel you'll need to regenerate the dataset_infos.json file

datasets-cli test ./datasets/swda --save_infos --all_configs --ignore_verifications

[gmihaila]

@lhoestq Thank you for your detailed comments! I fixed everything you suggested.

Please let me know if I'm missing anything else.

[lhoestq]

Thanks !

[lhoestq]

It looks like the Transcript and Utterance objects are missing, maybe we can mention it in the README ? Or just add them ? @gmihaila @bhavitvyamalik

[bhavitvyamalik]

Hi @lhoestq,
I'm working on this to add the full dataset

[gmihaila]

It looks like the Transcript and Utterance objects are missing, maybe we can mention it in the README ? Or just add them ? @gmihaila @bhavitvyamalik

@lhoestq Any info on how to add them?

[bhavitvyamalik]

@gmihaila, instead of using the current repo you should look into this. You can use the csv files uploaded in this repo (swda.zip) to access other fields and include them in this dataset. It has one dependency too, swda.py, you can download that separately and include it in your dataset's folder to be imported while reading the csv files.

Almost all the attributes of Transcript and Utterance objects are of the type str, int, or list. As far as trees attribute is concerned in utterance object you can simply parse it as string and user can maybe later convert it to nltk.tree object

[gmihaila]

@bhavitvyamalik Thank you for the clarification!

I didn't use that because it doesn't have the splits. I think in combination with what I used would help.

Let me know if I can help! I can make those changes if you don't have the time.

[bhavitvyamalik]

I'm a bit busy for the next 2 weeks. I'll be able to complete it by end of January only. Maybe you can start with it and I'll help you?
Also, I looked into the official train/val/test splits and not all the files are there in the repo I used so I think either we'll have to skip them or put all of that into just train

[gmihaila]

Yes, I can start working on it and ask you to do a code review.

Yes, not all files are there. I'll try to find papers that have the correct and full splits, if not, I'll do like you suggested.

Thank you again for your help @bhavitvyamalik !