Added biostars_qa dataset and pre-processing scripts

[cannin]

Adds BioStars QA dataset and Resolves: #2236

[github-actions]

❌ pre-commit failed.
Please run pre-commit run --all-files locally and commit the changes.
Find more information in the repository's CONTRIBUTING.md

[olliestanley]

Would it be possible to put the code in each file into a function, and then have a single script which calls both functions? That would make it a little easier to reproduce this dataset quickly if needed in future

[olliestanley]

Thank you!

[simorgh10]

Hello,

I started to delve into Open Assistant code, and I am going through the latest commits. I am also new to Open Source contribution. So I beg your indulgence.

In commit 2997fd1, parameters passed to function get_biostars_dataset are shadowed by local variables.

Seems to a be minor detail though. Need a new issue for that ?

def get_biostars_dataset(start_idx=9557161, accept_threshold=1000000, sleep=0.1, folder="biostars"):
    ...
    start_idx = 9557161
    accept_threshold = 1000000
    sleep = 0.1

[olliestanley]

Hello,

I started to delve into Open Assistant code, and I am going through the latest commits. I am also new to Open Source contribution. So I beg your indulgence.

In commit 2997fd1, parameters passed to function get_biostars_dataset are shadowed by local variables.

Seems to a be minor detail though. Need a new issue for that ?

def get_biostars_dataset(start_idx=9557161, accept_threshold=1000000, sleep=0.1, folder="biostars"):
    ...
    start_idx = 9557161
    accept_threshold = 1000000
    sleep = 0.1

Good spot. I don't think this is going to be a problem, but you're welcome to make a pull request fixing it, no need to make an issue first!