Tofu pipeline

[J-Dymond]

Basic tofu dataset pipeline set up. There are 4 different granularity settings, which are admittedly defined in quite a convoluted way, but I think the comments and readme explain how to use them.

[philswatton]

This is good work - much better than my first commit when I started here!

Some comments contained below/. The big ones are around refactoring data_utils.py - some of this code has some duplication and in general data_utils.py feels like it can be a little shorter. I've made some initial suggestions in this direction, happy to discuss during out meeting tomorrow.

Finally: I'd suggest renaming data_utils.py to load_data.py or even just tofu.py

[jack89roberts]

Finally: I'd suggest renaming data_utils.py to load_data.py or even just tofu.py

I'd go tofu.py

[jack89roberts]

On all the comments re indexing etc. I don't think we need to reinvent the wheel if something already works. But my main suggestion is to consider whether it would be helpful to create author_id and question_id columns in the dataset at the beginning, which could also be used in the various sampling/filtering functions. One pro of this is it might make some analysis easier later - e.g. we wouldn't need to keep propagating information about which questions/authors have been removed to figure out which rows in the retain/forget sets correspond to which authors (as we could just refer to the author_id).

HuggingFace Datasets also have some in-built functions that could help: https://huggingface.co/docs/datasets/en/process . E.g. with an author_id column the author_level split could just be something like (probably not the most efficient way):

forget_set = all_data.filter(lambda row: row["author_id"] in forgotten_author_numbers)
retain_set = all_data.filter(lambda row: row["author_id"] not in forgotten_author_numbers)

[philswatton]

On all the comments re indexing etc. I don't think we need to reinvent the wheel if something already works. But my main suggestion is to consider whether it would be helpful to create author_id and question_id columns in the dataset at the beginning, which could also be used in the various sampling/filtering functions. One pro of this is it might make some analysis easier later - e.g. we wouldn't need to keep propagating information about which questions/authors have been removed to figure out which rows in the retain/forget sets correspond to which authors (as we could just refer to the author_id).

HuggingFace Datasets also have some in-built functions that could help: https://huggingface.co/docs/datasets/en/process . E.g. with an author_id column the author_level split could just be something like (probably not the most efficient way):

forget_set = all_data.filter(lambda row: row["author_id"] in forgotten_author_numbers)
retain_set = all_data.filter(lambda row: row["author_id"] not in forgotten_author_numbers)

I was thinking about this too - am open to arguments either way for now, but we'll almost certainly want to have our own indexing for each level of hierarchy when we start generating our own data in #2

[jack89roberts]

Good to merge pending the last few changes we discussed.