Skip to content

Load unlabelled data for alignment training#112

Merged
gabrieletijunaityte merged 9 commits into
developfrom
aligment_experiments
May 29, 2026
Merged

Load unlabelled data for alignment training#112
gabrieletijunaityte merged 9 commits into
developfrom
aligment_experiments

Conversation

@vdplasthijs

@vdplasthijs vdplasthijs commented May 29, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

  • Notebook & GEE scripts I used to sample unlabelled data.
  • Added option in datamodule to split by "split" column in dataset.df
  • Dataset can load unlabelled data. Assuming filename structure s2bms/model_ready_s2bms-unlabelled-[...].csv. I've added an arg use_unlabelled_data, and if True, it uses this file instead. In the future we could merge the two model ready files for alignment training, if we like.
  • Added new config file. I've tested alignment training with both original and unlabelled data, and both work well.
  • Add an exception (E266) to hooks to allow double-hash comments. (## ...). Hope that's OK? I use it a lot because then if I (un)comment a block of code with shortcut, it doesn't affect double-hashed comments.
  • Added the csv file. I know it's a bit large (8 MB), but though it would be good to have it as an example in the repo.

Before submitting

  • Did you make sure title is self-explanatory and the description concisely explains the PR?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you test your PR locally with pytest command?

@vdplasthijs

Copy link
Copy Markdown
Collaborator Author

@gabrieletijunaityte now also supports loading multiple datasets. You can either pass dataset name as a str or list, and it loads accordingly. I've added the split of the new unlabelled data, and the merged splits, so now it just reads the split from the file.
Apologies I also pushed something else re adjectives, it's unrelated but doesn't interfere with anything, so happy to merge it already.

@gabrieletijunaityte gabrieletijunaityte merged commit e1e74d9 into develop May 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants