Is there a limitation of using dataset for different algs?

[mrbeann]

Firstly, thank you for building this awesome benchmark. While I try the example with different datasets (e.g., I try astra with youtube dataset), I got some errors like this,

    loss = cross_entropy_with_probs(predict_l, batch['labels'].to(device))
KeyError: 'labels'

Can this be fixed?

[JieyuZ2]

Hey did you use the new youtube dataset in the classification folder in this link?

The youtube dataset is updated, but the dataset.zip hasn't. I could run astra with youtube dataset

[JieyuZ2]

Oh did you use Bert backbone?

[JieyuZ2]

@mrbeann hey, if you're using BERT backbone for astra, then there's a bug and I just fixed it. Could you update to the latest commit and see if it works for you?

[mrbeann]

Yeah, I use BERT. And I just pull the newest version. It seems some errors are fixed but it still raises errors.

    weak_labels = batch['weak_labels'].long().to(device)
KeyError: 'weak_labels'

[JieyuZ2]

@mrbeann sry, one minute!

[JieyuZ2]

@mrbeann should work now!

[mrbeann]

Thanks for the quick fix, it works. And, it will be great if this repo can include some benchmark results of different algorithms.

[JieyuZ2]

hey @mrbeann for benchmark results plz check out our NeurIPS paper! For newly-added methods, the results should be released in our future paper!