-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Regarding Exploit Only Baseline and other errors #2
Comments
I also have an unrelated question. Why is |
Another error I encountered is when I try to run After all these changes I've been stuck here for quite a while - ![]() The progress bar doesn't seem to be running and the end time estimate is not populated Do let me know if you have any suggestions to fix these in a different way? EDIT: I checked in after a few hours and it seems to be running (The ETA is 50ish hours for T0Mix alongside the 3B Model) |
WEIGHT_INIT_SAMPLES is only used for the UCB1 algorithm which has an explicit reward initialization phase. See algorithms 1 and 2 from the paper (https://arxiv.org/pdf/2302.00674.pdf) In reality, you could also initialize the rewards for EXP3, but we didn't for our experiments. |
Regarding the weight initialization. The script currently is set to run over 2 quantities of weight initialization samples (this line) and 5 different seeds (this line). You can reduce those to just use a single weight initialization samples, and a single seed which will significantly speed up the initialization. Thanks for pointing this out, I think I know what the problem is. At one point, I moved the weight initialization into the trainer class (here), but didn't make it compatible with the multirun_train_mixed script. For now, the solution is to compute gradients prior to the Exploit-only method, as you're currently doing. And I will add that information into the instructions. Thank you for finding this bug! Let me know if it still doesn't work for some reason after pre-computing the gradients. |
Thanks for the reference, I'll check it out! |
Okay, I've made a few fixes and been able to run the Try pulling the newest version of the code base and let me know if you can run |
Thank you for the quick response! Just to confirm should I still be running the gradient alignment computations first? or can they be run in parallel now? |
Also, I did catch this error once: The solution that I found for that is to change lines 1639-1640 from: # Initialize weights if needed
if self.args.weight_initialization_samples > 0:
self._initialize_weights(train_dataloader, target_dataloader, model) to # Initialize weights if needed
if self.args.weight_initialization_samples > 0:
if hasattr(model, "name_or_path"):
self._initialize_weights(train_dataloader, target_dataloader, model)
else:
self._initialize_weights(train_dataloader, target_dataloader, self.model) I'm hesitant to make that change unless it's required though, because that will affect both the EXP3 and UCB1 trainers as well. |
The |
Got it! |
Sorry for so many back and forths. When I was debugging the Now I've successfully been able to precompute the alignments, and then train the exploit-only model. To replicate our experiments you do need to run |
Thanks for all the fixes! |
Any update on this? Have you succeeded with the exploit-only baseline? Have you been able to run either the EXP3 or UCB1 methods? Just checking to see if you've found any other bugs |
Hey |
Hi!
Thanks for the great code
I was able to run the explore only baseline fairly easily but facing some issues with the exploit only baseline. I'll go over the issues I encountered and how I tried fixing them step by step.
args
, the attribute is being extracted fromtraining_args
where it does not existNone
on the same line so I decide to add a path myself as the default path isNone
and the bash file does not provide a path. To do so I add the arg in the bash file so my bash file now looks like this -Weight save file outputs/precomputed_weight_grads/exploit_only/42/google/t5-xl-lm-adapt/T0Mixture/copa/1000/initial_similarities/1000_copa_T0Mixture_google-t5-xl-lm-adapt_42.json does not exist
. I looked into the code and noticed that the error is caused here however the file is assumed to be there apriori. How does one create this file?This also extends to other runs such as the UCB1 run. There is a command to explicitly create the gradients for all datasets separately however I thought it would be run implicitly based on the README. Do let me know if I need to run it separately.
The text was updated successfully, but these errors were encountered: