Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

W&B config warnings when training a sweep agent #78

Open
Adamits opened this issue Jun 29, 2023 · 0 comments
Open

W&B config warnings when training a sweep agent #78

Adamits opened this issue Jun 29, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@Adamits
Copy link
Collaborator

Adamits commented Jun 29, 2023

Currently, when training a sweep agent, we need to start the agent within a hyperparameter config, which automatically logs all of the hyperparameters to W&B. We also need to initialize the pytorch-lightning WandbLogger, which under the hood attempts to log the config again. See here for details: wandb/wandb#2641.

Ideally, we would solve this by not updating twice, however, this may be out of our control as we rely on wandb and pytorch-lightning for those behaviors.

A work-around is to suppress the W&B warning message. So far, warnings.filterwarnings does not work -- but we should investigate this more.

@Adamits Adamits added the bug Something isn't working label Jun 29, 2023
@kylebgorman kylebgorman changed the title W&B config update warning when training a sweep agent. W&B config warnings when training a sweep agent Jun 29, 2023
kylebgorman added a commit that referenced this issue Jul 20, 2023
As I understand it this is not the warning discussed in #78 but it's
good to silence anyways.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant