Features: WandB logging part as a Hydra option

[chrstnkgn]

What

To Add

• Change WandB logging part as a Hydra option to:
  • make WandB logger able to be turned on/off
  • automatically assign experiment name when running the script
  • Add option to use WandB when running the experiment without Ray

Why

• Initially, I added a WandB logging option supported by Ray to keep track of the experimental results
• Then found out that it would be nicer (in terms of both convenience and code clarity) to make WandB optional regardless of the usage of Ray

How

• WandB option into Hydra, default setting: ON
• WandB option when not using ray
• Come up with WandB convention (How to set the project name?) -> Any opinion or discussion would be appreciated

[kdg1993]

So sorry for bothering you by replying to the closed issue 😭 but I think this issue is the most proper way to ask

First of all, I think that add log config directory and make wandb.yaml file is a brilliant move!!

After seeing that, two questions suddenly came to my mind

1. Is there any plan to make a default.yaml in the logging direction under config directory?
2. When wandb logging is turned off and the raytune also, then best_model.pth is the only thing what we got for now
   So, is there any blueprint for logging things in this circumstance? (Since you are pretty much used to handling this category, I ask it just for brainstorming)

[chrstnkgn]

Thank you for having such queries 😃 any kind of discussion is welcomed!

1. I don't think I understood the meaning of this question fully, but if my understandings are correct, we're defining logging directory in config.yaml for now (which works as a default config file) but it might change in the future as we are planning to conform the structure of our config files into pl-hydra template-like structure.
2. I think we can make a simple callback function that can save the outputs as a log either in json or csv, but I don't have a detailed plans for it yet. any suggestions and discussions would be really helpful.

[kdg1993]

I'm so grateful for these kind answers @chrstnkgn

1. It was my mistake that I didn't consider pl-hydra future plan before asking! Thanks again for reminding me the blueprint of this structure and the answer is very helpful
2. I considered simple things to add such as the last epoch model kind of things but the callback is what I didn't consider. Again your answer inspired me a lot but since my lack of experience in making a callback, if you find a good reference or plan, please let me know about it