-
Notifications
You must be signed in to change notification settings - Fork 7.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Visualizations on W&B #3404
Comments
@ppwwyyxx what do you think about this? Would this be a useful add-on? |
Would be useful for me! I ended up adding it myself to debug some models. |
Hi @nbardy would you mind sharing the code on how you added this? |
@BenSpex Im unfortunately busy focused on synthesis. But looking at detectron 2 integration next quarter. I'm happy to upstream logging when I iron it out. Their Fully Connected forums often have examples of community integrations. |
@nbardy awesome, thanks for coming back. let me know when you need someone to test the integration. |
@BenSpex responded to you in the W&B forum :) |
Hi all, the visualizations look pretty awesome and would be a great addition to detectron2! |
@ppwwyyxx thanks. What would be the best way to share the code. Should I open a WIP draft PR to facilitate easier disscussion? I'm happy to do it via other mediums if you prefer. Regarding changes - I think the only thing that is needed to log these visualizations efficiently is some predictions to be stored in event storage after evalhook is called. |
Yeah a draft PR could be a good starting point. If what is needed is just to access the predictions, it seems a better approach is to implement a new evaluator (subclass of |
@ppwwyyxx I've made a draft PR with a design proposal. Sorry for the dealy, was waiting on some UI changes |
🚀 Feature
Allow visualization of training progress using media panels and tables.
Motivation & Examples
This is based on this issue.
I'm an engineer at W&B and I've been working with object detection tasks. I regularly use some of the following visualizations on W&B dashboards. Would the following features be useful to other detection users? I'd like to know what the maintainers think.
Bounding box & Segmentation maps debugger
W&B supports interactive media panels, where you can track how training progresses by adjusting the steps, confidence scores, and classes of predictions in real-time.
![ezgif com-gif-maker (2)](https://user-images.githubusercontent.com/15766192/130792944-77c59dbd-861b-4161-9c62-5f71e44bac7e.gif)
Try it live here
The Media panel also supports debugging segmentation maps.
![semantic segmentation (1)](https://user-images.githubusercontent.com/15766192/130792997-887b706c-13b2-4f18-8e7b-10ded88debfa.gif)
Dataset visualization and versioning
With W&B tables, you can visualize, query, and filter your datasets in your browser.
Quickly compare results across different training epochs, datasets, hyperparameter choices, model architectures etc. For example, take a look at this comparison of two models on the same test images →
Model versioning and DAGs
![ezgif com-gif-maker (1) (1)](https://user-images.githubusercontent.com/15766192/130798266-0647a420-f5bd-442e-b556-d022f65fbdf3.gif)
We can have the user set
Based on that, we'll log models after every
model logging period
with aliasbest
if the current model performs best on thedesired metric
The text was updated successfully, but these errors were encountered: