[Feature]: ability to specify panel configurations in python API

[verbose-void]

Description

for example, every time i want to create a composite panel (let's say i am tracking 3 metrics, mean, max, and min of some value), i have to go in by hand every time i create a new project to define these panels.

same thing with stuff like specifying the x-axis reference values (like if im tracking evaluation and training steps seaparately, i have to go manually select the x-axis to use for every single panel).

EDIT: bonus points if you can also do this same thing but for the "all runs" view (ie. specify figures for how to compare k experiments within a project)

Suggested Solution

probably something like this:

# create an aggregate panel
wandb.panels.configure(
    "train/aggregates",
    type=wandb.line_plot,  # or a str or something
    y_sources=["train/loss_mean", "train/loss_max", "train/loss_min"],
    name="Loss Statistics",
)

# now maybe we want all "train/*" logs to use the "train/steps" numbers as the x-axis?
wandb.panels.configure("train/*", x_source="train/step")  # excludes "train/step" from x-axis modifications

# logging the metrics remains the same:
wandb.log({
    "train/loss_mean": loss.mean().item(),
    "train/loss_max": loss.mean().item(),
    "train/loss_min": loss.mean().item(),
    "train/step": epoch_idx, 
})

ideally a wandb.panels interface would have more than just these features, as it would be nice to be able to do everything from python that we can do from the UI.

i'm just tired of having to go through and click 100 times in the UI for every new wandb project.

Alternatives

alternatively, maybe you could make it so that you can "copy-panels" from a different project name. creating multiple experiments under the same project works that way.

Additional Context

the goal would be to allow people to specify everything they can in the UI-interactions through the python interface.

[sergiynesterenko90]

Yes please! I've wanted to do exactly the same thing. I was manually rearranging panels every single time I ran a new run in order to track what I really cared about. Would have been much better to have it be part of my config script somehow.

[MBakirWB]

Hi @verbocado, have you considered utilizing custom charts to create a preset panel you can repeatedly use with new runs? This would be the most direct approach here. If custom charts are not ideal use case here, could you speak to the importance of the feature request to your workflow and how custom charts aren't best approach and we'll include your feedback to the feature request. Thanks

[JoanaMarieL]

Hello @verbocado , following up here if you've manage to try utilizing custom charts? :) Thank you~

[verbose-void]

@JoanaMarieL i haven't tried it yet, but it looks promising!

[morganmcg1]

Hey @verbocado , I think the Reports API might help you here too potentially. See the "SDK" tabs in the different sections of the Reports docs

https://docs.wandb.ai/guides/reports/create-a-report

[JoanaMarieL]

Hi @verbocado , since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!