[Feedback/Enhancement] Unable to render key "xxxx" of type "string"... message in the Dashboard

[neomatrix369]

• Weights and Biases version: 0.8.31
• Python version: 3.7.2
• Operating System: MacOS High Sierra

Description

Using the wandb.log() call, I have been sending data for param tracking of the models I'm currently working on, and I get this in the Dashboard:

I think this is partially user error, but then there are a few scenarios when non-numeric data is passed to W&B.

Scenario 1: currently it's not plotted and we get the above messages
Scenario 2: user can be warned about it in a separate tab (list of errors/warnings tab and/or logs) and not show them at all in the dashboard
Scenario 3: not flag them as error/warning but rather collect them and show some sort of a histogram or matrix chart plotted against time (just like the other metrics shown in the dashboard). This is super useful as we may have non-numeric data that we want to see the spread and frequency across our training, I can already think of its use with NLP and related topics.
Scenario 4: combine the above in different forms and abstract them behind flags or settings so the user can toggle to the kind of behaviour they want, and the default should be some warning in the logs or in the dashboard or both.

What I Did

I removed the calls that involved sending non-numeric data to W&B and instead passed those via the wandb.config calls and then started getting this, which is desired:

Although would be good to have the scenarios discussed as mentioned in the above section.

[cvphelps]

Hi @neomatrix369, this is a very reasonable request. @altay will some of this be solved by panelbank?

[neomatrix369]

I'll like to send a list of single values or list of tuples with wandb.log() for e.g.:

Scenario 1

<looping>
    <each training loop>
    wandb.log({values: [1,2,3,4....10]}

or

Scenario 2

<looping>
    <each training loop>
    wandb.log({
            values1: [1,2,3,4....10],
            values2: [1,2,3,4....10],
            ...
     })

or

Scenario 3

<end of training>
wandb.log( { {a: 0.9},  {b: 0.7}, {c: 0.8}....{z: 0.10} } )

or

Scenario 4

<end of training>
wandb.log({
       values1: [1,2,3,4....10],
       values2: [1,2,3,4....10],
       ...
})

Will it or can the above be used to plot these values in the dashboard as a visualisation?

[vanpelt]

Hey @neomatrix369 how would you like to plot the array values? For the nested dictionary keys, that should work and the keys would be level_1.level2.etc.

[neomatrix369]

Hey @neomatrix369 how would you like to plot the array values? For the nested dictionary keys, that should work and the keys would be level_1.level2.etc.

The nested dictionary is fine workaround.

With wrt plotting array values, it would be great to see them as bar, stacked-bar or line graphs - would this work. Showing the delta of change in the values can also be useful.

I can do this manually via the dashboard with a bunch of workarounds but its much smoother to do it via giving it values like this and the system figuring out how to show it.

(I have updated the scenarios above and given them names, see #971 (comment))

Is my answer clear and helping understand the original issue?

[neomatrix369]

I also found Histogram under this https://docs.wandb.com/library/log#rich-media - this might solve a part of my usage above (for continuous values), but seeing a line or a stacked bar chart would be awesome, especially for the individual dictionary elements.

[ariG23498]

Hey @neomatrix369
In the past year we've majorly reworked the CLI and UI for Weights & Biases. We're closing issues older than 6 months. Please comment to reopen.

[neomatrix369]

Hey @neomatrix369
In the past year we've majorly reworked the CLI and UI for Weights & Biases. We're closing issues older than 6 months. Please comment to reopen.

Does it mean this feedback has been integrated into the Dashboard or CLI? How can I find out? The above enhancements would be great to have and also improves UI/UX for all users.

[vanpelt]

Hey @neomatrix369 we now handle these un-renderable in a much cleaner way. Give it a try and let us know if you have other suggestions for improving it further.

[neomatrix369]

Hey @neomatrix369 we now handle these un-renderable in a much cleaner way. Give it a try and let us know if you have other suggestions for improving it further.

Ok cool @vanpelt I will soon have a look at it, a couple of projects may need some visualisations (and I will use W&B for them)