ValueError: Field "duration" has type "timedelta64[ns]" which is not supported by Altair.

[CiaranHaines]

It worked fine with 2 columns of datetimes and 2 columns of integers. I used df.apply(f(x)) to create a timedelta column and got the following warning. Full text:

/opt/conda/lib/python3.7/site-packages/IPython/core/formatters.py:918: UserWarning:
Unexpected error in rendering Lux widget and recommendations. Falling back to Pandas display.
Please report the following issue on Github: https://github.com/lux-org/lux/issues

/opt/conda/lib/python3.7/site-packages/lux/core/frame.py:632: UserWarning:Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/lux/core/frame.py", line 594, in ipython_display
self.maintain_recs()
File "/opt/conda/lib/python3.7/site-packages/lux/core/frame.py", line 451, in maintain_recs
self._widget = rec_df.render_widget()
File "/opt/conda/lib/python3.7/site-packages/lux/core/frame.py", line 681, in render_widget
widgetJSON = self.to_JSON(self._rec_info, input_current_vis=input_current_vis)
File "/opt/conda/lib/python3.7/site-packages/lux/core/frame.py", line 721, in to_JSON
recCollection = LuxDataFrame.rec_to_JSON(rec_infolist)
File "/opt/conda/lib/python3.7/site-packages/lux/core/frame.py", line 749, in rec_to_JSON
chart = vis.to_code(language=lux.config.plotting_backend, prettyOutput=False)
File "/opt/conda/lib/python3.7/site-packages/lux/vis/Vis.py", line 334, in to_code
return self.to_vegalite(**kwargs)
File "/opt/conda/lib/python3.7/site-packages/lux/vis/Vis.py", line 310, in to_vegalite
self._code = renderer.create_vis(self)
File "/opt/conda/lib/python3.7/site-packages/lux/vislib/altair/AltairRenderer.py", line 99, in create_vis
chart_dict = chart.chart.to_dict()
File "/opt/conda/lib/python3.7/site-packages/altair/vegalite/v4/api.py", line 373, in to_dict
dct = super(TopLevelMixin, copy).to_dict(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 328, in to_dict
context=context,
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 62, in _todict
for k, v in obj.items()
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 63, in
if v is not Undefined
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 58, in _todict
return [_todict(v, validate, context) for v in obj]
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 58, in
return [_todict(v, validate, context) for v in obj]
File "/opt/conda/lib/python3.7/site-packages/altair/utils/schemapi.py", line 56, in _todict
return obj.to_dict(validate=validate, context=context)
File "/opt/conda/lib/python3.7/site-packages/altair/vegalite/v4/api.py", line 363, in to_dict
copy.data = _prepare_data(original_data, context)
File "/opt/conda/lib/python3.7/site-packages/altair/vegalite/v4/api.py", line 84, in _prepare_data
data = _pipe(data, data_transformers.get())
File "/opt/conda/lib/python3.7/site-packages/toolz/functoolz.py", line 627, in pipe
data = func(data)
File "/opt/conda/lib/python3.7/site-packages/toolz/functoolz.py", line 303, in call
return self._partial(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/altair/vegalite/data.py", line 19, in default_data_transformer
return curried.pipe(data, limit_rows(max_rows=max_rows), to_values)
File "/opt/conda/lib/python3.7/site-packages/toolz/functoolz.py", line 627, in pipe
data = func(data)
File "/opt/conda/lib/python3.7/site-packages/toolz/functoolz.py", line 303, in call
return self._partial(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/altair/utils/data.py", line 149, in to_values
data = sanitize_dataframe(data)
File "/opt/conda/lib/python3.7/site-packages/altair/utils/core.py", line 317, in sanitize_dataframe
"".format(col_name=col_name, dtype=dtype)
ValueError: Field "duration" has type "timedelta64[ns]" which is not supported by Altair. Please convert to either a timestamp or a numerical value.

[CiaranHaines]

Forgot the context: I was writing this into a notebook on Kaggle.com. See Notebook here, with issue occuring in the penultimate codeblock for the display of toy_times after the assignment to toy_times['duration']

[dorisjlee]

Hi @CiaranHaines,

Thank you for reporting this problem and sending the notebook to help us reproduce the error!
It looks like Altair (i.e., the plotting backend that Lux uses) does not support plotting of the time duration data type (timedelta64[ns]). There are two quick ways we can work around the problem for now:

1. You can switch the plotting backend to matplotlib adding the following code before you create the toy_times["duration"] column:

lux.config.plotting_backend="matplotlib"

You can see that the time duration here is plotted as its own bar, which is not very understandable.

2. You can convert the duration column into seconds (as an integer). This way, Lux should be able to visualize the numbers for you by default. We can do this by adding .dt.seconds in your column definition to convert the timedelta resulting from the sum to number of seconds.

toy_times['duration'] = toy_times.apply(lambda x: sum_time(x['start'], x['end']), axis=1).dt.seconds
toy_times

You can see now that there is a distribution of duration times using the second approach.
Let us know if this resolves the issue for you!

[dorisjlee]

Note for dev: Improve error message when Altair is unable to handle custom datetime types (e.g., timedelta, interval). Possibly look into making automatic conversions to display visualizations.

[CiaranHaines]

Cheers xxxx

…

On Wed, 15 Sep 2021, 18:11 Doris Lee, ***@***.***> wrote: Hi @CiaranHaines <https://github.com/CiaranHaines>, Thank you for reporting this problem and sending the notebook to help us reproduce the error! It looks like Altair (i.e., the plotting backend that Lux uses) does not support plotting of the time duration data type (timedelta64[ns]). There are two ways to work around the problem: 1. You can switch the plotting backend to matplotlib adding the following code before you create the toy_times["duration"] column: lux.config.plotting_backend="matplotlib" [image: Screen Shot 2021-09-15 at 10 05 12 AM] <https://user-images.githubusercontent.com/5554675/133477891-c9a8398e-3fc4-4e60-909e-178530dc6609.png> You can see that the time duration here are plotted as its own bar, which is not very understandable. 2) You can convert the duration column into seconds (as an integer). This way, Lux should be able to visualize the numbers for you by default. We can do this by adding .dt.seconds in your column definition to convert the timedelta resulting from the sum to number of seconds. toy_times['duration'] = toy_times.apply(lambda x: sum_time(x['start'], x['end']), axis=1).dt.secondstoy_times [image: Screen Shot 2021-09-15 at 10 10 28 AM] <https://user-images.githubusercontent.com/5554675/133478597-c8fceebb-d938-445f-ac27-603d05c0150a.png> You can see now that there is a distribution of duration times using the second approach. Let us know if this resolves the issue for you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#419 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVT6ETMVC66Z5LSN7PNMMSDUCDHVHANCNFSM5D77A4RA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.