Add context formatter option to DatetimeTickFormatter

[RiccardoGiro]

The current default formatting scheme for datetime axes seems a little confusing (in my opinion), especially during zooming operations: in particular, one gradually loses the reference information about the current year/month/day displayed on the datetime axis. This aspect becomes particularly critical when a transition interval between years/months/days needs to be represented.
To give further insight, a comparison between MATLAB's default datetime axis formatting scheme (which I believe provides clearer information with respect to Bokeh) and Bokeh is shown below.

On a multiple-year time scale, there are no problems at all:

In the following instance, both libraries still function properly:

At this point, Bokeh axis formatting starts to become unclear, hiding information about the year. The latter should instead be displayed, since the data points to the left are related to December 2019, while the right ones refer to January 2020. Moreover, Bokeh datetimes are ambiguously represented (it could be dd/mm or even mm/dd):

Similar to the previous case. It should be noted that MATLAB recognizes that the datetime values are all related to the same year: therefore, such quantity is automatically displayed in the lower right corner, in order to provide the reader a clear reference. It would be nice to have such a feature in Bokeh, too:

At this point, Bokeh-generated plots are borderline unreadable. The user cannot understand which is the true date/time represented on the x-axis, leading to intepretation ambiguities:

If one increases even more the granularity level, visualization gets even worse in Bokeh. On the contrary, MATLAB keeps displaying in the bottom-right corner the necessary information in order to understand the date/time represented in the plot, and such information is automatically adapted according to the current time scale (on a hour/minute level, one must provide: day, month and year):

Do you think it's reasonable to implement a few changes and improve the current datetime formatting to make it more readable?

[jbednar]

Thanks for the detailed analysis! I'd love to see functionality like that in Bokeh.

[bryevdv]

Hi @RiccardoGiro thanks for the issue. I've had similar unformed thoughts over the years but surprisingly you are the first person to raise an actual issue, and user submitted issues are an important barometer of minimum urgency, so nothing has really happened to date.

I'd first only mention that the date time axis tries to do something reasonable across a wide range of scales. From folks working with historical data over centuries, to experimentalists getting sensor datetime timestamps on the order of picoseconds (and who don't care about seeing the year, naturally). Of course, satisfying every case with one tool is impossible which is why Bokeh often relies on making flexibility and configurability available to users. So, along those lines, all of the tick formats shown above from matlab are achievable with Bokeh, just require some configuration on the user's part.

That said I agree that most everything a move can use some improvement, especially the MM/YY scale format. There's a range of options that could be take (together or separately):

• Improve some of the existing defaults so that they are better, i.e. these properties described on DateTimeTickFormatter. This could be done immediately with no additional feature work. As a matter of policy it should probably land on a minor release, not a point release. 2.1 is coming up in the next ~6 weeks. Would you like to submit a PR to update some of the default property values?

• Add feature work to support including additional "context" for axes. I will describe that in a separate reply.

[bryevdv]

Ok, so what about the "extra context" idea. Bokeh axes are already some of the most complicated parts of a Bokeh plot. My main criteria is to do things in a flexible, modular way (that also does not make any breaking API changes for 2.x series). Here are some off-the-cuff thoughts:

• As a matter of internal implementation, have all tick formatters report an (optional) formatted tick context, i.e. for month-day scales, a formatted year could be reported as the context. By default most tick formatters would report null.

• For Datetime tickers, there could be a property to specify a format for the tick context. This could be single format that is used across all scales, or we could allow every scale on a datetime ticker so specify and independent format for each scale (e.g. on month-day you might have a year as the context, bit on hour-day you might show a full date as the context)

• a CustomJSTickContextFormatter might be nice to allow people to do very sophisticated things.

Ok so that would handle the "data" side of things, now where do we pipe this information to, i.e. how best to display it? I think here we should try to be as flexible as possible. Some people might want the context at the bottom of the axis. Some might want an annotation in the plot above the axis. I don't have well formed thoughts here. We could add a TickContext annotation to a plot, that gets configured with an axis, and looks for the information to display. Or we could attach something directly to an axis that is responsible for displaying tick context. Absent other consideration I think I'd prefer the first, but in terms of making it easy to provide sensible automatic defaults I think it might have to be the second.

[jbednar]

All the above discussion sounds reasonable and appropriate to me.

all of the tick formats shown above from matlab are achievable with Bokeh, just require some configuration on the user's part.

Just to highlight a specific issue with the configurability, I've found that customizing the formatters only helps at a particular scale (whether it's a timescale or whatever), because the appropriate formatting (as illustrated in those examples) differs depending on the scale, which varies dramatically every time a user zooms. So one thing that could be done (unless it's there and I missed it) is to make the custom formatting conditional on an expressed range. I.e. can we easily say that "for an x axis range between 0.001 and 0.01, use this custom formatting"? Being able to express such conditional customizations easily would help relieve a lot of the pressure on getting things exactly right at the default-behavior level (though I agree with the OP that the default behavior could be much better along the lines shown above).

[bryevdv]

So one thing that could be done (unless it's there and I missed it) is to make the custom formatting conditional on an expressed range. I.e. can we easily say that "for an x axis range between 0.001 and 0.01, use this custom formatting"?

Maybe you can expand on this? This seems to me to be exactly what the different scales of the DatetimeTickFormatter already do, i.e. they apply different formats, depending on the overall range start/end that is currently active. Are you saying you want to be able to specify different start/end ranges (or add additional ones)? Or are you here referring to the tick context idea?

If you mean the former, there is not currently any "adaptive" tick formatter that is an analogue to AdaptiveTicker that is the basis of picking nice ticks at different scales. But, it also seems reasonable that that functionality could be factored out of DatetimeTickFormatter into an AdaptiveTickFormatter that you could configure with your own scales with their own sub-formatters.

[jbednar]

I'm not referring to the tick context idea (though that's a great idea!). Yes, the existing different scales already do that, it's just that I have very often wanted to override what it does (not just for Datetimes, but for any TickFormatter) in a specific range, where the default behavior for that range was not appropriate for my use case. E.g. I often want to set the number of digits of precision on a floating-point axis, but only when zoomed in enough for that to be relevant. I know how to override the behavior for all ranges at once, but when that's not appropriate I don't know how to specify custom behavior that applies just in a specific range. So yes, I think factoring that out would help a lot!

[RiccardoGiro]

Improve some of the existing defaults so that they are better, i.e. these properties described on DateTimeTickFormatter. This could be done immediately with no additional feature work. As a matter of policy it should probably land on a minor release, not a point release. 2.1 is coming up in the next ~6 weeks. Would you like to submit a PR to update some of the default property values?

Sure, I will make a PR about that.

As a matter of internal implementation, have all tick formatters report an (optional) formatted tick context, i.e. for month-day scales, a formatted year could be reported as the context. By default most tick formatters would report null.

For Datetime tickers, there could be a property to specify a format for the tick context. This could be single format that is used across all scales, or we could allow every scale on a datetime ticker so specify and independent format for each scale (e.g. on month-day you might have a year as the context, bit on hour-day you might show a full date as the context)

I think that makes perfectly sense. It would be nice to provide the user a set of default values which can be easily accessed and modified, for instance:

from bokeh.models.formatters import DatetimeTickFormatter as dtf


# Examples of access and modification of a specific context value
dtf.months.context = ['%Y'] # This would print a partial date context, formatted as "2020"
dtf.minutes.context = ['%b %d, %Y'] # This would print a full date context, formatted as "Jan 15, 2020"

Ok so that would handle the "data" side of things, now where do we pipe this information to, i.e. how best to display it?

I think that the best positions to display the context are either on the bottom right or left corners of a figure. In general, there is not much free space available and the possible solutions are limited, as:

1. The central body is used to display the plot;
2. The upper space is already taken up by the title; the latter should always be visible and separated from everything else, since its main purpose is to univocally identify what is being displayed in the figure;
3. The right side doesn't seem a particularly appealing place, as it is often occupied by colorbars and such;
4. Concerning the left and bottom sides, axes and the corresponding tick labels are drawn along their whole length, while their centers are used to display the measurement units.

For the specific case of displaying a datetime tick context, I would suggest the bottom-left corner, since time is usually represented along the horizontal axis and it is read from left to right. However, displaying the context in such a place might be clogging up too much that corner (already occupied by tick labels and by the origin of the axes), so this option must be evaluated carefully. For everything else, instead, the right corner seems to be more appropriate.

[RiccardoGiro]

I have recently managed to find a solution to this problem. Below you can find some examples which display Bokeh plots with MATLAB-style datetime tick formatting:




You can find below a prototypal example of the code which updates the DateTime ticks. The function is automatically triggered whenever the limits on the horizontal axis change. fig is a handle to a Bokeh figure and xlim is a 2-element vector containing Pandas Timestamps, which correspond to the currently detected limits on the horizontal axis. The "context" string needs to be declared after having generated the figure and after having plotted the curves of interest, using the following line of code: fig.add_layout(Title(text='', align='right', text_font_style='normal'), 'below')

[carve11]

@RiccardoGiro I very much support your suggestion. I was thinking if the "context" string should be on a new line just below the very first tick label? Is there a risk of overlooking the context string if it is below the axis label? In the figure below I just tried to play around with FuncTickFormatter to illustrate what I mean.

from bokeh.models import FuncTickFormatter
from bokeh.plotting import figure, output_file, save
import pandas as pd

output_file("date_axis_FuncTick.html")

p = figure(width=600, height=200, x_axis_type = 'datetime')
p.circle(
    x = pd.date_range("2018-01-01", periods=4, freq="H"),
    y = [6, 2, 4, 10],
    size = 30
    )
p.xaxis.axis_label = 'Date'
p.xaxis[0].formatter = FuncTickFormatter(code="""
    const dateObjectName = new Date(tick);
    const date_str = dateObjectName.toISOString().match(/(\d{4}\-\d{2}\-\d{2})T(\d{2}\:\d{2})/);

    if (tick == ticks[0]) {
        const newTick = date_str[2] + '\\n' + date_str[1];
        return newTick
    } else {
        return date_str[2];
    }
""")

save(p)

[RiccardoGiro]

Coming from a MATLAB background, the best solution I prefer would be to have the context string at the same height as the label and placed on the rightmost part of the plot, but I could not find an easy way to do so; however, it's just personal taste. As far as your solution is concerned, I don't particularly enjoy having the context on the left side because it's too close to the origin of the axes, and in my opinion tends to clog too much a delicate area of the figure. Good job anyway! :)

[bryevdv]

@carve11 I really want to thank you for your comment here. Multi-line tick labels are a fairly recent addition to Bokeh, and I had not made any connection between them, and this issue.

The code for "nice" datetime ticking and spacing is ~15 years old (ported from Chaco) and has not really been touched much in almost a decade. Not wanting to disturb any of that is probably the main reason no-one has yet dug into this. But your approach above suggests a much simpler path that should be straightforward to implement. I've tentatively triaged this for the 3.0 release.

[bryevdv]

For concreteness, what I plan to explore: Adding a context property to DatetimeTickFormatter [1] that can accept a secondary formatter, plus a few other properties to control the positioning of the extra context (i.e. which tick(s) to have context lines added below).

[1] or possibly more generally, a context per-scale