Confidence intervals in python plotly

[jatinrajani]

Hi is there any method for plotting confidence interval lines in plotly in scatterplot in python.I am not able to find anything relevant to it in docs and issues

[nicolaskruchten]

There is no built-in confidence interval feature, no.

[MarcoGorelli]

@jatinrajani here's something I've come up with which kinda matches seaborn:

COLOR_PALETTE = [
    (0.12156862745098039, 0.4666666666666667, 0.7058823529411765),
    (1.0, 0.4980392156862745, 0.054901960784313725),
    (0.17254901960784313, 0.6274509803921569, 0.17254901960784313),
    (0.8392156862745098, 0.15294117647058825, 0.1568627450980392),
    (0.5803921568627451, 0.403921568627451, 0.7411764705882353),
    (0.5490196078431373, 0.33725490196078434, 0.29411764705882354),
    (0.8901960784313725, 0.4666666666666667, 0.7607843137254902),
    (0.4980392156862745, 0.4980392156862745, 0.4980392156862745),
    (0.7372549019607844, 0.7411764705882353, 0.13333333333333333),
    (0.09019607843137255, 0.7450980392156863, 0.8117647058823529),
]


def lineplot(df, x, y, hue=None, color_palette=None):
    def plot_hue(df_, name, label, color):
        rgb = f"rgba({color[0]*255}, {color[1]*255}, {color[2]*255}"
        fill_rgba = f"{rgb},0)"
        line_rgba = f"{rgb},1)"
        fig = px.line(
            df_,
            x=x,
            y="value",
            color="variable",
            color_discrete_map={f"{y}_lower": fill_rgba, f"{y}_upper": fill_rgba},
        )

        fig.update_traces(
            name="interval", selector=dict(name=f"{y}_upper"), showlegend=False
        )
        fig.update_traces(fill="tonexty")
        if name is None:
            fig.update_traces(
                fillcolor="rgba(0,0,0,0)",
                selector=dict(name=y),
                line_color=line_rgba,
                showlegend=False,
            )
        else:
            fig.update_traces(
                fillcolor="rgba(0,0,0,0)",
                selector=dict(name=name),
                name=label,
                line_color=line_rgba,
            )
        fig.update_traces(
            fillcolor="rgba(0,0,0,0)",
            showlegend=False,
            selector=dict(name=f"{y}_lower"),
        )
        return fig

    if hue is None:
        group = [x]
    else:
        group = [hue, x]
    err = df.groupby(group)[y].std() / np.sqrt(df.groupby(group)[y].size())
    pdf = df.groupby(group)[y].mean().reset_index()
    pdf[f"{y}_upper"] = pdf[y] + 1.96 * pdf.set_index(group).index.map(err)
    pdf[f"{y}_lower"] = pdf[y] - 1.96 * pdf.set_index(group).index.map(err)
    pdf = pd.melt(pdf, id_vars=group)

    if hue is None:
        ascending = [True, True]
    else:
        ascending = [False, True, True]
    pdf = pdf.sort_values(group + ["variable"], ascending=ascending)

    figs = []
    if color_palette is None:
        color_palette = COLOR_PALETTE

    if hue is not None:
        for hue_, color in zip(pdf[hue].unique(), color_palette):
            df_ = pdf[pdf[hue] == hue_].copy()
            figs.append(plot_hue(df_, y, hue_, color))
    else:
        figs.append(plot_hue(pdf, None, None, color_palette[0]))

    data = figs[0].data
    for fig in figs[1:]:
        data = data + fig.data
    figure = go.Figure(data=data)

    if hue is None:
        figure.update_layout(
            xaxis_title=x,
            yaxis_title=y,
        )
    else:
        figure.update_layout(
            xaxis_title=x,
            yaxis_title=y,
            legend_title=hue,
        )
    return figure

Example:

fmri = sns.load_dataset("fmri")
lineplot(fmri, x='timepoint', y='signal', hue='event')

Comparison with seaborn:

sns.lineplot(data=fmri, x="timepoint", y="signal", hue="event")

@nicolaskruchten might something like this be considered in plotly.express?

[nicolaskruchten]

This is a nice function!

At the moment, the small number of folks who maintain Plotly.py (i.e. just me) don't really have the bandwidth to add this kind of thing to PX, or to QA it or document it or maintain it in the long-run, so we're considering this kind of statistical manipulation to be out of scope for this library. The trendline functionality is about as far as we're planning on going in this direction for now.

[MarcoGorelli]

No worries, thanks for your reply!

I've put this in a little package, in which I'll put some other functions to mimic one-liners from seaborn, as soon as I have permission from employer I'll open source it

[acxz]

Sorry if this is a necrobump, but came across this issue randomly and thought I'd share my own solution to this:

fig_simp.add_trace(
        go.Scatter(
            x=torch.cat([predict_input_data[:, 0],
                         torch.flip(predict_input_data[:, 0], [0])]),
            y=torch.cat([upper[:, 0], torch.flip(lower[:, 0], [0])]),
            mode='lines',
            line={
                'color': 'rgba(255, 255, 255, 0)',
            },
            fillcolor='black',
            opacity=0.2,
            fill='toself',
            name='confidence region',
            showlegend=True
        )
    )

Essentially you create two lines in the trace and then you use the fill='toself' option to fill the area between the lines with the specified fillcolor and opacity. You also make sure to set the color of the line to be transparent via 'color': 'rgba'(255, 255, 255, 0)'. The transparency (alpha value) is determined by the last value in the rgba list. Setting it 0 makes the line fully transparent. Another weird thing to keep in mind is that the way the line is drawn it essentially wraps around the xaxis. What I mean by that is that if your xaxis is from 0-10, and you have two lists representing the upper and lower confidence lines, they should be concatenated together like so: [0 - 10, 10, 0] and the corresponding yaxis values (i.e. upper and flipped lower confidence line). That is why you see the calls to flip and then the concatenation in the above snippet. Sounds weird but it does the job in a pretty quick way, haha.

Here is how it looks:

For the full code see: https://github.com/acxz/pl-utils/blob/master/examples/gp_visualization.py#L84-L100

[gvwilson]

Hi - we are trying to tidy up the stale issues and PRs in Plotly's public repositories so that we can focus on things that are still important to our community. Since this one has been sitting for several years, I'm going to close it; if it is still a concern, please add a comment letting us know what recent version of our software you've checked it with so that I can reopen it and add it to our backlog. Thanks for your help - @gvwilson

[acxz]

Looks like this issue has been resolved a long time ago. See: 9fe9868 and the current webpage: https://plotly.com/python/continuous-error-bars/