DataTable does not display the last row

[mitchm17]

bokeh: 1.0.2
python: 3.7
browser: Microsoft Edge, Chrome, Internet Explorer

Expect the DataTable widget to show all rows of data, however, the last row is cut off by the horizontal scroll bar and there is no way to see the last row. The only way to see it is to add a row of blanks to the data source, which as you can imagine is not great.

Below is my minimal example code. I am using Tabs, but for simplicity sake I did not include the full code for tab1. I'll just make it a Spacer widget.
I am using bokeh serve to run this because there is interactivity going on in the code.

from random import randint, seed

def start_visualization():
    # ----- Sample Data Initialization -----
    seed(123456)
    data = dict(names = [f"John Smith_{i}" for i in range(1,31)]+[''],
                ages  = [randint(18, 60) for i in range(30)]+[''],
                weight= [randint(160,260) for i in range(30)]+['']
                )
    dtypes  = {col: list(set([type(x) for x in data[col] if x != ''])) 
               for col in data}
    
    # ----- Main Tab Data Table -----
    source  = ColumnDataSource(data)
    columns = [TableColumn(field="names", title="Patient Name"),
               TableColumn(field="ages", title="Patient Age"),
               TableColumn(field="weight", title="Patient Weight")]
    
    data_table = DataTable(source=source, columns=columns, 
                           reorderable=True, scroll_to_selection=True,
                           width=450, height=300 
                           )
    # SideNote: reorderable is not working...

    # ----- Layout Setup -----
    button_row    = row(add_data, Spacer(width=50), delete_data)
    mainLayout    = column(button_row, data_table)
    report_layout = column(Spacer(width=50))
    tab1          = Panel(child=report_layout, title="Run a Report")
    tab2          = Panel(child=mainLayout, title="Main Tab")
    layout        = Tabs(tabs=[tab1, tab2])
    curdoc().add_root(layout)
# end def start_visualization

start_visualization()

Changing the height doesn't help anything unless I change the height to show ALL rows. I also tried fit_columns, but that didn't fix the problem either, and there are a lot more columns than what is included in the example.

I would be very grateful to anyone who can provide a solution!

[mitchm17]

This could maybe be fixed by allowing the user to scroll vertically past the last row, like another 5 or 10 rows of height...

[bryevdv]

@mitchm17 Thank you for the report. In order to work an issue, we really must have a complete minimal test case submitted to work against. Please update the issue when you have a chance. Additionally, it is possible that #8085 already fixed this, can you test against dev build 1.1dev2 or later? https://bokeh.pydata.org/en/latest/docs/installation.html#developer-builds

[mitchm17]

My code uses another script, and the main.py script is over 400 lines, do you want me to include all of that? Or I can attach the files, would that be better?

I'll test this agains the newest dev build. Do you know how to get the newest build in anaconda?

[bryevdv]

@mitchm17 Dev builds are only published to a specific conda channel, not defaults. Installation instructions for dev builds are in the link above, or are you asking something different?

Regarding the code, in the ideal case it's possible to create a minimal example that just has enough code to demonstrate the issue and nothing else (or pare down some other code to that point). If that's not possible then a link to the code is next best, but in general the simpler and easier it is to reproduce an issue, the higher the chance it will be gotten to sooner.

[mitchm17]

Sorry, I missed that you included that link.

I followed the installation instructions above, using the conda install -c bokeh/channel/dev bokeh command, ran my code again and it is still the same issue.

It now says bokeh version is still 1.0.2

I hope the following is a complete minimal example...

# ----- Imports -----
import pandas as pd
from random import randint, seed


# ----- Bokeh Imports -----
from bokeh.models         import (
        Button,
        CheckboxGroup,
        ColumnDataSource,
        Div,
        MultiSelect,
        Panel,
        Spacer,
        Tabs,
        TextInput,
)
from bokeh.models.widgets import (
        DataTable,
        TableColumn
        )
from bokeh.plotting       import (
        curdoc,
        output_file,
        show
        )
from bokeh.layouts        import (
        column, 
        row, 
        )

# ----- Globals -----
global report_count, curr_report, name
report_count = 1
curr_report = -1
name = ''

def start_visualization():
    # ----- Main Tab Data Read-in -----
    output_file("sample.html")
    seed(123456)
    data = dict(names = [f"John Smith_{i}" for i in range(1,31)]+[''],
                ages  = [randint(18, 60) for i in range(30)]+[''],
                weight= [randint(160,260) for i in range(30)]+['']
                )
    dtypes  = {col: list(set([type(x) for x in data[col] if x != ''])) 
               for col in data}
    
    # ----- Main Tab Data Table -----
    source  = ColumnDataSource(data)
    columns = [TableColumn(field="names", title="Patient Name"),
               TableColumn(field="ages", title="Patient Age"),
               TableColumn(field="weight", title="Patient Weight")]
    
    data_table = DataTable(source=source, columns=columns, 
                           reorderable=True, scroll_to_selection=True,
                           width=450, height=300 
                           )

    # ----- Summary Report Tab -----
    filter_width   = 400
    summary_width  = 600
    summary_height = 400
    
    # ----- Reset Filters widget -----
    reset_filters = Button(label="Reset all filters", width=filter_width,
                           button_type="primary")
    def reset_all_filters():
        report_name.value  = ''
        save_report.active = []
        for filt in report_filters:
            if 'slider' in filt.name:
                filt.value = (filt.start, filt.end)
            elif 'multi' in filt.name:
                filt.value = ['--']
            else:
                continue
            # end if filt.name
        # end for filt
    # end def
    reset_filters.on_click(reset_all_filters)
    
    # ----- Filter directions div widget -----
    report_directions = Div(text="""Select values for each column listed below.
                            For categorical columns, use Ctrl+ to select more than one
                            category. For numerical columns, use the sliders to select
                            the minimum and maximum value for the column.
                            """, width=filter_width)
    
    # ----- Report Title Input Widget -----
    report_name  = TextInput(title="Enter the report name:", value='')

    save_report  = CheckboxGroup(labels=["Save report"], active=[])
    
    col_data = data['names']
    choices  = list(set(col_data))
    report_filters = [MultiSelect(title="Select the name(s) in 'names' that you want to search for:", 
                       value=['--'], options = ['--']+choices, 
                       name="names_multi_filter", 
                       width=filter_width, size=6)]
    
    def save_report_summary(stuff_to_pickle):
        saved_title = str(report_name.value)
        print(saved_title)
    # end def
    
    report_button = Button(label="Run the report summary", 
                           button_type="primary", width=filter_width)
    
    def report_summary():
        saved_report = []
        comp_title   = f"'{report_name.value}' Data: "
        comp_data    = pd.DataFrame(data)
        comp_data    = comp_data[comp_data != ''].dropna().reset_index(drop=True)
        
        for i, col in enumerate(data):
            if "multi" in report_filters[i].name:
                categories       = report_filters[i].value
                saved_report.append((str(report_filters[i].name), list(categories)))
                if categories == ['--']:
                    comp_title += f'all {col}, '
                    continue
                else:
                    comp_title += f'{col} that are in {categories}'
                    comp_data = comp_data[comp_data[col].isin(categories)]
                # end if categories
            else:
                saved_report.append((str(report_filters[i].name), tuple(report_filters[i].value),
                                     float(report_filters[i].start), float(report_filters[i].end)))
                comp_data[col] = comp_data[col].astype(float)
                minVal, maxVal = report_filters[i].value
                comp_title     += f'{col} between {minVal} and {maxVal}'
                # Check if the min and max vals are equal to the min and max of range slider,
                # if they are then there is no point in doing anything more for this filter
                if minVal == report_filters[i].start and maxVal == report_filters[i].end:
                    comp_title += ', ' if i != len(data)-1 else ''
                    continue
                # end if minVal, maxVal are at max range
                comp_data = comp_data[comp_data[col].between(minVal, maxVal)]
            # end if "multi"
            if i != len(data)-1:
                comp_title += ', '
            # end if
        # end for i, col
        
        if save_report.active:
            save_report_summary(saved_report)
        # end if
        
        stats_title             = Div(text=f"""<b>'{report_name.value}' 
                                      Summary Statistics</b>""", 
                                      width=summary_width)
        comp_title              = Div(text=f"""<b>{comp_title}</b>""", 
                                      width=summary_width)
        stats_table, comp_table = hf.get_summary_tables(comp_data, columns, dtypes, 
                                                        summary_width, summary_height)
        report_output           = row(Spacer(width=50), 
                                      column(stats_title, stats_table, 
                                             comp_title, comp_table))
        layout.tabs[0].child.children[1] = report_output
    # end def
    
    report_button.on_click(report_summary)
    
    # ----- Layout Setup -----
    mainLayout    = column(data_table)
    report_input  = column(reset_filters, report_directions, 
                           row(report_name, save_report), 
                           *report_filters, report_button)
    report_output = Spacer(width=summary_width)
    report_layout = row(report_input, report_output)
    tab1          = Panel(child=report_layout, title="Run a Report")
    tab2          = Panel(child=mainLayout, title="Main Tab")
    layout        = Tabs(tabs=[tab1, tab2])
    show(layout)
    curdoc().add_root(layout)
# end def start_visualization

start_visualization()

[bryevdv]

It now says bokeh version is still 1.0.2

Hrm, it should say 1.1dev2 However, with the new complete MRE I can easily test locally

[bryevdv]

@mitchm17 this is how things look on master:

Just to be sure the last row should be blank, I also printed the data:

{'names': ['John Smith_1', 'John Smith_2', 'John Smith_3', 'John Smith_4', 'John Smith_5', 'John Smith_6', 'John Smith_7', 'John Smith_8', 'John Smith_9', 'John Smith_10', 'John Smith_11', 'John Smith_12', 'John Smith_13', 'John Smith_14', 'John Smith_15', 'John Smith_16', 'John Smith_17', 'John Smith_18', 'John Smith_19', 'John Smith_20', 'John Smith_21', 'John Smith_22', 'John Smith_23', 'John Smith_24', 'John Smith_25', 'John Smith_26', 'John Smith_27', 'John Smith_28', 'John Smith_29', 'John Smith_30', ''], 'ages': [36, 19, 29, 18, 60, 22, 21, 35, 19, 25, 32, 58, 49, 29, 19, 51, 26, 42, 59, 41, 20, 29, 21, 46, 50, 46, 47, 50, 28, 51, ''], 'weight': [231, 165, 173, 260, 169, 172, 224, 245, 171, 170, 176, 203, 233, 198, 216, 225, 258, 239, 201, 207, 192, 191, 174, 194, 194, 170, 174, 187, 216, 250, '']}

so the blank fields appear to represent whats actually in the data.

So I am going to go ahead and close this, you should see the fix in the next release. If you are still seeing an issue after 1.1. comes out, we can revisit/re-open

[mitchm17]

Okay sounds good! Glad it is working with the new build!

Thank you @bryevdv !

[mitchm17]

Hey @bryevdv , there seems to be a new problem with 1.1.dev2

After I run a report summary, the data table titles seem to get hidden. Also, when you look at the Main Tab tab, after the report is run, the two buttons that were in the layout above the data table are now shrunk in width size and are now hidden for the most part.

Let me know what you think...

# ----- Imports -----
import pandas as pd
from random import randint, seed
# ----- Bokeh Imports -----
from bokeh.models         import (
        Button,CheckboxGroup,ColumnDataSource,
        Div,MultiSelect,Panel,Spacer,
        Tabs,TextInput)
from bokeh.models.widgets import (
        DataTable,NumberFormatter,TableColumn)
from bokeh.plotting       import (
        curdoc,output_file,show)
from bokeh.layouts        import (
        column,row)
# ----- Globals -----
global report_count, curr_report, name
report_count = 1
curr_report = -1
name = ''
def start_visualization():
    # ----- Main Tab Data Read-in -----
    dt_btn_width    = 200
    dt_table_width  = 450
    dt_table_height = 300
    output_file("sample.html")
    seed(123456)
    data = dict(names = [f"John Smith_{i}" for i in range(1,31)]+[''],
                ages  = [randint(18, 60) for i in range(30)]+[''],
                weight= [randint(160,260) for i in range(30)]+['']
                )
    dtypes  = {col: list(set([type(x) for x in data[col] if x != ''])) 
               for col in data}
    # ----- Main Tab Data Table -----
    source  = ColumnDataSource(data)
    columns = [TableColumn(field="names", title="Patient Name"),
               TableColumn(field="ages", title="Patient Age"),
               TableColumn(field="weight", title="Patient Weight")]
    data_table = DataTable(source=source, columns=columns, 
                           reorderable=True, scroll_to_selection=True,
                           width=dt_table_width, height=dt_table_height
                           )
    # ----- Add/Delete Data Entries -----
    add_data    = Button(label="Add data", width=dt_btn_width, 
                         button_type='primary')
    delete_data = Button(label="Delete row(s)", width=dt_btn_width, 
                         button_type='primary')
    # ----- Summary Report Tab -----
    filter_width   = 400
    summary_width  = 600
    # ----- Reset Filters widget -----
    reset_filters = Button(label="Reset all filters", width=filter_width,
                           button_type="primary")
    def reset_all_filters():
        report_name.value  = ''
        save_report.active = []
        for filt in report_filters:
            if 'slider' in filt.name:
                filt.value = (filt.start, filt.end)
            elif 'multi' in filt.name:
                filt.value = ['--']
            else:
                continue
            # end if filt.name
        # end for filt
    # end def
    reset_filters.on_click(reset_all_filters)
    # ----- Filter directions div widget -----
    report_directions = Div(text="""Select values for each column listed below.
                            For categorical columns, use Ctrl+ to select more than one
                            category. For numerical columns, use the sliders to select
                            the minimum and maximum value for the column.
                            """, width=filter_width)
    # ----- Report Title Input Widget -----
    report_name  = TextInput(title="Enter the report name:", value='')
    save_report  = CheckboxGroup(labels=["Save report"], active=[])
    col_data = data['names']
    choices  = sorted(set(col_data))
    report_filters = [MultiSelect(title="Select the name(s) in 'names' that you want to search for:", 
                      value=['--'], options = ['--']+choices, 
                      name="names_multi_filter", 
                      width=filter_width, size=6)]
    def save_report_summary(stuff_to_pickle):
        pass
    # end def
    report_button = Button(label="Run the report summary", 
                           button_type="primary", width=filter_width)
    def col_format(x, dtypes):
        if (x != 'index') and ((int in dtypes[x]) or (float in dtypes[x])):
            return NumberFormatter(format='0[.]0[00]')  
        else:
            return None
        # end if
    # end def
    def get_summary_tables(comp_data, columns, dtypes, summary_width, summary_height):
        print(f'passed in summary width: {summary_width}')
        stats        = ColumnDataSource(comp_data.describe(include='all'))
        col_title    = lambda x: f"{x} Statistics" if x != 'index' else "Statistic"
        stats_cols   = [TableColumn(field=col, title=col_title(col),
                                    formatter=col_format(col, dtypes)) for 
                        col in stats.column_names]
        stats_table  = DataTable(source=stats, columns=stats_cols, width=summary_width)
        comp_source  = ColumnDataSource(comp_data)
        comp_columns = columns
        comp_table   = DataTable(source=comp_source, columns=comp_columns,
                                 width=summary_width, height=summary_height)
        return stats_table, comp_table
    # end def
    def report_summary(event):
        saved_report = []
        comp_title   = f"'{report_name.value}' Data: "
        stats_title  = f"'{report_name.value}' Summary Statistics"
        comp_data    = pd.DataFrame(data)
        comp_data    = comp_data[comp_data != ''].dropna().reset_index(drop=True)
        data_cols    = [x for x in data][:1]
        for i, col in enumerate(data_cols):
            if "multi" in report_filters[i].name:
                print("about to do multi")
                categories       = report_filters[i].value
                saved_report.append((str(report_filters[i].name), list(categories)))
                if categories == ['--']:
                    comp_title += f'all {col}, '
                    continue
                else:
                    comp_title += f'{col} that are in {categories}'
                    comp_data = comp_data[comp_data[col].isin(categories)]
                # end if categories
            else:
                saved_report.append((str(report_filters[i].name), tuple(report_filters[i].value),
                                     float(report_filters[i].start), float(report_filters[i].end)))
                comp_data[col] = comp_data[col].astype(float)
                minVal, maxVal = report_filters[i].value
                comp_title     += f'{col} between {minVal} and {maxVal}'
                # Check if the min and max vals are equal to the min and max of range slider,
                # if they are then there is no point in doing anything more for this filter
                if minVal == report_filters[i].start and maxVal == report_filters[i].end:
                    comp_title += ', ' if i != len(data)-1 else ''
                    continue
                # end if minVal, maxVal are at max range
                comp_data = comp_data[comp_data[col].between(minVal, maxVal)]
            # end if "multi"
            if i != len(data)-1:
                comp_title += ', '
            # end if
        # end for i, col
        if save_report.active:
            save_report_summary(saved_report)
        # end if
        stats_title             = Div(text=f"""<b>{stats_title}</b>""", 
                                      width=summary_width)
        comp_title              = Div(text=f"""<b>{comp_title}</b>""", 
                                      width=summary_width)
        stats_table, comp_table = get_summary_tables(comp_data, columns, dtypes, 
                                                        summary_width, 300)
        report_output           = row(Spacer(width=50), 
                                      column(stats_title, stats_table, 
                                             comp_title, comp_table))
        layout.tabs[0].child.children[1] = report_output
    # end def
    report_button.on_click(report_summary)
    # ----- Layout Setup -----
    button_row    = row(add_data, Spacer(width=50), delete_data)
    mainLayout    = column(button_row, data_table)
    report_input  = column(reset_filters, report_directions, 
                           row(report_name, save_report), 
                           *report_filters, report_button)
    report_output = Spacer(width=summary_width)
    report_layout = row(report_input, report_output)
    tab1          = Panel(child=report_layout, title="Run a Report")
    tab2          = Panel(child=mainLayout, title="Main Tab")
    layout        = Tabs(tabs=[tab1, tab2])
    show(layout)
    curdoc().add_root(layout)
# end def start_visualization
start_visualization()

Here's what it looks like when the buttons shrink and the data table covers them up:

[bryevdv]

@mitchm17 I think those are #8614 which is known and will be resolved before 1.1 is released

[mitchm17]

@bryevdv cool. Is there a way I can revert back to 1.0.4 in the meantime? Does 1.1 have a release date yet?

[bryevdv]

Is there a way I can revert back to 1.0.4 in the meantime?

conda install bokeh=1.0.4 should work

Does 1.1 have a release date yet?

hopefully in the next two weeks

[mitchm17]

@bryevdv great, thanks. I'll keep a look out!