Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple output_file in a single process can fail when models are shared #9767

Open
jaseuts opened this issue Mar 10, 2020 · 19 comments
Open
Milestone

Comments

@jaseuts
Copy link

jaseuts commented Mar 10, 2020

I have 2 anaconda environments in Windows 7:

  • env1 has jupyter lab 0.32 and bokeh 0.12.16,
  • env2 has jupyter lab 2.0.1 and bokeh 2.0.0.

In a jupyter notebook, I have 2 cells of code:

Cell 1

source1 = ColumnDataSource(fertility_df[fertility_df['Continent']=='ASI'])
p1 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p1.circle('fertility', 'female literacy', source=source1)

source2 = ColumnDataSource(fertility_df[fertility_df['Continent']=='EUR'])
p2 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p2.circle('fertility', 'female literacy', source=source2)

source3 = ColumnDataSource(fertility_df[fertility_df['Continent']=='AF'])
p3 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p3.circle('fertility', 'female literacy', source=source3)

source4 = ColumnDataSource(fertility_df[fertility_df['Continent']=='LAT'])
p4 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p4.circle('fertility', 'female literacy', source=source4)

row1 = [p1, p2]
row2 = [p3, p4]
layout = gridplot([row1, row2])
output_file('bokeh.html')
show(layout)

Cell 2

tab1 = Panel(child=p1, title='Latin America')
tab2 = Panel(child=p2, title='Africa')
tab3 = Panel(child=p3, title='Asia')
tab4 = Panel(child=p4, title='Europe')

layout = Tabs(tabs=[tab1, tab2, tab3, tab4])
output_file('bokeh.html')
show(layout)

In the env1, I could run cell 1, cell 2, and cell 1 again without any problem.

However, in env2, after executing cell 1, then trying to run cell 2, I got an error :

RuntimeError: Models must be owned by only a single document, WheelZoomTool(id='1130', ...) is already in a doc

My questions are:

  • Is it a bug in bokeh 2.0.0?
  • What does it mean by Models in the error message?
@jaseuts jaseuts added the TRIAGE label Mar 10, 2020
@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

@jaseuts "Model" means any Bokeh object with properties that gets synchronized between Python and JavaScript, e.g. the WheelZoomTool mentioned in the error message (but also: plots, ranges, glyphs, data sources, layouts...)

I can't reproduce this on my own with a similar set of commands in a notebook I already have, so we will absolutely need a complete notebook we can inspect and run in order to investigate.

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

Noting: if there is an issue here, I would not be surprised if it is specific to Tabs/Panel

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

Hi @bryevdv, thanks for your explanation of models. Could you please elaborate what you meant? Why did it work in older version of bokeh (0.12.6) but got error in current version 2.0.0? How can I send you my notebook for inspection?

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

@jaseuts I don't know, that's why I am asking you to provide a complete notebook, so that we can reproduce what you are seeing, and investigate.

As an aside, can I ask why you are showing the same plot multiple times in different cells? I would regard that as unusual usage and it's possible things only worked before "by accident".

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

@bryevdv, a legitimate question: that notebook is to follow a tutorial of Data Visualisation With Bokeh. Each cell to show how to use a certain bokeh's features. For example, cell 3 to use gridplot, cell 4 to use Tabs. As I don't know how to upload the workbook, here's its entire contents
Cell 1:

import pandas as pd
import numpy as np
from bokeh.plotting import figure
from bokeh.io import output_file, show, curdoc
from bokeh.plotting import ColumnDataSource
from bokeh.models import HoverTool, Slider, CategoricalColorMapper, Select, CheckboxGroup, RadioGroup, Toggle
from bokeh.layouts import row, column, gridplot, widgetbox
from bokeh.models.widgets import Panel, Tabs, Button

Cell 2:

fertility_df = pd.read_csv('Data/female_literacy_fertility.csv', thousands=',')
fertility = fertility_df['fertility'].tolist()
female_literacy = fertility_df['female literacy'].tolist()
fertility_africa = fertility_df.loc[fertility_df['Continent']=='AF']['fertility'].tolist()
female_literacy_africa = fertility_df.loc[fertility_df['Continent']=='AF']['female literacy'].tolist()
fertility_latinamerica = fertility_df.loc[fertility_df['Continent'].isin( ['LAT'])]['fertility'].tolist()
female_literacy_latinamerica = fertility_df.loc[fertility_df['Continent'].isin(['LAT'])]['female literacy'].tolist()

latin_america = ColumnDataSource(fertility_df[fertility_df['Continent']=='LAT'])
africa = ColumnDataSource(fertility_df[fertility_df['Continent']=='AF'])

Cell 3:

source1 = ColumnDataSource(fertility_df[fertility_df['Continent']=='ASI'])
p1 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p1.circle('fertility', 'female literacy', source=source1)

source2 = ColumnDataSource(fertility_df[fertility_df['Continent']=='EUR'])
p2 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p2.circle('fertility', 'female literacy', source=source2)

source3 = ColumnDataSource(fertility_df[fertility_df['Continent']=='AF'])
p3 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p3.circle('fertility', 'female literacy', source=source3)

source4 = ColumnDataSource(fertility_df[fertility_df['Continent']=='LAT'])
p4 = figure(x_axis_label='fertility (children per woman)', y_axis_label='female literacy (% population)')
p4.circle('fertility', 'female literacy', source=source4)

row1 = [p1, p2]
row2 = [p3, p4]
layout = gridplot([row1, row2])
output_file('bokeh.html')
show(layout)

Cell 4:

tab1 = Panel(child=p1, title='Latin America')
tab2 = Panel(child=p2, title='Africa')
tab3 = Panel(child=p3, title='Asia')
tab4 = Panel(child=p4, title='Europe')

layout = Tabs(tabs=[tab1, tab2, tab3, tab4])
output_file('bokeh.html')
show(layout)

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

@jaseuts unfortunately that code requires an additional data file, so it cannot be run as-is. We need you to either:

  • re-work the example code to be complete and self-contained, or
  • provide the necessary data file

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

As an aside, we have our own live tutorial notebooks that can be used instead:

https://mybinder.org/v2/gh/bokeh/bokeh-notebooks/master?filepath=tutorial%2F00%20-%20Introduction%20and%20Setup.ipynb

I think the actual fix here is to try and get whoever made this other tutorial to update it to be better and more idiomatic. Can you specify where this tutorial is, or who made it?

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

@bryevdv, please find attached
image

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

What is that? Is that datacamp? Nothing on the image states who made that. @jaseuts please be more forthcoming with information. I genuinely want to help but you are making this difficult. Always err on the side of giving more information when asking for help from OSS maintainers.

If that is DataCamp then that is a very old tutorial, I have no idea how or if it is upgraded in the many years since I helped create it, and I would suggest just avoiding it altogether and using the project tutorial notebooks (linked above) instead. They are much more up to date, more comprehensive, and we make sure they demonstrate the kind of usage we intend.

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

@bryevdv , sorry I didn't know those information was relevant, and I didn't mean to make it difficult. Yes, the course is in DataCamp, created by Team Anaconda. Thanks for the link to bokeh live tutorial notebook, I'll definitely have a look. Still, I think there's a bug with version 2.0.0 because version 0,12.6 works without error when I used the same model in 2 different document (running cell 3, cell 4, and cell 3 again)

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

@jaseuts There have been two major-number bumps since 0.12.6, each of which is a valid place for breaking changes to be introduced. Even if this construction did work in 0.12.6 it's not any guaranteed reason it must for for 2.0 I personally think this usage is atypical and unusual and not necessarily something we need or intend to support. There is nothing like this in any of the official documentation.

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

Thanks, @bryevdv. I can understand what you are coming from. Appreciated your help. Cheers.

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

All that said, if you can provide a complete, minimal, self-contained example I am still happy to take a look.

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

Thanks for your offer, @bryevdv , but I'm not sure what you meant by self-contained example. This is all I have (saved as .txt instead of .ipynb for uploading)
bokeh problem.txt

@bryevdv
Copy link
Member

bryevdv commented Mar 10, 2020

I mean something I can personally run, because it has everything necessary. As I mentioned before, I can't runtime notebook code above because it uses a data file file that I don't have. You could rework the code to use synthetic or fake data so that other people can run it.

@jaseuts
Copy link
Author

jaseuts commented Mar 10, 2020

Oh, I see. I downloaded the data from here https://assets.datacamp.com/production/repositories/401/datasets/5aae6591ddd4819dec17e562f206b7840a272151/literacy_birth_rate.csv.
Alternatively I copy the whole thing from that website as below:

Country ,Continent,female literacy,fertility,population
Chine,ASI,90.5,1.769,1324655000.0
Inde,ASI,50.8,2.682,1139964931.64188
USA,NAM,99,2.077,304060000.0
Indonésie,ASI,88.8,2.132,227345082.0
Brésil,LAT,90.2,1.827,191971506.0
Pakistan,ASI,40,3.872,166111487.118278
Bangladesh,ASI,49.8,2.288,160000128.0
Nigéria,AF,48.8,5.173,151212254.0
Fédération de Russie,EUR,99.4,1.393,141950000.0
Japan,ASI,99,1.262,127704000.0
Mexique,LAT,91.5,2.156,106350433.683525
Philippines,ASI,93.9,3.026,90348437.0
Viet Nam,ASI,90.2,2.033,86210781.0
Germany,EUR,99,1.324,82110097.0
Egypte,AF,57.8,2.816,81527172.0
Ethiopie,AF,22.8,5.211,80713434.0
Turquie,ASI,81.3,2.1,73914260.0
"Iran, République islamique d'",ASI,77.2,1.781,71956321.6875886
Thaïlande,ASI,91.5,1.822,67386383.0
Rép. Démocratique du Congo,AF,56.1,5.908,64256635.0
France,EUR,99,1.881,62277432.0
UK,EUR,99,1.852,61414062.0
Italie,EUR,98.5,1.39,59832179.0
Myanmar,ASI,89.2,2.281,49563019.0
Afrique du Sud,AF,88.1,2.505,48687000.0
South Korea,ASI,96.6,1.224,48607000.0
Ukraine,EUR,99.6,1.361,46258200.0
Espagne,EUR,96.9,1.468,45555716.0
Colombie,LAT,93.4,2.404,45012096.0
République-Unie de Tanzanie,AF,66.3,5.52,42483923.0
Soudan,AF,59.6,4.058,41347723.0
Argentine,LAT,97.7,2.223,39882980.0
Kenya,AF,82.8,4.859,38765312.0
Pologne,EUR,99.3,1.267,38125759.0
Algérie,AF,63.9,2.342,34373426.0
Canada,NAM,99,1.579,33311400.0
Ouganda,AF,66.8,6.254,31656865.0
Maroc,AF,44.1,2.334,31605616.0
Iraq,ASI,69.2,3.961,30711152.2927428
Afghanistan,ASI,12.6,6.505,29021098.9974162
Pérou,LAT,84.6,2.53,28836700.0
Népal,ASI,45.4,2.823,28809526.0
Venezuela,LAT,94.9,2.498,27935000.0
Ouzbékistan,ASI,98.9,2.248,27313700.0
Malaisie,ASI,89.8,2.508,27014337.0
Arabie saoudite,ASI,80.2,3.04,24645685.9216375
Rép. populaire démocratique de Corée,ASI,100,1.854,23818753.0
Ghana,AF,59.3,4.22,23350927.0
Yémen,ASI,42.8,5.1,22917485.0
Mozambique,AF,40.1,4.967,22382533.0
Roumanie,EUR,96.9,1.325,21513622.0
Côte d'Ivoire,AF,44.3,4.514,20591302.0
République arabe syrienne,ASI,77.2,3.173,20581289.8697702
Sri Lanka,ASI,89.1,2.308,20156204.3380748
Madagascar,AF,65.3,4.62,19110941.0
Cameroun,AF,67.8,4.541,19088385.0
Angola,AF,57,5.637,18020668.0
Chili,LAT,98.7,1.926,16803952.0
Netherlands,EUR,99,1.747,16445593.0
Kazakhstan,ASI,99.5,2.294,15674833.0
Burkina Faso,AF,21.6,5.841,15233884.0
Malawi,AF,65.8,5.455,14846182.0
Niger,AF,15.1,7.069,14704318.0
Cambodge,ASI,70.9,2.859,14562008.0
Guatemala,LAT,68.7,4.018,13686128.0
Equateur,LAT,81.7,2.513,13481424.0
Mali,AF,18.2,5.405,12705736.0
Zambie,AF,61,5.737,12620219.0
Zimbabwe,AF,88.8,3.363,12462879.0
Sénégal,AF,33,4.89,12211181.0
Grèce,EUR,95.9,1.385,11237094.0
Cuba,LAT,99.8,1.505,11204735.0
Tchad,AF,21.9,6.081,10913667.0
Belgium,EUR,99,1.784,10708433.0
Portugal,EUR,92.9,1.378,10622413.0
Czech rep,EUR,99,1.45,10424336.0
Tunisie,AF,71,1.841,10327800.0
Hongrie,EUR,98.9,1.37,10038188.0
République dominicaine,LAT,88.3,2.612,9952711.0
Guinée,AF,26.4,5.329,9833055.0
Rwanda,AF,66.1,5.33,9720694.0
Bolivie,LAT,86,3.371,9694113.0
Bélarus,EUR,99.7,1.281,9680850.0
Sweden,EUR,99,1.871,9219637.0
Azerbaïdjan,ASI,99.2,2.153,8680100.0
Bénin,AF,28.1,5.378,8662086.0
Burundi,AF,59.9,4.45,8074254.0
Switzerland,EUR,99,1.46,7647675.0
Bulgarie,EUR,97.9,1.436,7623395.0
Serbie,EUR,96.2,1.612,7350221.0
Honduras,LAT,83.5,3.19,7318789.0
Israel,ASI,95.9,2.752,7308800.0
Tadjikistan,ASI,99.5,3.35,6836083.0
Papouasie-Nouvelle-Guinée,OCE,55.6,4.01,6576822.0
Togo,AF,53.7,4.166,6458605.0
Jamahiriya arabe libyenne,AF,81.3,2.642,6294181.0
Paraguay,LAT,93.5,2.977,6237855.0
Rép. démocratique populaire lao,ASI,63.2,3.415,6205341.0
El Salvador,LAT,81.4,2.295,6133910.0
Jordanie,ASI,88.9,3.019,5906042.87150034
Nicaragua,LAT,77.9,2.683,5667325.0
Sierra Leone,AF,28.9,5.165,5559853.0
Denmark,EUR,99,1.849,5493621.0
Finland,EUR,100,1.836,5313399.0
Kirghizistan,ASI,99.1,2.518,5277900.0
Turkménistan,ASI,99.3,2.43,5043618.0
Erythrée,AF,54.5,4.528,4926877.0
Singapour,ASI,91.6,1.263,4839400.0
Norway,EUR,100,1.885,4768212.0
Costa Rica,LAT,96.2,1.943,4519126.0
Emirats arabes unis,ASI,91.5,1.899,4484935.0
Croatie,EUR,98,1.442,4434000.0
Ireland,EUR,99,1.953,4425675.0
République centrafricaine,AF,41.1,4.697,4339263.0
Géorgie,ASI,99.7,1.582,4307011.0
New Zealand,OCE,99,2.025,4268900.0
Liban,ASI,86,1.841,4193758.0
Libéria,AF,53,5.011,3793400.0
Bosnie-Herzégovine,EUR,95.9,1.212,3773100.0
République de Moldova,EUR,97.8,1.502,3633369.0
Panama,LAT,92.8,2.516,3398823.0
Lituanie,EUR,99.7,1.367,3358115.0
Uruguay,LAT,98.5,2.089,3334052.0
Mauritanie,AF,49.5,4.388,3215043.0
Albanie,EUR,98.7,1.854,3143291.0
Arménie,ASI,99.4,1.748,3077087.0
Oman,ASI,80.9,2.978,2785361.0
Koweït,ASI,93.1,2.152,2728040.83948046
Jamaïque,LAT,90.8,2.362,2687200.0
Mongolie,ASI,97.8,1.988,2641216.0
Lettonie,EUR,99.8,1.426,2266094.0
Namibie,AF,87.7,3.29,2129854.0
Lesotho,AF,95.1,3.264,2049429.0
L'ex-Rép. yougoslave de Macédoine,EUR,95.4,1.436,2041342.0
Slovénie,EUR,99.7,1.393,2021316.0
Botswana,AF,83.5,2.822,1921122.0
Gambie,AF,34.3,4.969,1660200.0
Guinée-Bissau,AF,36.5,5.659,1575446.0
Gabon,AF,83.2,3.24,1448159.0
Estonie,EUR,99.8,1.693,1340675.0
Trinité-et-Tobago,LAT,98.2,1.647,1333388.0
Qatar,ASI,90.4,2.36,1280862.0
Maurice,AF,84.8,1.792,1268854.0
Swaziland,AF,85.6,3.45,1167834.0
Chypre,EUR,96.7,1.516,862434.0
Bahreïn,ASI,89.4,2.233,775585.0
Bhoutan,ASI,38.7,2.563,686789.0
Guinée équatoriale,AF,89.1,5.283,659197.0
Comores,AF,67.8,3.885,643571.312838574
"Macao, Chine",ASI,90.7,0.966,526178.0
Suriname,LAT,88.4,2.373,515124.0
Cap-Vert,AF,79.3,2.663,498672.0
Malte,EUR,93.5,1.251,411950.0
Brunéi Darussalam,ASI,93.3,2.052,392280.0
Bahamas,LAT,96.5,3.371,337668.0
Iceland,EUR,99,2.093,317414.0
Maldives,ASI,98.4,2,305027.0
Vanuatu,OCE,79.5,3.883,233866.0
Samoa,OCE,98.5,3.852,178869.0
Sao Tomé-et-Principe,AF,83.3,3.718,160174.0
Aruba,LAT,98,1.732,105455.0
Tonga,ASI,99.1,3.928,103566.0
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
,,,,
Country ,Continent,female literacy,fertility,
Antigua-et-Barbuda,,99.4,,
Antilles néerlandaises,,96.3,,
Iles Caïmanes,,99,,
Seychelles,,92.3,,
Territoires autonomes palestiniens,,90.9,,
WORLD,WORLD,77,,

@bryevdv
Copy link
Member

bryevdv commented Mar 22, 2020

Here is a drastically more minimal example:

Screen Shot 2020-03-22 at 1 16 57 PM

@jaseuts just for future FYI: this is the kind of thing that maintainers really need when they ask for a minimal, reproducing, example. OSS is a collaboration, and users need to put in some effort to whittle down their problems to things that can be digested as quickly and easily as possible.

@bryevdv bryevdv changed the title RuntimeError: Models must be owned by only a single document, WheelZoomTool(id='1130', ...) is already in a doc Multiple output_file in a single process can fail when models are shared Mar 22, 2020
@bryevdv bryevdv added this to the short-term milestone Mar 22, 2020
@bryevdv
Copy link
Member

bryevdv commented Mar 22, 2020

FWIW I don't think this is common in real-world usage:

  • output_file in notebooks is not common
  • re-using the same plot is not common

Accordingly I would not say this is a very high priority to address.

Offhand, there is probably an issue with OutputDocumentFor. It is supposed to "re-home" any models temporarily in a new document so that exactly this error does not occur. It's supposed to recursively traverse the object graph to do this, but I'm guess something about the tabs/panel structure is just overlooked in this process.

@felixbrinquis
Copy link

Good evenng @bryevdv !

As I can see other users had the same problem as me dealing with tabs and panels.

In my case I developed a whole personal project and in the last step, when putting all together in the dashboard this error arose.
I've tried to solve it many times with no successful solution.

I can provide the full project. It is located here:
https://github.com/felixbrinquis/PythonGPXAnalyzer
Some output files can also be found here: https://drive.google.com/drive/folders/1_oNaZILjx0tPaTvUP9jn5SXnl7Hu77cH?usp=sharing

The error is generated when running a second input file without restart the Spyder core.

The main idea was to generate several output files in a bucle, which can be performed by modifying the option LecturaIndividual = 'N' in LecturaDirectorio.py or just changingn the input file name and rerun.

If someone comes up with any idea to avoid this error I would be very thanked, maybe changing the way to create the output, a different arrangement… whatever.

Thank You beforehand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants