Problem while loading notebooks

[dani-doni]

Hi! I'm pretty new to programming. I'm using Jupiter notebook in local to do some works on data and Mercury is amazing to see the results (thank you very much for what you're doing)! but this morning I had this problem:

Yesterday everythings works well. This morning I cancelled some old notebook and run Mercury, but everythings stop working. I had this message: "Problem while loading notebooks. Please try again later or contact Mercury administrator." and in the consolle panel: "Problem during loading recent notebooks. SyntaxError: Unexpected token 'N', ..."nantola", NaN, "Noni"... is not valid JSON
(anonymous) @ notebooksSlice.ts:466"

Can you please help me?

Thanks
Daniele

[pplonski]

Hi @dani-doni,

Thanks for reporting the issue. Maybe try to clean Mercury server with command:

mercury run clear

It should clean local database and just start Mercury again:

mercury run

[dani-doni]

I've tryed and re-run mercury

but I have still the message: "Problem while loading notebooks. Please try again later or contact Mercury administrator."

[pplonski]

Please try to run Mercury in verbose mode, maybe we will some some error msgs:

mercury run --verbose

Apologize for problems.

[dani-doni]

I have this:

Version: 2.3.6
The notebook Dashboard_comuni_immobiliare.ipynb will be updated
Successfully updated a notebook (id:1)
The notebook Folium_map_with_layer.ipynb will be updated
Successfully updated a notebook (id:2)
The notebook Mercury_prova.ipynb will be updated
Successfully updated a notebook (id:3)
Performing system checks...

System check identified no issues (0 silenced).
October 24, 2023 - 12:48:43
Django version 4.2.2, using settings 'server.settings'
Starting ASGI/Daphne version 4.0.0 development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
DJ INFO 2023-10-24 12:48:43,441 runserver HTTP GET / 200 [0.02, 127.0.0.1:55489]
DJ INFO 2023-10-24 12:48:43,750 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.02, 127.0.0.1:55489]
DJ INFO 2023-10-24 12:48:43,805 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.05, 127.0.0.1:55489]
DJ INFO 2023-10-24 12:48:43,823 runserver HTTP GET /api/v1/1/welcome/ 200 [0.05, 127.0.0.1:55504]
DJ INFO 2023-10-24 12:48:43,830 runserver HTTP GET /api/v1/1/notebooks/ 200 [0.06, 127.0.0.1:55494]
[2023-10-24 12:48:46,175: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2023-10-24 12:48:46,179: DEBUG/MainProcess] | Worker: Building graph...
[2023-10-24 12:48:46,180: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Pool, Autoscaler, StateDB, Consumer}
[2023-10-24 12:48:46,353: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2023-10-24 12:48:46,354: DEBUG/MainProcess] | Consumer: Building graph...
[2023-10-24 12:48:46,385: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Heart, Mingle, Tasks, Control, Agent, Gossip, event loop}

 -------------- celery@LAPTOP-UJP242IH v5.3.4 (emerald-rush)
--- ***** -----
-- ******* ---- Windows-10-10.0.19045-SP0 2023-10-24 12:48:46
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         server:0x1ce43fe2ed0
- ** ---------- .> transport:   sqla+sqlite:///celery.sqlite
- ** ---------- .> results:     sqlite:///celery.sqlite
- *** --- * --- .> concurrency: 1 (gevent)
-- ******* ---- .> task events: ON
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
                .> ws               exchange=ws(direct) key=ws

[tasks]
  . apps.accounts.tasks.task_init_site
  . apps.accounts.tasks.task_send_invitation
  . apps.accounts.tasks.task_send_new_member
  . apps.notebooks.tasks.task_watch
  . apps.tasks.tasks.task_execute
  . apps.tasks.tasks_export.export_to_pdf
  . apps.ws.tasks.task_start_websocket_worker
  . celery.accumulate
  . celery.backend_cleanup
  . celery.chain
  . celery.chord
  . celery.chord_unlock
  . celery.chunks
  . celery.group
  . celery.map
  . celery.starmap
  . mercury.server.celery.execute_notebook
  . mercury.server.celery.scale_down_task
  . server.celery.execute_notebook
  . server.celery.scale_down_task

[2023-10-24 12:48:46,459: DEBUG/MainProcess] | Worker: Starting Pool
[2023-10-24 12:48:46,460: DEBUG/MainProcess] ^-- substep ok
[2023-10-24 12:48:46,462: DEBUG/MainProcess] | Worker: Starting Consumer
[2023-10-24 12:48:46,462: DEBUG/MainProcess] | Consumer: Starting Connection
[2023-10-24 12:48:46,465: WARNING/MainProcess] C:\Users\Daniele\AppData\Local\Programs\Python\Python311\Lib\site-packages\celery\worker\consumer\consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

[2023-10-24 12:48:46,485: INFO/MainProcess] Connected to sqla+sqlite:///celery.sqlite
[2023-10-24 12:48:46,485: DEBUG/MainProcess] ^-- substep ok
[2023-10-24 12:48:46,486: DEBUG/MainProcess] | Consumer: Starting Events
[2023-10-24 12:48:46,488: WARNING/MainProcess] C:\Users\Daniele\AppData\Local\Programs\Python\Python311\Lib\site-packages\celery\worker\consumer\consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
  warnings.warn(

[2023-10-24 12:48:46,489: DEBUG/MainProcess] ^-- substep ok
[2023-10-24 12:48:46,489: DEBUG/MainProcess] | Consumer: Starting Heart
[2023-10-24 12:48:46,489: DEBUG/MainProcess] ^-- substep ok
[2023-10-24 12:48:46,490: DEBUG/MainProcess] | Consumer: Starting Tasks
[2023-10-24 12:48:46,511: DEBUG/MainProcess] ^-- substep ok
[2023-10-24 12:48:46,512: DEBUG/MainProcess] | Consumer: Starting event loop
[2023-10-24 12:48:46,514: INFO/MainProcess] celery@LAPTOP-UJP242IH ready.
[2023-10-24 12:48:46,514: DEBUG/MainProcess] basic.qos: prefetch_count->4
DJ INFO 2023-10-24 12:49:13,467 runserver HTTP GET / 200 [0.01, 127.0.0.1:55550]
DJ INFO 2023-10-24 12:49:13,663 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.01, 127.0.0.1:55550]
DJ INFO 2023-10-24 12:49:13,682 runserver HTTP GET /api/v1/get-site/single-site/ 200 [0.01, 127.0.0.1:55555]
DJ INFO 2023-10-24 12:49:13,694 runserver HTTP GET /api/v1/1/welcome/ 200 [0.01, 127.0.0.1:55558]
DJ INFO 2023-10-24 12:49:13,700 runserver HTTP GET /api/v1/1/notebooks/ 200 [0.02, 127.0.0.1:55550]

[pplonski]

Logs look good. I have no idea what might be the reason.

I would start with fresh virtual environment and copy notebooks there.

[dani-doni]

I started with a fresh virtual env but I had the same issue. So, I started from zero with a super easy code in the notebooks.

I found out that everythings works properly when the code is easy:

import mercury as mr 
import pandas as pd

#Create the mercury app
app = mr.App(title="Dashboard Comuni Italiani", description="bla bla")

d = {'nome_comune': ["Abano Terme", "Gottolengo"], 'col2': [3, 4]}
df = pd.DataFrame(data=d)

#Create the select widget 
selected = mr.Select (label="Scegli il comune", 
                      value="Abano Terme",
                      choices=df.nome_comune.unique())

widget = mr.NumberBox(data=selected.value, title="Andamento popolazione")
widget

When I substitute the dataframe with a bigger one, problems start to come. Here is the new df (csv file of 3.5Mb):

db = r"C:\Users\Daniele\Desktop\Immobiliare\04_dati_e_business case\PHYTON_IMMOBILIARE\6_progetto_Mercury\Dati_popolazione.csv"
df = pd.read_csv(db, encoding='latin-1')

I had these:

and then If I run "mercury run clean" and "mercury run" I come back to first screen:

And if I change the big df with the small one and "mercury run clean" and "mercury run", everythings come back to work properly.

Is the file to big?

[pplonski]

Hi @dani-doni,

Thank you for description. I've tried to reproduce your issue on fresh install but I couldn't. Could you please send me your data file? Maybe I could reproduce the issue with your file. The file size 3.5 MB shouldn't be a problem.

[dani-doni]

Dati_popolazione.csv

Here is the file. Thank you very much!

[pplonski]

Thank you @dani-doni. I'm able to reproduce the issue locally with your dataset. Looks like some nasty bug. Searching for the reason ...

[pplonski]

OK, I found the issue. There is problem with coding values in widgets with non ASCII characters, like à. I would like to fix it today. I will let you know when ready. Grazie per l'aiuto!

[pplonski]

@dani-doni it wasn't issue with encoding as I thought. The issue was with NaN values in data. Anyway, it is fixed now. Please update Mercury with the following steps:

# clear previous instance
mercury run clear

# install new version (2.3.7)
pip install -U mercury

# start mercury
mercury run

Thank you for your help in finding this bug.

[dani-doni]

It's working!! Grazie mille del fantastico lavoro!

[dani-doni]

Just last rapid question: whit everything resolved I'm trying to deploy my jupiter on Mercury cloud. How do I have to change my file reference in the code?

and I have this at the end:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Daniele\\Desktop\\Immobiliare\\04_dati_e_business case\\PHYTON_IMMOBILIARE\\6_progetto_Mercury\\Dati_popolazione.csv'

in my code I have still the file reference in local:

db = r"C:\Users\Daniele\Desktop\Immobiliare\04_dati_e_business case\PHYTON_IMMOBILIARE\6_progetto_Mercury\Dati_popolazione.csv"
df = pd.read_csv(db, encoding='latin-1')

but of course it doesn't work when deployed (I uploaded the file named "Dati_popolazione" in mercury cloud and I don't know how to do refer to it) Thanks

[pplonski]

Please change the path to file to the current directory:

db = "Dati_popolazione.csv"
df = pd.read_csv(db, encoding="latin-1")

[dani-doni]

I'm sorry to bother you but I still have a problem with deploy.
If I deploy it's work but I still have the message " Problem while loading notebooks. Please try again later or contact Mercury administrator"

If I share it with another email it's not working:

And after few times also for my email it doesn't work anymore.

I tryed to cut my code and only use mr.numberbox but the problem persists:

import mercury as mr 
import pandas as pd

#Open and read the file with data + create a dataframe with data "df"
db = "Dati_popolazione.csv"
df = pd.read_csv(db, encoding='latin-1')

#Create the mercury app
app = mr.App(title="Dashboard Comuni Italiani", description="Dashboard di analisi dei comuni italiani come supporto all'analisi di investimenti immobiliari")

#Create the select widget (only one city)
selected = mr.Select (label="Scegli il comune", 
                      value="Abano Terme",
                      choices=df.nome_comune.unique())

#filter the original df with the selected city: "new_df"
new_df = df[df.nome_comune==selected.value]

#create widget with the main information of city
mr.NumberBox([
    mr.NumberBox(data=new_df.nome_comune[new_df.index[0]], title="Nome Comune"),
    mr.NumberBox(data=new_df.reg[new_df.index[0]], title="Regione"),
    mr.NumberBox(data=new_df.prov[new_df.index[0]], title="Provincia"),
])

[pplonski]

Sorry @dani-doni for problems, I didn't update code in Mercury Cloud servers ... Sorry, I will let you know when updated. On your side, you will need to remove notebooks and upload them again, no code changes needed.

[dani-doni]

Okey thanks a lot! let me know when all it's done, have a great day!