Question about nuboard metrics in nuplan_framework.ipynb

[IvanChen777]

Hi, I have a problem about the metrics shown in the nuboard, after I finish the training and simulation, I get all the metrics parquet in the metrics folder, however, when I want to load the nuboard, it seems that there is nothing there, does anyone know how to solve this issue? If so, please let me know. Thank you in advance:)

[michael-motional]

Hey @IvanChen777, this is most likely an issue with the metrics aggregation not running, can you try the suggestions in this comment and let me know if they solve the problem?

[IvanChen777]

Sure, @michael-motional, I will tell you the result if it works.

[dmachapu]

Hi, I have a problem about the metrics shown in the nuboard, after I finish the training and simulation, I get all the metrics parquet in the metrics folder, however, when I want to load the nuboard, it seems that there is nothing there, does anyone know how to solve this issue? If so, please let me know. Thank you in advance:)

Hi @IvanChen777 , I guess you are atleast multiple steps ahead of me, I am having trouble in launching the nuboard itself.

Bokeh is giving me below error, any suggestions would greatly help me. Thanks.

ERROR:tornado.application:Uncaught exception GET /autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none (::1)
HTTPServerRequest(protocol='http', host='localhost:34763', method='GET', uri='/autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none', version='HTTP/1.1', remote_ip='::1')
Traceback (most recent call last):
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/tornado/web.py", line 1713, in _execute
result = await result
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/views/autoload_js_handler.py", line 62, in get
session = await self.get_session()
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/views/session_handler.py", line 145, in get_session
session = await self.application_context.create_session_if_needed(session_id, self.request, token)
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/contexts.py", line 242, in create_session_if_needed
self._application.initialize_document(doc)
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/application/application.py", line 192, in initialize_document
h.modify_document(doc)
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/application/handlers/function.py", line 143, in modify_document
self._func(doc)
File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/nuboard.py", line 113, in main_page
overview_tab = OverviewTab(doc=self._doc, experiment_file_data=experiment_file_data)
File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/tabs/overview_tab.py", line 33, in init
super().init(doc=doc, experiment_file_data=experiment_file_data)
File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/base/base_tab.py", line 55, in init
self.planner_checkbox_group.on_click(self._click_planner_checkbox_group)
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/core/has_props.py", line 360, in getattr
self._raise_attribute_error_with_matches(name, properties)
File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/core/has_props.py", line 368, in _raise_attribute_error_with_matches
raise AttributeError(f"unexpected attribute {name!r} to {self.class.name}, {text} attributes are {nice_join(matches)}")
AttributeError: unexpected attribute 'on_click' to CheckboxGroup, possible attributes are active, align, aspect_ratio, classes, context_menu, css_classes, disabled, flow_mode, height, height_policy, inline, js_event_callbacks, js_property_callbacks, labels, margin, max_height, max_width, min_height, min_width, name, resizable, sizing_mode, styles, stylesheets, subscribed_events, syncable, tags, visible, width or width_policy
ERROR:tornado.access:500 GET /autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none (::1) 55.76ms

[IvanChen777]

Sure, @michael-motional, I will tell you the result if it works.
Hi, thank you for your information. Until now, the nuboard could display nothing for the metrics, but I find out that in the scenario part, there are many useful information, like simulating the scenario. And since the nuplan v1.1 could directly show the metrics in the simulation part. I think this problem is not so urgent.

[IvanChen777]

Hi, I have a problem about the metrics shown in the nuboard, after I finish the training and simulation, I get all the metrics parquet in the metrics folder, however, when I want to load the nuboard, it seems that there is nothing there, does anyone know how to solve this issue? If so, please let me know. Thank you in advance:)

Hi @IvanChen777 , I guess you are atleast multiple steps ahead of me, I am having trouble in launching the nuboard itself.

Bokeh is giving me below error, any suggestions would greatly help me. Thanks.

ERROR:tornado.application:Uncaught exception GET /autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none (::1) HTTPServerRequest(protocol='http', host='localhost:34763', method='GET', uri='/autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none', version='HTTP/1.1', remote_ip='::1') Traceback (most recent call last): File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/tornado/web.py", line 1713, in _execute result = await result File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/views/autoload_js_handler.py", line 62, in get session = await self.get_session() File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/views/session_handler.py", line 145, in get_session session = await self.application_context.create_session_if_needed(session_id, self.request, token) File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/server/contexts.py", line 242, in create_session_if_needed self._application.initialize_document(doc) File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/application/application.py", line 192, in initialize_document h.modify_document(doc) File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/application/handlers/function.py", line 143, in modify_document self._func(doc) File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/nuboard.py", line 113, in main_page overview_tab = OverviewTab(doc=self._doc, experiment_file_data=experiment_file_data) File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/tabs/overview_tab.py", line 33, in init super().init(doc=doc, experiment_file_data=experiment_file_data) File "/home/divya/nuplan-devkit/nuplan/planning/nuboard/base/base_tab.py", line 55, in init self.planner_checkbox_group.on_click(self._click_planner_checkbox_group) File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/core/has_props.py", line 360, in getattr self._raise_attribute_error_with_matches(name, properties) File "/home/divya/miniconda3/envs/nuplan/lib/python3.9/site-packages/bokeh/core/has_props.py", line 368, in _raise_attribute_error_with_matches raise AttributeError(f"unexpected attribute {name!r} to {self.class.name}, {text} attributes are {nice_join(matches)}") AttributeError: unexpected attribute 'on_click' to CheckboxGroup, possible attributes are active, align, aspect_ratio, classes, context_menu, css_classes, disabled, flow_mode, height, height_policy, inline, js_event_callbacks, js_property_callbacks, labels, margin, max_height, max_width, min_height, min_width, name, resizable, sizing_mode, styles, stylesheets, subscribed_events, syncable, tags, visible, width or width_policy ERROR:tornado.access:500 GET /autoload.js?bokeh-autoload-element=p1002&bokeh-absolute-url=http://localhost:34763&resources=none (::1) 55.76ms

Sorry, I cannot help you for solving this problem directly. However, I have 2 potential solutions, they are 1. try to check the version of bokeh. 2. if you use a remote machine, try to go to use that machine directly, not remotely. For me, I could not display that in a remote way, but I could display it if I use the machine directly. I hope that these suggestions could help you.

[michael-motional]

Sure, @michael-motional, I will tell you the result if it works.
Hi, thank you for your information. Until now, the nuboard could display nothing for the metrics, but I find out that in the scenario part, there are many useful information, like simulating the scenario. And since the nuplan v1.1 could directly show the metrics in the simulation part. I think this problem is not so urgent.

Hey @IvanChen777, are you saying even without altering the experiment name, you don't see metrics? If so, can you share the command you use when running the simulation. We fixed the issue referenced here in the upcoming release, but if the workaround doesn't work for you, there may be another problem. Let me know and I will look into it!

[michael-motional]

Closing, but let me know if you're noticing issues