Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running demo, using 04.bag, I see nothing. #8

Closed
MontyTHall opened this issue Sep 26, 2018 · 11 comments
Closed

Running demo, using 04.bag, I see nothing. #8

MontyTHall opened this issue Sep 26, 2018 · 11 comments

Comments

@MontyTHall
Copy link

MontyTHall commented Sep 26, 2018 •
edited
Loading

When I launch the limo node, I see the following flash by quickly
[ERROR] [1537984722.112431485]: Skipped loading plugin with error: XML Document '/home/xzcjml/catkin_limo/src/mono_lidar_depth/tracklets_depth_ros_tool/nodelet_plugins.xml' has no Root Element. This likely means the XML is malformed or missing..

and also
[ERROR] [1537984722.275178784]: "sensor/camera" passed to lookupTransform argument target_frame does not exist.

But generally, the console is emitting messages. Here's ros starup before I unpause rosbag player

SUMMARY
========

PARAMETERS
 * /add_outlier_flag_node/has_depth: True
 * /add_outlier_flag_node/publisher_topic: /tracklets_depth/...
 * /add_outlier_flag_node/subscriber_depth_topic: /tracklets_depth/...
 * /add_outlier_flag_node/verbosity: info
 * /gamma_correction_nodelet_left/auto_gamma: True
 * /gamma_correction_nodelet_left/gamma: 1.2
 * /label_dilation_nodelet/half_kernel_size: 8
 * /label_dilation_nodelet/publisher_topic: /semantic_labels/...
 * /label_dilation_nodelet/subscriber_topic: /sensor/camera/co...
 * /label_dilation_nodelet/verbosity: info
 * /mono_lidar/active_path_publisher_topic: /estimate/active_...
 * /mono_lidar/calib_source_frame_id: estimate/vehicle
 * /mono_lidar/calib_target_frame_id: estimate/sensor/c...
 * /mono_lidar/camera_info_subscriber_topic: /sensor/camera/gr...
 * /mono_lidar/critical_rotation_difference: 0.03
 * /mono_lidar/dump_path: /tmp/poses_dump.txt
 * /mono_lidar/height_over_ground: 0.31
 * /mono_lidar/landmarks_publisher_topic: /estimate/landmarks
 * /mono_lidar/max_number_landmarks_far_bin: 400
 * /mono_lidar/max_number_landmarks_middle_bin: 400
 * /mono_lidar/max_number_landmarks_near_bin: 400
 * /mono_lidar/max_size_optimization_window: 12
 * /mono_lidar/max_solver_time: 0.4
 * /mono_lidar/min_median_flow: 5.0
 * /mono_lidar/min_number_connecting_landmarks: 18
 * /mono_lidar/outlier_labels_yaml: /home/xzcjml/catk...
 * /mono_lidar/outlier_rejection_num_iterations: 1
 * /mono_lidar/outlier_rejection_quantile: 0.95
 * /mono_lidar/path_publisher_topic: /estimate/complet...
 * /mono_lidar/prior_global_frame: estimate_prior/lo...
 * /mono_lidar/prior_speed: 5.0
 * /mono_lidar/prior_vehicle_frame: 
 * /mono_lidar/robust_loss_depth_thres: 0.16
 * /mono_lidar/robust_loss_reprojection_thres: 1.6
 * /mono_lidar/show_debug_image: False
 * /mono_lidar/shrubbery_weight: 0.9
 * /mono_lidar/tf_child_frame_id: estimate/vehicle
 * /mono_lidar/tf_parent_frame_id: estimate/local_cs...
 * /mono_lidar/tf_waiting_time: 0.1
 * /mono_lidar/time_between_keyframes_sec: 0.3
 * /mono_lidar/tracklets_subscriber_topic: /tracklets_depth/...
 * /mono_lidar/trajectory_publisher_topic: /estimate/trajectory
 * /mono_lidar/verbosity: debug
 * /rosdistro: kinetic
 * /rosversion: 1.12.14
 * /semantic_labels_nodelet/matches_publisher_topic: /tracklets_depth/...
 * /semantic_labels_nodelet/roi_height: 3
 * /semantic_labels_nodelet/roi_width: 3
 * /semantic_labels_nodelet/subscriber_labels_topic: /semantic_labels/...
 * /semantic_labels_nodelet/subscriber_matches_topic: /tracklets_depth/...
 * /semantic_labels_nodelet/verbosity: debug
 * /tf2_static_alias_vehicle_to_rig_left_estimate/from_source_frame_id: sensor/camera/gra...
 * /tf2_static_alias_vehicle_to_rig_left_estimate/from_target_frame_id: sensor/camera
 * /tf2_static_alias_vehicle_to_rig_left_estimate/to_source_frame_id: estimate/sensor/c...
 * /tf2_static_alias_vehicle_to_rig_left_estimate/to_target_frame_id: estimate/sensor/c...
 * /tf2_static_alias_vehicle_to_rig_right_estimate/from_source_frame_id: sensor/camera/gra...
 * /tf2_static_alias_vehicle_to_rig_right_estimate/from_target_frame_id: sensor/camera
 * /tf2_static_alias_vehicle_to_rig_right_estimate/to_source_frame_id: estimate/sensor/c...
 * /tf2_static_alias_vehicle_to_rig_right_estimate/to_target_frame_id: estimate/sensor/c...
 * /tracklets_depth_node/config_depth_estimator: /home/xzcjml/catk...
 * /tracklets_depth_node/config_tracklet_depth: /home/xzcjml/catk...
 * /tracklets_depth_node/verbosity: info
 * /use_sim_time: True
 * /viso_feature_tracking_nodelet_left/config: /home/xzcjml/catk...
 * /viso_feature_tracking_nodelet_left/verbosity: info

NODES
  /
    add_outlier_flag_node (matches_conversion_ros_tool/add_outlier_flag)
    feature_matching_nodelet_manager (nodelet/nodelet)
    gamma_correction_nodelet_left (nodelet/nodelet)
    label_dilation_nodelet (nodelet/nodelet)
    mono_lidar (keyframe_bundle_adjustment_ros_tool/mono_lidar)
    semantic_labels_nodelet (nodelet/nodelet)
    semantic_labels_nodelet_manager (nodelet/nodelet)
    tf2_static_alias_vehicle_to_rig_left_estimate (util_nodes_tf2_ros_tool/static_transform_alias)
    tf2_static_alias_vehicle_to_rig_right_estimate (util_nodes_tf2_ros_tool/static_transform_alias)
    tracklets_depth_node (tracklets_depth_ros_tool/tracklet_depth_node)
    transform_local_cs (tf2_ros/static_transform_publisher)
    transform_local_cs_camera_gt (tf2_ros/static_transform_publisher)
    transform_local_cs_left_camera_estimate_prior (tf2_ros/static_transform_publisher)
    transform_local_cs_vehcile_estimate (tf2_ros/static_transform_publisher)
    transform_local_estimate_prior (tf2_ros/static_transform_publisher)
    transform_vehicle_camera_vehicle_gt (tf2_ros/static_transform_publisher)
    transform_vehicle_left_camera_estimate (tf2_ros/static_transform_publisher)
    transform_vehicle_left_camera_estimate_prior (tf2_ros/static_transform_publisher)
    viso_feature_tracking_nodelet_left (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[feature_matching_nodelet_manager-1]: started with pid [18972]
process[gamma_correction_nodelet_left-2]: started with pid [18973]
process[viso_feature_tracking_nodelet_left-3]: started with pid [18974]
process[tracklets_depth_node-4]: started with pid [18983]
process[tf2_static_alias_vehicle_to_rig_left_estimate-5]: started with pid [19000]
[ INFO] [1537985361.460632902]: Loading nodelet /viso_feature_tracking_nodelet_left of type viso_feature_tracking_ros_tool/VisoFeatureTrackingNodelet to manager feature_matching_nodelet_manager with the following remappings:
[ INFO] [1537985361.460684455]: /src/image -> /sensor/camera/grayscale/left/image_rect_g
[ INFO] [1537985361.460697607]: /tgt/matches -> /matches/grayscale/left
[ERROR] [1537985361.462564799]: Skipped loading plugin with error: XML Document '/home/xzcjml/catkin_limo/src/mono_lidar_depth/tracklets_depth_ros_tool/nodelet_plugins.xml' has no Root Element. This likely means the XML is malformed or missing..
[ INFO] [1537985361.462697064]: waitForService: Service [/feature_matching_nodelet_manager/load_nodelet] has not been advertised, waiting...
process[tf2_static_alias_vehicle_to_rig_right_estimate-6]: started with pid [19013]
process[transform_vehicle_left_camera_estimate-7]: started with pid [19027]
[ INFO] [1537985361.478147242]: Initializing nodelet with 8 worker threads.
process[transform_local_cs_vehcile_estimate-8]: started with pid [19040]
process[transform_vehicle_left_camera_estimate_prior-9]: started with pid [19068]
process[transform_local_cs_left_camera_estimate_prior-10]: started with pid [19086]
process[transform_local_estimate_prior-11]: started with pid [19094]
process[transform_local_cs-12]: started with pid [19107]
process[transform_vehicle_camera_vehicle_gt-13]: started with pid [19135]
Starting node: TrackletDepth Estimator 
process[transform_local_cs_camera_gt-14]: started with pid [19152]
process[add_outlier_flag_node-15]: started with pid [19162]
process[semantic_labels_nodelet_manager-16]: started with pid [19176]
process[label_dilation_nodelet-17]: started with pid [19187]
process[semantic_labels_nodelet-18]: started with pid [19198]
process[mono_lidar-19]: started with pid [19211]
[ERROR] [1537985361.613059464]: Skipped loading plugin with error: XML Document '/home/xzcjml/catkin_limo/src/mono_lidar_depth/tracklets_depth_ros_tool/nodelet_plugins.xml' has no Root Element. This likely means the XML is malformed or missing..
[ INFO] [1537985361.687958870]: Loading nodelet /label_dilation_nodelet of type image_preproc_ros_tool/LabelDilationNodelet to manager semantic_labels_nodelet_manager with the following remappings:
[ INFO] [1537985361.690018349]: Loading nodelet /semantic_labels_nodelet of type matches_conversion_ros_tool/SemanticLabelsNodelet to manager semantic_labels_nodelet_manager with the following remappings:
Initalize parameters
[ INFO] [1537985361.703956360]: waitForService: Service [/semantic_labels_nodelet_manager/load_nodelet] has not been advertised, waiting...
config_tracklets_depth: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/config_tracklets_depth_ros_tool.yaml
[ INFO] [1537985361.705460324]: waitForService: Service [/semantic_labels_nodelet_manager/load_nodelet] has not been advertised, waiting...
[ INFO] [1537985361.706346711]: Initializing nodelet with 8 worker threads.
config_tracklets_depth: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/mono_lidar_fusion_parameters.yaml
Load settings from: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/config_tracklets_depth_ros_tool.yaml

Load settings from: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/mono_lidar_fusion_parameters.yaml

Initialize DepthEstimator Pre
Config file DepthEstimator: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/mono_lidar_fusion_parameters.yaml
Load settings from: /home/xzcjml/catkin_limo/src/limo/demo_keyframe_bundle_adjustment_meta/res/mono_lidar_fusion_parameters.yaml

DepthEstimator parameters: 

_____ pixel area _____ 
neighbor_search_mode: 0
pixelarea_search_witdh: 6
pixelarea_search_height: 9
pixelarea_search_offset_x: 0
pixelarea_search_offset_y: 0

_____ Nearest Neighbors search _____
do_use_nearestNeighborSearch: 0
nnSearch_count10

_____ Radius search _____

do_use_radiusSearch: 1
radiusSearch_radius: 10
radiusSearch_count_min: 3

_____ Histogram Segmentation _____

do_use_histogram_segmentation: 1
histogram_segmentation_bin_witdh: 0.3
histogram_segmentation_min_pointcount: 1

_____ Depth segmentation _____

do_use_depth_segmentation: 0
depth_segmentation_max_treshold_gradient: 10
depth_segmentation_max_neighbor_distance: 0.2
depth_segmentation_max_neighbor_distance_gradient: 0.02
depth_segmentation_max_neighbor_to_seepoint_distance: 0.5
depth_segmentation_max_neighbor_to_seepoint_distance_gradient: 0.05
depth_segmentation_max_seedpoint_to_seepoint_distance: 0.05
depth_segmentation_max_seedpoint_to_seepoint_distance_gradient: 0.5
depth_segmentation_max_pointcount: 4

____ depth tresholds _____
treshold_depth_enabled: 1
treshold_depth_mode: 0
treshold_depth_max: 100
treshold_depth_min: 0

_____ depth local reshold _____
treshold_depth_local_enabled: 1
treshold_depth_local_mode: 0
treshold_depth_local_valuetype: 1
treshold_depth_local_value: 0.5

_____ PCA _____ 
do_use_PCA: 0
pca_debug: 1
pca_treshold_3_abs_min: 0.005
pca_treshold_3_2_rel_max: 15
pca_treshold_2_1_rel_min: 1.5

_____ ransac plane estimation _____ 
do_use_ransac_plane: 1
ransac_plane_distance_treshold: 0.2
ransac_plane_max_iterations: 600
ransac_plane_probability: 0.99
ransac_plane_use_refinement: 1
ransac_plane_refinement_treshold: 10.2
ransac_plane_use_camx_treshold: 0
ransac_plane_treshold_camx: 0.2
ransac_plane_point_distance_treshold: 0.2
ransac_plane_debug_visualize: 0

_____ Plane Depth Estimation ____ 
plane_estimator_use_triangle_maximation: 0
plane_estimator_z_x_min_relation: 0
plane_estimator_use_leastsquares: 0
plane_estimator_use_mestimator: 1
_____ misc _____ 
do_use_cut_behind_camera: 1
do_use_triangle_size_maximation: 1
do_check_triangleplanar_condition: 1
triangleplanar_crossnorm_treshold: 0.1
viewray_plane_orthoganality_treshold: 0.1
set_all_depths_to_zero: 0

do_debug_singleFeatures: 0
do_publish_points: 0
do_depth_calc_statistics: 0

Pusblisher name pointcloud camera cs: /monolidar_debug/cloud_camera_cs
Publisher name pointcloud interpolated: /monolidar_debug/cloud_interpolated
Publisher name image projection cloud: /monolidar_debug/image_cloud_projection
Publisher name calc depth stats: /monolidar_debug/depthcalc_stats
Publisher name tracklets depth with depth: /tracklets_depth/tracklets/left
[ INFO] [1537985361.808389733, 0.000000001]: Waiting for the ROSBAG to start
[ERROR] [1537985361.808679949, 0.000000001]: "sensor/camera" passed to lookupTransform argument target_frame does not exist. 
[ INFO] [1537985361.812748781, 0.000000001]: Started '/tf2_static_alias_vehicle_to_rig_left_estimate' in namespace '/'.
Subscribed topics: 
/clock
/tf
/tf_static
Advertised topics: 
/rosout
/tf2_static_alias_vehicle_to_rig_left_estimate/parameter_descriptions
/tf2_static_alias_vehicle_to_rig_left_estimate/parameter_updates
/tf_static
/diagnostics

[ INFO] [1537985361.823090849, 0.000000001]: Started '/tf2_static_alias_vehicle_to_rig_right_estimate' in namespace '/'.
Subscribed topics: 
/clock
/tf
/tf_static
Advertised topics: 
/rosout
/tf2_static_alias_vehicle_to_rig_right_estimate/parameter_descriptions
/tf2_static_alias_vehicle_to_rig_right_estimate/parameter_updates
/tf_static
/diagnostics

[ INFO] [1537985361.835780134]: Started '/add_outlier_flag_node' in namespace '/'.
Subscribed topics: 
/clock
/tracklets_depth/tracklets/left
/add_outlier_flag_node/in_topic
Advertised topics: 
/rosout
/add_outlier_flag_node/parameter_descriptions
/add_outlier_flag_node/parameter_updates
/tracklets_depth/tracklets/left/with_outlier_dummy

[DEBUG] [1537985361.889962880]: Verbosity set to debug
[DEBUG] [1537985361.948585216]: [mono_lidar]
Node mono_lidar has the following parameters:
	/mono_lidar/verbosity:debug
	/mono_lidar/verbosity_debug:debug
	/mono_lidar/verbosity_info:info
	/mono_lidar/verbosity_warning:warning
	/mono_lidar/verbosity_error:error
	/mono_lidar/verbosity_fatal:fatal
	/mono_lidar/diag_pub_topic:out_topic_diagnosed
	/mono_lidar/diagnostic_updater_name:MonoLidarUpdater
	/mono_lidar/diagnostic_updater_hardware_id:MonoLidar
	/mono_lidar/diagnostic_updater_rate:1
	/mono_lidar/diagnostic_updater_rate_tolerance:1
	/mono_lidar/prior_vehicle_frame:
	/mono_lidar/prior_global_frame:estimate_prior/local_cs_vehicle
	/mono_lidar/tf_waiting_time:0.1
	/mono_lidar/calib_target_frame_id:estimate/sensor/camera/grayscale/left
	/mono_lidar/calib_source_frame_id:estimate/vehicle
	/mono_lidar/tf_parent_frame_id:estimate/local_cs_vehicle
	/mono_lidar/tf_child_frame_id:estimate/vehicle
	/mono_lidar/max_number_landmarks_near_bin:400
	/mono_lidar/max_number_landmarks_middle_bin:400
	/mono_lidar/max_number_landmarks_far_bin:400
	/mono_lidar/time_between_keyframes_sec:0.3
	/mono_lidar/min_median_flow:5
	/mono_lidar/critical_rotation_difference:0.03
	/mono_lidar/robust_loss_depth_thres:0.16
	/mono_lidar/robust_loss_reprojection_thres:1.6
	/mono_lidar/min_number_connecting_landmarks:18
	/mono_lidar/max_size_optimization_window:12
	/mono_lidar/shrubbery_weight:0.9
	/mono_lidar/outlier_rejection_quantile:0.95
	/mono_lidar/outlier_rejection_num_iterations:1
	/mono_lidar/height_over_ground:0.31
	/mono_lidar/prior_speed:5
	/mono_lidar/show_debug_image:0
	/mono_lidar/max_solver_time:0.4
	/mono_lidar/dump_path:/tmp/poses_dump.txt
	/mono_lidar/outlier_labels_yaml:/home/xzcjml/catkin_limo/src/limo/keyframe_bundle_adjustment_ros_tool/res/outlier_labels.yaml
	/mono_lidar/tracklets_subscriber_topic:/tracklets_depth/tracklets/left/with_outlier_label
	/mono_lidar/tracklets_subscriber_queue_size:5
	/mono_lidar/camera_info_subscriber_topic:/sensor/camera/grayscale/left/camera_info
	/mono_lidar/camera_info_subscriber_queue_size:5
	/mono_lidar/active_path_publisher_topic:/estimate/active_path
	/mono_lidar/active_path_publisher_queue_size:5
	/mono_lidar/path_publisher_topic:/estimate/complete_path
	/mono_lidar/path_publisher_queue_size:5
	/mono_lidar/landmarks_publisher_topic:/estimate/landmarks
	/mono_lidar/landmarks_publisher_queue_size:5
	/mono_lidar/planes_publisher_topic:planes_topic
	/mono_lidar/planes_publisher_queue_size:5

[DEBUG] [1537985361.949780054]: Depth thres=0.16
[DEBUG] [1537985361.949820374]: Repr thres=1.6
[DEBUG] [1537985361.949852170]: Quantile=0.95
[DEBUG] [1537985361.949876629]: Number iterations=1
[DEBUG] [1537985361.951116869]: MonoLidar: outlier labels 
 -1 0 1 2 3 5 24 25 26 27 28 29 30 31 32 33

[DEBUG] [1537985361.951884009]: MonoLidar: shrubbery weight: 0.9 labels: 
 21 22 23

[DEBUG] [1537985361.978310628]: MonoLidar: Getting transform from vehicle to camera frame

Adding output topics to visualize from rviz didn't help either. However, I was able to visualize the input images, left and right and the velodyne point cloud. So the bag player is working.
@johannes-graeter
Copy link
Owner

johannes-graeter commented Sep 27, 2018 •
edited
Loading

I forgot to add the nodelet_plugins.xml -.-
Should be fixed by now...
just pull https://github.com/johannes-graeter/mono_lidar_depth again, resource and tell me if it works now.

@johannes-graeter johannes-graeter mentioned this issue Sep 27, 2018
Closed
@MontyTHall
Copy link
Author

The error message went away but nothing shows up in rviz. I just noticed that there's a virtual box image, I'm going to give that a spin.

@zxhou
Copy link

zxhou commented Oct 27, 2018

I meet the same problem. The program is running but there is nothing shows up in rviz. Although I changed several fixed frame, it still doesn't work.

@tuandle
Copy link

tuandle commented Oct 29, 2018

@johannes-graeter Thanks for open-source this.
Here is what I got from Rviz. However, I am not sure what to make out of it. The trajectories (blue-active path and green-complete path) and estimate landmarks just stopped being outputted midway. Is that because I am running in debug mode?
Also, is it possible that you share a video screen capture of your PC running the package with 04.bag? So that we can have a general idea of what to expect :)
ask_limo
Thanks again!

@johannes-graeter
Copy link
Owner

By "debug mode" you mean that you compile the program with debug compiler flags or do you compile it with the script I supply in the repo? "debug" ros output to the terminal should not be a problem.
What kind of machine are you running it on? (CPU specs and RAM)

@tuandle
Copy link

tuandle commented Oct 29, 2018 •
edited
Loading

@johannes-graeter yes, I used your script.
My laptop specs is like this:

tuan@tuan:~$ sudo lshw -short
H/W path         Device     Class          Description
======================================================
                            system         GL502VSK
/0                          bus            GL502VSK
/0/0                        memory         64KiB BIOS
/0/8                        memory         32GiB System Memory
/0/8/0                      memory         16GiB SODIMM Synchronous 2400 MHz (0,4 ns)
/0/8/1                      memory         [empty]
/0/8/2                      memory         16GiB SODIMM Synchronous 2400 MHz (0,4 ns)
/0/8/3                      memory         [empty]
/0/e                        memory         256KiB L1 cache
/0/f                        memory         1MiB L2 cache
/0/10                       memory         6MiB L3 cache
/0/11                       processor      Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz

@johannes-graeter
Copy link
Owner

Ok thats odd. Any error message thrown when the trajectory stops being published?

@tuandle
Copy link

tuandle commented Oct 29, 2018 •
edited
Loading

@johannes-graeter No, I don't think there is any error output, except for the very beginning when it complains about sensor/camera frame can't be found. I capture my screen here.
The final screenshot is like this:
ask_limo2
So is this the expected output?

@johannes-graeter
Copy link
Owner

thanks for the screencast, that was helpful.
so the problem is that even though the specs of your computer look ok, the optimization for the full bundle adjustment each time a keyframe is chosen takes way too long (around 1 sec which is 4 times more than on my computer with similar specs). That's why after a while queues of the rosnodes overflow and data gets lost... This is a bit hard to debug from remote though... First thing you can check is if ceres is build in release mode. Did you build it on your machine or did you use the prebuilts from apt-get install? We'll get to it ;)

This was referenced Oct 29, 2018
Closed
Closed
@johannes-graeter
Copy link
Owner

johannes-graeter commented Oct 29, 2018 •
edited
Loading

made a new issue since this got off-topic.
@tuandle please respond there.

@gychen-n
Copy link

The error message went away but nothing shows up in rviz. I just noticed that there's a virtual box image, I'm going to give that a spin.

Hello, have you solved this error? How?
[ERROR] [1537984722.275178784]: "sensor/camera" passed to lookupTransform argument target_frame does not exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@MontyTHall @tuandle @johannes-graeter @zxhou @gychen-n