segmentation #5

HJMGARMIN · 2023-11-01T06:21:36Z

Great work!
How can I make the same segmentation results on TUM dataset? Which param should I change in the yaml file? The second picture is my result on "rgbd_dataset_freiburg3_long_office_household" with PointCloudMapping.type: "octree_point" and Segmentation.on: 1

The text was updated successfully, but these errors were encountered:

luigifreda · 2023-11-01T09:07:06Z

Hi,
I am glad you liked PLVS. Thanks.

As you understood, to enable segmentation, you need to set PointCloudMapping.type: "octree_point" and Segmentation.on: 1
These are the parameters I normally use

This is what I normally get (in an intermediate frame) as shown in the videos

HJMGARMIN · 2023-11-01T11:36:07Z

Thanks for your reply, but it seems does not work on my device. And I notice that the color of the same objects in "connected components" is changing. My yaml file of TUM3 is below:
`%YAML:1.0

#--------------------------------------------------------------------------------------------

Viewer Parameters

#--------------------------------------------------------------------------------------------

Viewer.on: 1 is ON, 0 is OFF

Viewer.on: 1

Viewer.KeyFrameSize: 0.05
Viewer.KeyFrameLineWidth: 1.0
Viewer.GraphLineWidth: 0.9
Viewer.PointSize: 2.0
Viewer.LineSize: 1.0
Viewer.CameraSize: 0.08
Viewer.CameraLineWidth: 3.0
Viewer.ViewpointX: 0.0
Viewer.ViewpointY: -0.7
Viewer.ViewpointZ: -1.8
Viewer.ViewpointF: 500.0

Viewer.ViewpointYtop: -4.0

#--------------------------------------------------------------------------------------------

Camera Parameters. Adjust them!

#--------------------------------------------------------------------------------------------
File.version: "1.0"

Camera.type: "PinHole"

Camera calibration and distortion parameters (OpenCV)

Camera1.fx: 535.4
Camera1.fy: 539.2
Camera1.cx: 320.1
Camera1.cy: 247.6

Camera1.k1: 0.0
Camera1.k2: 0.0
Camera1.p1: 0.0
Camera1.p2: 0.0

Camera.width: 640
Camera.height: 480

Camera frames per second

Camera.fps: 30

Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)

Camera.RGB: 1

Close/Far threshold. Baseline times.

Stereo.ThDepth: 40.0
Stereo.b: 0.0747

Depth map values factor

RGBD.DepthMapFactor: 5000.0

#--------------------------------------------------------------------------------------------

ORB Parameters

#--------------------------------------------------------------------------------------------

ORB Extractor: Number of features per image

ORBextractor.nFeatures: 1000

ORB Extractor: Scale factor between levels in the scale pyramid

ORBextractor.scaleFactor: 1.2

ORB Extractor: Number of levels in the scale pyramid

ORBextractor.nLevels: 8

ORB Extractor: Fast threshold

Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.

Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST

You can lower these values if your images have low contrast

ORBextractor.iniThFAST: 20
ORBextractor.minThFAST: 7

#--------------------------------------------------------------------------------------------

Line Segments Tracking

#--------------------------------------------------------------------------------------------

Line.on: 1 is ON, 0 is OFF

Line.on: 1

set to 0 if keeping all lines

Line.nfeatures: 100

number of levels in the scale pyramid

Line.nLevels: 3

scale factor between levels in the scale pyramid

Line.scaleFactor: 1.2

noise standard deviation on line extraction (in pixel) CHANGED!

Line.sigma: 2

min. line length (relative to img size)

Line.minLineLength: 0.02

weighting of lines w.r.t. points in success/failure tracking conditions

Line.lineTrackWeigth: 2

min. line length 3D (meters)

Line.minLineLength3D: 0.01

mu weight for endpoint-endpoint dist in line 3D dist

Line.muWeightForLine3dDist: 0.3

for using LSD extractor (much slower) instead of EDLine lines extractor

Line.LSD.on: 0

#--------------------------------------------------------------------------------------------

Map Object

#--------------------------------------------------------------------------------------------

MapObject.on: 1 is ON, 0 is OFF

MapObject.on: 0

Images of the planar objects (separated by empty spaces)

MapObject.imgs: "teddy.png"
MapObject.numMinInliers: 20
MapObject.maxReprojectionError: 1
MapObject.maxSim3Error: 0.01

#--------------------------------------------------------------------------------------------

KeyFrame Generation

#--------------------------------------------------------------------------------------------

on: 1 is ON, 0 is OFF

KeyFrame.fovCentersBasedGeneration.on: 0
KeyFrame.maxFovCentersDistance: 0.4

#--------------------------------------------------------------------------------------------

Sparse Mapping

#--------------------------------------------------------------------------------------------

SparseMapping.filename: "sparse_map_rgbd.atlas"

use saved map on start: 1 is ON, 0 is OFF

SparseMapping.reuseMap: 0

freeze map during optimizations: 1 is ON, 0 is OFF

SparseMapping.freezeMap: 0

save map on shutdown: 1 is ON, 0 is OFF

SparseMapping.saveMap: 0

force immediate relocalization (or wait for loop-closing thread for relocalization): 1 is ON, 0 is OFF

SparseMapping.forceRelocalization: 1

#--------------------------------------------------------------------------------------------

Depth Noise Model

#--------------------------------------------------------------------------------------------

used for scaling depth noise model

Depth.sigmaZfactor: 6

#--------------------------------------------------------------------------------------------

PointCloud Mapping

#--------------------------------------------------------------------------------------------

PointCloudMapping.on: 1 is ON, 0 is OFF

PointCloudMapping.on: 1

#PointCloudMapping.type: voxelgrid, octomap, octree_point, chisel, fastfusion, voxblox

NOTE: chisel and fastfusion require rectified images if you do not use pointcloud generation

PointCloudMapping.type: "octree_point"

PointCloudMapping.resolution: 0.01
PointCloudMapping.numKeyframesToQueueBeforeProcessing: 1
PointCloudMapping.downSampleStep: 2
PointCloudMapping.minDepth: 0.1
PointCloudMapping.maxDepth: 5
PointCloudMapping.removeUnstablePoints: 1

PointCloudMapping.resetOnSparseMapChange: 1
PointCloudMapping.cloudDeformationOnSparseMapChange: 0

[octree_point] specific params

PointCloudMapping.pointCounterThreshold: 3

[voxblox] specific params

PointCloudMapping.voxbloxIntegrationMethod: "fast"

[octree_point, chisel, voxblox] specific params

PointCloudMapping.useCarving: 1 is ON, 0 is OFF

PointCloudMapping.useCarving: 0

depth filtering

PointCloudMapping.filterDepth.on: 0
PointCloudMapping.filterDepth.diameter: 7
PointCloudMapping.filterDepth.sigmaDepth: 0.02
PointCloudMapping.filterDepth.sigmaSpace: 5

for loading

load dense map on start: 1 is ON, 0 is OFF

PointCloudMapping.loadMap: 0

voxblox also accepts its .proto file!

PointCloudMapping.loadFilename: "volumetric_map_0.ply"

#--------------------------------------------------------------------------------------------

Segmentation

#--------------------------------------------------------------------------------------------

Segmentation.on: 1 is ON, 0 is OFF

working only with [octree_point]

Segmentation.on: 1

Segmentation.maxDepth: 2

the following params work well with PointCloudMapping.downSampleStep: 2

Segmentation.erosionDilationOn: 1
Segmentation.minFi: 0.92
Segmentation.maxDelta: 0.01
Segmentation.singleDepth.minComponentArea: 80
Segmentation.lineDrawThinckness: 2
Segmentation.maxAngleForNormalAssociation: 20
Segmentation.labelsMatchingMinOverlapPerc: 0.2
Segmentation.labelsMatchingMinOverlapPoints: 0
Segmentation.globalLabelsMatchingMinOverlapPerc: 0.2

#--------------------------------------------------------------------------------------------

Gl Objects

#--------------------------------------------------------------------------------------------

pre-load static GL objects to be visualized in the map

GlObjects.on: 0

GlObjects.list:

- { filename: "map_0.ply", type: "cloudmesh", pose: [ 0, 0, 0, 0, 0, 0] }

- { filename: "sassi.ply", type: "cloudmesh", pose: [ 0, 0, 0, 0, 0, 0] }

`
and my results are below. The segmentation results are worse than yours. The colors split on the same object are very messy.

luigifreda · 2023-11-01T22:11:05Z

Hi,
Thanks for sharing your pics. I have some questions for you:

What is your exact computer configuration? (OS, CPU, GPU, GCC version)
Are you under docker, virtual box, or in a native OS?
Which GPU are you using?
How did you exactly install PLVS?
How are you exactly running the test? Could you please attach the used script file and configuration file? (please, attach the file and do not copy-paste the text)
Did you change any part of the code? Could you please run git diff >& /tmp/log.txt and share the generated log file as an attachment?
Did you touch any GUI slider before making the snapshots?

I tested PLVS on different laptops and desktop computers (with NVIDIA GPUs). I have never seen such a result ... unless you change the parameters.

It seems you are also getting a different volumetric map with much fewer points than normal. Can you share a pic of the volumetric map you are getting?

HJMGARMIN · 2023-11-02T10:06:37Z

OS: Ubuntu 18.04
CPU: i7-11700
GPU: RTX2080
I installed PLVS using the script you perovided.
Because I can not upload .sh file, I comvert them to .txt.
run_tum_rgbd.txt
config.txt
I did not change any part of the main code.
log.txt
I just open "Show Segments" in the GUI and do not touch other GUI slider.
The volumetric map on my computer is :

When I run PLVS with segments, the state of GPU is as below.

luigifreda · 2023-11-02T21:17:22Z

I see you are using Ubuntu 18. I tested the procedures and the code under Ubuntu 20.

Can you test the code under docker with Ubuntu 20? For instance, you can use https://github.com/luigifreda/rosdocker

luigifreda · 2023-11-03T07:59:06Z

Update. I got PLVS working back on Ubuntu 18.
Indeed, i was not able to build PLVS with C++17 under Ubuntu 18. I am wondering how you got it built with C++17?? (You didn't reply to my GCC question)
Moreover, you are not using a local opencv install. Please, install a local opencv.

What I recommend on Ubuntu 18:

Revert your changes in config.sh at root level of the repository (we want to use the local opencv install)
Go into config.sh at root level of the repository and set CPP_STANDARD_VERSION=11
(here https://github.com/luigifreda/plvs/blob/master/config.sh#L14)
Clean everything by running ./clean.sh
Install a local opencv by running ./install_local_opencv.sh (as suggested in the main README)
Run ./build.sh (as suggested in the main README)

D1st3f · 2024-03-19T13:05:53Z

I have bad result form segmentation:

And when segmentation off:

Also:
PointCloudMapping.type: "octree_point"
Segmentation.on: 1

D1st3f · 2024-03-19T13:09:43Z

Another GUI screenshots:

P.S. Tried without touching sliders in GUI, same result.

D1st3f · 2024-03-19T13:24:10Z

Look's like something broken i normals

CQnancy · 2024-03-28T09:45:26Z

In file included from /home/techinao/catkin_ws/src/plvs/Thirdparty/volumetric_mapping/volumetric_mapping/volumetric_map_base/src/world_base.cc:40:0:
/usr/include/pcl_conversions/pcl_conversions.h:42:10:fatal error: ros/ros.h: No such file or directory。Do you have this question? If so, how was it resolved?

luigifreda · 2024-03-29T09:13:37Z

Did you run the script install_dependencies.sh?
What is your OS and configuration?

CQnancy · 2024-03-31T08:47:06Z

Did you run the script install_dependencies.sh? What is your OS and configuration?

I ran it one by one./clean. sh, ./install_dependency. sh, ./Install_local_opencv.sh and./build.sh, Ubuntu 18.04.6

luigifreda · 2024-04-01T12:27:43Z

Hey @CQnancy,
thanks for your feedback. I worked to improve PLVS support under Ubuntu 18.04.
You can now pull, run clean.sh, install deps, and repeat the build procedure (no need to reinstall local opencv if you already did it).

luigifreda closed this as completed Nov 1, 2023

This was referenced Mar 19, 2024

Segmentation error #27

Closed

deploy under Ubuntu 18.04 #29

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

segmentation #5

segmentation #5

HJMGARMIN commented Nov 1, 2023 •

edited

Loading

luigifreda commented Nov 1, 2023 •

edited

Loading

HJMGARMIN commented Nov 1, 2023 •

edited

Loading

luigifreda commented Nov 1, 2023 •

edited

Loading

HJMGARMIN commented Nov 2, 2023

luigifreda commented Nov 2, 2023 •

edited

Loading

luigifreda commented Nov 3, 2023 •

edited

Loading

D1st3f commented Mar 19, 2024

D1st3f commented Mar 19, 2024

D1st3f commented Mar 19, 2024

CQnancy commented Mar 28, 2024

luigifreda commented Mar 29, 2024

CQnancy commented Mar 31, 2024

luigifreda commented Apr 1, 2024 •

edited

Loading

segmentation #5

segmentation #5

Comments

HJMGARMIN commented Nov 1, 2023 • edited Loading

luigifreda commented Nov 1, 2023 • edited Loading

HJMGARMIN commented Nov 1, 2023 • edited Loading

Viewer Parameters

Viewer.on: 1 is ON, 0 is OFF

Camera Parameters. Adjust them!

Camera calibration and distortion parameters (OpenCV)

Camera frames per second

Color order of the images (0: BGR, 1: RGB. It is ignored if images are grayscale)

Close/Far threshold. Baseline times.

Depth map values factor

ORB Parameters

ORB Extractor: Number of features per image

ORB Extractor: Scale factor between levels in the scale pyramid

ORB Extractor: Number of levels in the scale pyramid

ORB Extractor: Fast threshold

Image is divided in a grid. At each cell FAST are extracted imposing a minimum response.

Firstly we impose iniThFAST. If no corners are detected we impose a lower value minThFAST

You can lower these values if your images have low contrast

Line Segments Tracking

Line.on: 1 is ON, 0 is OFF

set to 0 if keeping all lines

number of levels in the scale pyramid

scale factor between levels in the scale pyramid

noise standard deviation on line extraction (in pixel) CHANGED!

min. line length (relative to img size)

weighting of lines w.r.t. points in success/failure tracking conditions

min. line length 3D (meters)

mu weight for endpoint-endpoint dist in line 3D dist

for using LSD extractor (much slower) instead of EDLine lines extractor

Map Object

MapObject.on: 1 is ON, 0 is OFF

Images of the planar objects (separated by empty spaces)

KeyFrame Generation

on: 1 is ON, 0 is OFF

Sparse Mapping

use saved map on start: 1 is ON, 0 is OFF

freeze map during optimizations: 1 is ON, 0 is OFF

save map on shutdown: 1 is ON, 0 is OFF

force immediate relocalization (or wait for loop-closing thread for relocalization): 1 is ON, 0 is OFF

Depth Noise Model

used for scaling depth noise model

PointCloud Mapping

PointCloudMapping.on: 1 is ON, 0 is OFF

NOTE: chisel and fastfusion require rectified images if you do not use pointcloud generation

[octree_point] specific params

[voxblox] specific params

[octree_point, chisel, voxblox] specific params

PointCloudMapping.useCarving: 1 is ON, 0 is OFF

depth filtering

for loading

load dense map on start: 1 is ON, 0 is OFF

voxblox also accepts its .proto file!

Segmentation

Segmentation.on: 1 is ON, 0 is OFF

working only with [octree_point]

the following params work well with PointCloudMapping.downSampleStep: 2

Gl Objects

pre-load static GL objects to be visualized in the map

- { filename: "map_0.ply", type: "cloudmesh", pose: [ 0, 0, 0, 0, 0, 0] }

- { filename: "sassi.ply", type: "cloudmesh", pose: [ 0, 0, 0, 0, 0, 0] }

luigifreda commented Nov 1, 2023 • edited Loading

HJMGARMIN commented Nov 2, 2023

luigifreda commented Nov 2, 2023 • edited Loading

luigifreda commented Nov 3, 2023 • edited Loading

D1st3f commented Mar 19, 2024

D1st3f commented Mar 19, 2024

D1st3f commented Mar 19, 2024

CQnancy commented Mar 28, 2024

luigifreda commented Mar 29, 2024

CQnancy commented Mar 31, 2024

luigifreda commented Apr 1, 2024 • edited Loading

HJMGARMIN commented Nov 1, 2023 •

edited

Loading

luigifreda commented Nov 1, 2023 •

edited

Loading

HJMGARMIN commented Nov 1, 2023 •

edited

Loading

luigifreda commented Nov 1, 2023 •

edited

Loading

luigifreda commented Nov 2, 2023 •

edited

Loading

luigifreda commented Nov 3, 2023 •

edited

Loading

luigifreda commented Apr 1, 2024 •

edited

Loading