Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-tuning fails #34

Closed
kirbiyik opened this issue Dec 2, 2019 · 4 comments
Closed

Auto-tuning fails #34

kirbiyik opened this issue Dec 2, 2019 · 4 comments

Comments

@kirbiyik
Copy link

kirbiyik commented Dec 2, 2019

I am trying to use the application with Azure Kinect through live input. After hassling with dependencies I am stuck at a point which I think not far from the the successful end. Here is the current state:

  • Building & compiling is OK without an error. All necessary packages are found. Optional packages realsense and structure are not found which I don't intend to use.
  • k4aviewer succesfully shows depth and RGB inputs.
  • When I run ./badslam and select Azure K4A Live Input this is the error message
time                            file:line     v| 
15:02:22.304              loguru.cpp:587   INFO| arguments: ./badslam
15:02:22.304              loguru.cpp:590   INFO| Current dir: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam
15:02:22.304              loguru.cpp:592   INFO| stderr verbosity: 2
15:02:22.304              loguru.cpp:593   INFO| -----------------------------------
15:02:22.304                 main.cc:439   WARN| No auto-tuning file found -> using default parameters. GPU performance is thus probably slightly worse than it could be.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
[2019-12-02 15:02:28.858] [error] [t=7833] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/calibration/calibration.c (391): get_camera_calibration(). cJSON_Parse error .50256603956222534,0.5046575665473938,0.49263244867324829,0.492673397064209,4.5602192878723145,3.1884105205535889,0.17583206295967102,4.8896327018737793,4.7015571594238281,0.89637881517410278,0,0,3.2404401281382889E-5,-0.0001623422431293875],"ModelType":"CALIBRATION_LensDistortionModelBrownConrady"},"Location":"CALIBRATION_CameraLocationD0","Purpose":"CALIBRATION_CameraPurposeDepth","MetricRadius":1.7399997711181641,"Rt":{"Rotation":[1,0,0,0,1,0,0,0,1],"Translation":[0,0,0]},"SensorHeight":1024,"SensorWidth":1024,"Shutter":"CALIBRATION_ShutterTypeUndefined","ThermalAdjustmentParams":{"Params":[0,0,0,0,0,0,0,0,0,0,0,0]}},{"Intrinsics":{"ModelParameterCount":14,"ModelParameters":[0.50009459257125854,0.50738322734832764,0.46948739886283875,0.62577122449874878,0.75097411870956421,-2.8726491928100586,1.6259859800338745,0.62961763143539429,-2.710111141204834,1.5602593421936035,0,0,-0.000392389134503901,0.0010423335479572415],"ModelType":"CALIBRATION_LensDistortionModelBr
[2019-12-02 15:02:28.858] [error] [t=7833] /home/vsts/work/1/s/extern/Azure-Kinect-Sensor-SDK/src/sdk/k4a.c (154): calibration_create(device->depthmcu, &device->calibration) returned failure in k4a_device_open()
Stack trace:
5       0x7f3704ef188f clone + 63
4       0x7f370760e6db /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f370760e6db]
3       0x7f370583466f /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd66f) [0x7f370583466f]
2       0x560aff517d68 ./badslam(+0x74d68) [0x560aff517d68]
1       0x560aff530ce9 ./badslam(+0x8dce9) [0x560aff530ce9]
0       0x7f370783cf52 loguru::StreamLogger::~StreamLogger() + 162
15:02:28.859    input_azurekinect.cc:283   FATL| WARNING: Failed to open K4A device!
Aborted
  • Result of badslam_test can be seen in this link. (Since it was too long, I've linked the file)
  • Since it says Cuda Error: too many resources requested for launch I've followed the instructions in the README. Auto tuning CUDA block size.
    To do that I need a dataset, live input would not work as it is stated in the README. I've downloaded Cable1 from ETH3D SLAM benchmark. The autotune asks for associated.txt calibration.txt depth(folder) rgb(folder). So I downloaded all of cables_1_mono.zip, cables_1_stereo.zip, cables_1_rgbd.zip, cables_1_imu.zip, cables_1_raw.zip and combine the data in them to get associated.txt calibration.txt depth(folder) rgb(folder).

Then I run this command to auto tune.
python3 scripts/auto_tune_parameters.py ~/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam ~/projects/cables_1 --sequential_ba --sequential_loop_detection

Output is

Running: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam /home/omer/projects/cables_1 --sequential_ba --sequential_loop_detection --auto_tuning_iteration 0
time                            file:line     v| 
14:45:06.990              loguru.cpp:587   INFO| arguments: /home/omer/projects/badslam/build_RelWithDebInfo/applications/badslam/badslam /home/omer/projects/cables_1 --sequential_ba --sequential_loop_detection --auto_tuning_iteration 0
14:45:06.990              loguru.cpp:590   INFO| Current dir: /home/omer/projects/badslam/applications/badslam
14:45:06.990              loguru.cpp:592   INFO| stderr verbosity: 2
14:45:06.990              loguru.cpp:593   INFO| -----------------------------------
14:45:07.049 rgbd_video_io_tum_datas:146    ERR| Cannot read calibration file at /home/omer/projects/cables_1/calibration.txt
14:45:07.049                 main.cc:515    ERR| Could not read dataset at /home/omer/projects/cables_1
14:45:07.056              loguru.cpp:467   INFO| atexit
Program call failed (return code: 1)

It says it cannot read calibration file and dataset. I am not sure if there is a problem in calibration file. I've also checked permissions of file, it's fine. Here is the content of calibration.txt

726.28741455078 726.28741455078 354.6496887207 186.46566772461

I can provide more logs if needed. Thanks in advance!

@puzzlepaint
Copy link
Collaborator

Thanks for the detailed report. It seems that there are two different issues here. First, the failure of the Azure Kinect SDK to initialize. I googled the error message and got this, does this help?

Second, the failure of BAD SLAM to load the calibration file. It is unclear to me why this fails. The content of the file looks fine, and since it fails in line 146, the program apparently also succeeds to open the file. However, it cannot parse the text for some reason. It should however be easy to debug this by inserting some debug output around here, for example output the content of string line and the result of sscanf.

@kirbiyik
Copy link
Author

kirbiyik commented Dec 2, 2019

Thanks @puzzlepaint. Regarding this, I've installed the SDK using provided Debian package. It's not possible to apply that change without building from source, right?

I could not find why it cannot parse the string line properly either. I just used stringstream to parse the line. So I run the auto-tuning it produced auto_tuning_result.txt which I then copied it to the resources directory. So now program sees the parameters but again I get Cuda Error: too many resources requested for launch error. I can show the output of test if you need.

Maybe I can decrease some parameters manually? But I don't know which ones to play with. Let me append the file here so you might suggest some tweaks. I have a GeForce RTX 2080 Ti btw.

MarkDeletedSurfelsCUDAKernel 64 1 0.000855296
ResetSurfelAccumForSurfelDeletionAndRadiusUpdateCUDAKernel<true> 256 1 0.000872832
AccumulatePoseEstimationCoeffsCUDAKernel 512 1 2.45507
FlagInvalidSurfelsInAccum2CUDAKernel 1024 1 0.00171536
UpdateSurfelPositionAndDescriptorCUDAKernel 512 1 0.00588928
ResetSurfelAccumCUDAKernel 512 1 0.00317818
AccumulateSurfelNormalOptimizationCoeffsCUDAKernel 512 1 0.433616
ResetSurfelAccum0to3CUDAKernel 512 1 0.00247328
DetermineActiveSurfelsKernel 512 1 0.194624
SetSurfelInactiveKernel 512 1 0.00260352
ComputePointRadiiAndRemoveIsolatedPixelsCUDAKernel 8 8 0.0173524
CountObservationsAndFreeSpaceViolationsCUDAKernel<true> 128 1 0.0705227
ComputeNormalsCUDAKernel 8 8 0.0221777
CreateSurfelsForKeyframeCUDAWriteFreeSpotListKernel 256 1 0.00109696
CreateSurfelsForKeyframeCUDASerializingKernel 32 16 0.0718495
ComputeMinMaxDepthCUDAKernel 32 16 0.0030521
DownsampleImagesCUDAKernel 8 8 0.119278
AccumulateSurfelPositionAndDescriptorOptimizationCoeffsCUDAKernel<true> 512 1 0.952228
CreateSurfelsForKeyframeCUDACreationAppendKernel 16 8 0.0784308
AccumulatePoseEstimationCoeffsFromImagesCUDAKernel_GradientXY 16 16 0.951445
CalibrateDepthCUDAKernel 8 8 0.0174465
CompactSurfelsCUDAKernel 256 1 0.0019665
CalibrateDepthAndTransformColorToDepthCUDAKernel 32 16 0.0195476
ComputeCostAndResidualCountFromImagesCUDAKernel_GradientXY 16 16 0.187392
BilateralFilteringAndDepthCutoffCUDAKernel 8 8 0.0591326
DetermineSupportingSurfelsCUDAKernel 32 1 0.452985
UpdateSurfelNormalCUDAKernel 512 1 0.00290547
WriteNewSurfelIndexAndInitializeObservationsCUDAKernel 256 1 0.0589264
ComputeBrightnessKernel 16 16 0.10004
FilterNewSurfelsCUDAKernel 32 1 0.036166
CountObservationsForNewSurfelsCUDAKernel 32 1 3.35606

@puzzlepaint
Copy link
Collaborator

I don't think that it is necessary to modify the SDK. You could for example try to insert the suggested fix

    QLocale::setDefault(QLocale::C);
    setlocale(LC_ALL, "C");

here after creating the QApplication object, as it seems that the construction of QApplication performs the locale change that seems to cause this issue.

It is surprising to me that the "too many resources requested" error happens for a GeForce RTX 2080 Ti. But it is expected that the test won't work, as they will not pick up the changed parameters from the auto tuning file (as indicated in the Readme). It should work in the actual application. If you want to fix the tests, you could try to insert the following:

    boost::filesystem::path program_dir = boost::filesystem::path(argv[0]).parent_path();
    if (!CUDAAutoTuner::Instance().LoadParametersFile(
        (program_dir / "resources"  / "auto_tuning_result.txt").string().c_str())) {
      LOG(WARNING) << "No auto-tuning file found -> using default parameters."
                      " GPU performance is thus probably slightly worse than it"
                      " could be.";
    }

in the test main function here (and probably #include <libvis/cuda/cuda_auto_tuner.h>) before it runs the tests. Also, this procedure will only work if the problematic CUDA kernel is run during creation of the auto tuning file (I haven't checked this here).

@kirbiyik
Copy link
Author

kirbiyik commented Dec 5, 2019

It seems that your advice worked. Appreciate your help, @puzzlepaint. Also, the output is really promising. Thanks for open sourcing such a work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants