Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nvidia Xavier NX Kernel Patching Hunk #1 FAILED at 1262 & 641 issue: New Solution Required #8254

Closed
dlee640 opened this issue Jan 27, 2021 · 53 comments

Comments

@dlee640
Copy link

dlee640 commented Jan 27, 2021


Required Info
Camera Model { D435 }
Firmware Version (Open RealSense Viewer --> Click info)
Operating System & Version {Ubuntu 18.04}
Kernel Version (Linux Only) {4.9 Tegra}
Platform {NVIDIA Xavier NX}
SDK Version { 2.41.0 }
Language {C++/Python }
Segment {Robot}

Issue Description

I have the same issue that has been posted in #8079 by @xxEoD2242.
The ultimate solution recommended was to reflash the Xavier with Jetpack 4.4.

However, I have 4.4.1 and I cannot reflash the Xavier since the system is mounted within the robot and I do not want to dissect the hardware.

Is there a different method such as building it from RSUSB backened? Can someone guide me to a direction for not requiring a kernel?

Or does someone know how to downgrade jetpack version without having to reflash the drive?

Thank you.

More details:

My error:

Apply Librealsense Kernel Patches
patching file drivers/media/usb/uvc/Makefile
patching file drivers/media/usb/uvc/uvc_driver.c
Hunk #1 succeeded at 764 with fuzz 2 (offset 595 lines).
patching file drivers/media/usb/uvc/uvcvideo.h
Hunk #1 succeeded at 392 with fuzz 2 (offset 254 lines).
patching file drivers/media/v4l2-core/v4l2-ioctl.c
Hunk #1 FAILED at 1262.
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/v4l2-core/v4l2-ioctl.c.rej
patching file include/uapi/linux/videodev2.h
Hunk #1 FAILED at 641.
1 out of 1 hunk FAILED -- saving rejects to file include/uapi/linux/videodev2.h.rej

My jetpack version:

unitree@nx:~/librealsense$ sudo apt-cache show nvidia-jetpack
Package: nvidia-jetpack
Version: 4.4.1-b50
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.4.1-b50), nvidia-opencv (= 4.4.1-b50), nvidia-cudnn8 (= 4.4.1-b50), nvidia-tensorrt (= 4.4.1-b50), nvidia-visionworks (= 4.4.1-b50), nvidia-container (= 4.4.1-b50), nvidia-vpi (= 4.4.1-b50), nvidia-l4t-jetson-multimedia-api (>> 32.4-0), nvidia-l4t-jetson-multimedia-api (<< 32.5-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.4.1-b50_arm64.deb
Size: 29412
SHA256: ec502e1e3672c059d8dd49e5673c5b2d8c606584d4173ee514bbc4376547a171
SHA1: 75a405f1ad533bfcd04280d1f9b237b880c39be5
MD5sum: 1267b31d8b8419d9847b0ec4961b15a4
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.4-b186
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.4-b186), nvidia-opencv (= 4.4-b186), nvidia-cudnn8 (= 4.4-b186), nvidia-tensorrt (= 4.4-b186), nvidia-visionworks (= 4.4-b186), nvidia-container (= 4.4-b186), nvidia-vpi (= 4.4-b186), nvidia-l4t-jetson-multimedia-api (>> 32.4-0), nvidia-l4t-jetson-multimedia-api (<< 32.5-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.4-b186_arm64.deb
Size: 29376
SHA256: 7f90f2def712d993bd84855355aec47a1b63ae54edefcbf4c963dbe0cb6fbf92
SHA1: bd1f0b05adbdf8f0df4dce4f655bb9615172594f
MD5sum: f9d2f9a9dfee794ef361b1bd1ec38130
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8

Package: nvidia-jetpack
Version: 4.4-b144
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 195
Depends: nvidia-container-csv-cuda (= 10.2.89-1), libopencv-python (= 4.1.1-2-gd5a58aa75), libvisionworks-sfm-dev (= 0.90.4.501), libvisionworks-dev (= 1.6.0.501), libnvparsers7 (= 7.1.0-1+cuda10.2), libnvinfer-plugin-dev (= 7.1.0-1+cuda10.2), libnvonnxparsers7 (= 7.1.0-1+cuda10.2), libnvinfer-samples (= 7.1.0-1+cuda10.2), libnvinfer-bin (= 7.1.0-1+cuda10.2), libvisionworks-samples (= 1.6.0.501), libvisionworks-tracking-dev (= 0.88.2.501), vpi-samples (= 0.2.0), tensorrt (= 7.1.0.16-1+cuda10.2), libopencv (= 4.1.1-2-gd5a58aa75), libnvinfer-doc (= 7.1.0-1+cuda10.2), libnvparsers-dev (= 7.1.0-1+cuda10.2), libnvidia-container0 (= 0.9.0beta.1), nvidia-container-csv-visionworks (= 1.6.0.501), cuda-toolkit-10-2 (= 10.2.89-1), graphsurgeon-tf (= 7.1.0-1+cuda10.2), libcudnn8 (= 8.0.0.145-1+cuda10.2), libopencv-samples (= 4.1.1-2-gd5a58aa75), nvidia-container-csv-cudnn (= 8.0.0.145-1+cuda10.2), python-libnvinfer-dev (= 7.1.0-1+cuda10.2), libnvinfer-plugin7 (= 7.1.0-1+cuda10.2), libvisionworks (= 1.6.0.501), libcudnn8-doc (= 8.0.0.145-1+cuda10.2), nvidia-container-toolkit (= 1.0.1-1), libnvinfer-dev (= 7.1.0-1+cuda10.2), nvidia-l4t-jetson-multimedia-api (>> 32.4-0), nvidia-l4t-jetson-multimedia-api (<< 32.5-0), libopencv-dev (= 4.1.1-2-gd5a58aa75), vpi-dev (= 0.2.0), vpi (= 0.2.0), libcudnn8-dev (= 8.0.0.145-1+cuda10.2), python3-libnvinfer (= 7.1.0-1+cuda10.2), python3-libnvinfer-dev (= 7.1.0-1+cuda10.2), opencv-licenses (= 4.1.1-2-gd5a58aa75), nvidia-container-csv-tensorrt (= 7.1.0.16-1+cuda10.2), libnvinfer7 (= 7.1.0-1+cuda10.2), libnvonnxparsers-dev (= 7.1.0-1+cuda10.2), uff-converter-tf (= 7.1.0-1+cuda10.2), nvidia-docker2 (= 2.2.0-1), libvisionworks-sfm (= 0.90.4.501), libnvidia-container-tools (= 0.9.0beta.1), nvidia-container-runtime (= 3.1.0-1), python-libnvinfer (= 7.1.0-1+cuda10.2), libvisionworks-tracking (= 0.88.2.501)
Conflicts: cuda-command-line-tools-10-0, cuda-compiler-10-0, cuda-cublas-10-0, cuda-cublas-dev-10-0, cuda-cudart-10-0, cuda-cudart-dev-10-0, cuda-cufft-10-0, cuda-cufft-dev-10-0, cuda-cuobjdump-10-0, cuda-cupti-10-0, cuda-curand-10-0, cuda-curand-dev-10-0, cuda-cusolver-10-0, cuda-cusolver-dev-10-0, cuda-cusparse-10-0, cuda-cusparse-dev-10-0, cuda-documentation-10-0, cuda-driver-dev-10-0, cuda-gdb-10-0, cuda-gpu-library-advisor-10-0, cuda-libraries-10-0, cuda-libraries-dev-10-0, cuda-license-10-0, cuda-memcheck-10-0, cuda-misc-headers-10-0, cuda-npp-10-0, cuda-npp-dev-10-0, cuda-nsight-compute-addon-l4t-10-0, cuda-nvcc-10-0, cuda-nvdisasm-10-0, cuda-nvgraph-10-0, cuda-nvgraph-dev-10-0, cuda-nvml-dev-10-0, cuda-nvprof-10-0, cuda-nvprune-10-0, cuda-nvrtc-10-0, cuda-nvrtc-dev-10-0, cuda-nvtx-10-0, cuda-samples-10-0, cuda-toolkit-10-0, cuda-tools-10-0, libcudnn7, libcudnn7-dev, libcudnn7-doc, libnvinfer-plugin6, libnvinfer6, libnvonnxparsers6, libnvparsers6
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.4-b144_arm64.deb
Size: 30376
SHA256: 0556dbf2044a9d12e2c26e1a342033d4816f5d84f87dfe7a82916449249bb04b
SHA1: 38e788c5cd84b3e4bd956dcd7c9e018276e17b18
MD5sum: 0a96d0fa91591f10b7c86022d068970d
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8
@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jan 28, 2021

Hi @dlee640 The link below leads to a guide for using the RSUSB method to build librealsense and the Python wrapper from source code at the same time on Jetson. It has been tested successfully with Nano and Xavier NX.

#7905 (comment)

@dlee640
Copy link
Author

dlee640 commented Jan 28, 2021

@MartyG-RealSense Thank you for your time to respond, but I don't think your response regarding python wrapper addresses my question correctly. (Or maybe my understanding is incorrect, and needs more detail :))

I do have Realsense SDK working properly. My issue is getting the ROS-wrapper to run on the Tegra 4.9 Linux system, since the 4.9 isn't one of the kernel that supports ROS-wrapper. We tried to address this by following the [instruction here](https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md). We decided to use 'Building from Source using Native Backend' method, but this method only works for Jetpack version 4.4. We have 4.4.1, which throws HUNK error and we cannot revert back since the SD card slot is off the reach. Thus, we need more detail on 'Building from Source using RSUSB Backend' method also detailed in the document.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jan 29, 2021

I researched the question of the possibility of changing JetPack version without reflashing. Multiple sources said that reflashing was necessary up until JetPack 4.3, but from 4.3 onwards an APT server that houses JetPack packages was introduced.

Regarding your request for more documentation about RSUSB backend: I would recommend using the method on building from source with CMake that I linked to earlier:

#8254 (comment)

If you want to investigate the possibility of building using the libuvc_installation.sh script mentioned in the Jetson instructions, further details about use of this script can be found here:

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md

@AlexKaravaev
Copy link

Is it going to be fixed in future releases?

@ev-mp
Copy link
Collaborator

ev-mp commented Feb 1, 2021

@AlexKaravaev , this task is already prioritized in the backlog.
I cannot provide exact timeline but patches for JetPack 4.4.1 will be available within weeks.

@dlee640
Copy link
Author

dlee640 commented Feb 2, 2021

Thank you all for your help. This issue has been resolved by deleting the liberalsense installation and reinstalling ros-wrapper. However, I am excited for Jetpack 4.4.1 patches!

@dlee640 dlee640 closed this as completed Feb 2, 2021
@MartyG-RealSense
Copy link
Collaborator

Thanks very much @dlee640 for the update - great to hear that you made progress!

@Andrea2202
Copy link

Hi, all! Any updates on the patch for Jetpack 4.4.1?

Thanks

@ev-mp
Copy link
Collaborator

ev-mp commented Feb 19, 2021

@Andrea2202 , @dlee640, @AlexKaravaev the patches for 4.4.1 and 4.5 have been recently merged into development branch via #8366.
You can try it from there - feedbacks will be appreciated.

Actually, there even more good news as with the release of v4.4.1, NVidia has apparently integrated the kernel patches for specific RealSense devices, including D415 and D435(i), into L4T. So those models' owners can now receive the full capabilities of the Depth camera including formats and metadata using the native Tegra kernel (no need for patches).

@AndreV84
Copy link

AndreV84 commented Jun 8, 2021

@MartyG-RealSense
Could you extend on kernel patching for JetsonNX for d455, please?

@MartyG-RealSense
Copy link
Collaborator

@AndreV84 What aspect of NX kernel patching do you want to know about please?

@AndreV84
Copy link

AndreV84 commented Jun 8, 2021

@MartyG-RealSense we are looking to get d455 on Jetson with slamcore;
They pointed out to complex Jetsopn kernel patching briefly as per their statement jetsonNX "doesn't fully support d455' unless patched

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 8, 2021

I located a guide from October 2020 for installing a D455 on Jetson NX, auto-translated it into English and have attached it to this message as a PDF called NX-D455.

Here is the link to the original version.

https://dev.classmethod.jp/articles/use-realsense-d455-with-jetson-xavier-nx/

NX-D455.pdf

@AndreV84
Copy link

AndreV84 commented Jun 10, 2021

@MartyG-RealSense The situation is as follows
if realsense sdk gets installed from apt repository it won't see d455
However if I install realsense from sources it will see d455
Moreover, I found some issue importing pyrealsense2 into python from any other folder than /usr/local/lib/python3.6/pyrealsense2
However the slamcore won't pick up the realsense sdk built from sources but would work with realsense sdk from apt repository

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 10, 2021

So long as the librealsense version is 2.38.1 or newer then it should officially support D455. A RealSense-compatible non SDK software product may need to specifically add support for a particular camera model though.

What SDK version is installed when you are installing from apt repository please?

Also, do you experience the same problem with source-code building if you use the latest 2.47.0 librealsense version?

@AndreV84
Copy link

AndreV84 commented Jun 10, 2021

@MartyG-RealSense Their team pointed out that 2.41 realsense-viewer on NX Jetson returns the following message

Per-frame metadata is not enabled at the OS level; Please follow the installation guide for details

you will see that if you inspect the metadata, the 'Per-frame metadata is not enabled'. This metadata is essential for us to align the IMU and image sensor datastreams

@AndreV84
Copy link

AndreV84 commented Jun 10, 2021

after more reinstallations of 2.41.0 from apt repository [ it i sth elatest versions fir arm arch64 availavle;
it gets detected in realsense viewer
but when enabling stereo module it would show errors, though it would show rgb stream

realsense-viewer 
 10/06 16:34:15,976 ERROR [547826405824] (http-downloader.cpp:141) Download error from URL: http://realsense-hw-public.s3-eu-west-1.amazonaws.com/Releases/rs_versions_db.json, error info: HTTP response code said error
 10/06 16:34:21,507 ERROR [548085616656] (types.h:313) xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error
 10/06 16:35:21,458 ERROR [548085616656] (types.h:313) xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error
 10/06 16:35:26,958 WARNING [547826405824] (ds5-timestamp.cpp:69) UVC metadata payloads not available. Please refer to the installation chapter for details.
 10/06 16:35:32,845 ERROR [548085616656] (types.h:313) xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error
 10/06 16:35:50,840 WARNING [548085616656] (ds5-motion.cpp:477) IMU Calibration is not available, default intrinsic and extrinsic will be used.
 10/06 16:35:50,848 WARNING [547692970432] (backend-hid.cpp:715) HID set_power 1 failed for /sys/devices/3610000.xhci/usb2/2-3/2-3.4/2-3.4:1.5/0003:8086:0B5C.0001/HID-SENSOR-200073.1.auto/iio:device2/buffer/enable
 10/06 16:36:17,723 ERROR [548085616656

@MartyG-RealSense
Copy link
Collaborator

2.41.0 dates back to before Intel changed the links for the apt repos from http to https in March 2021 (which is likely the cause of the 'Download error from URL' line in the log above).

@AndreV84
Copy link

@MartyG-RealSense
with 2.47 built from sources

realsense-viewer
 10/06 17:51:29,011 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,011 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,011 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,011 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,012 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,012 WARNING [547757903648] (messenger-libusb.cpp:42) control_transfer returned error, index: 1, error: No data available, number: 3d
 10/06 17:51:29,012 ERROR [547885088784] (uvc-device.cpp:862) Probe-commit control transfer failed with error: RS2_USB_STATUS_PIPE

@MartyG-RealSense
Copy link
Collaborator

The last error in the list is likely a consequence of the preceding control_transfer returned warnings, which can indicate that there is a serious communication problem with the device if they are generating continuously.

Which firmware version is your camera currently using please?

@AndreV84
Copy link

@MartyG-RealSense

 firmware version: 05.12.06.00

slamcore seems only supports camera firmware versions 5.12.3.0, 5.12.6.0, 5.12.7.100, 5.12.8.200 or 5.12.11.0.

@MartyG-RealSense
Copy link
Collaborator

I would suggest upgrading to the most recent version on that list (5.12.11.0) if you are using a very recent SDK like 2.47.0.

@AndreV84
Copy link

@MartyG-RealSense
with sdk from sources 2.41 with 5.12.11.0

Backend rs2_open_multiple sensor:0x559e1e1310, profiles:0x559e2c2df0, count:2
...
10/06 18:55:15,317 ERROR [548147507216] (types.h:315) xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error
Input/output error
Error parsing config: Input/output error

with 2.41.0 from apt with same firmware:

realsense-viewer 
 10/06 19:03:02,109 ERROR [547834798528] (http-downloader.cpp:141) Download error from URL: http://realsense-hw-public.s3-eu-west-1.amazonaws.com/Releases/rs_versions_db.json, error info: HTTP response code said error
 10/06 19:03:13,305 ERROR [548092878864] (types.h:313) xioctl(VIDIOC_S_FMT) failed Last Error: Input/output error

with 5.12.11.0 2.47.0 would show multiple errors also with 5/12/11/- firmware

@MartyG-RealSense
Copy link
Collaborator

Once a RealSense installation has this many problems then I believe that it is better to start fresh with a clean Ubuntu installation than try to fix the existing one and deal with continuous different errors.

@AndreV84
Copy link

we just installed ubuntu yesterday to try d455 with slamcore
we could reflash it again

@AndreV84
Copy link

@MartyG-RealSense
after reinstalling Ubuntu
On clean install installed from apt repository 2.41.0

realsense-viewer 
 10/06 21:42:23,278 ERROR [547381821888] (http-downloader.cpp:141) Download error from URL: http://realsense-hw-public.s3-eu-west-1.amazonaws.com/Releases/rs_versions_db.json, error info: HTTP response code said error
 10/06 21:42:29,778 WARNING [547381821888] (ds5-timestamp.cpp:69) UVC metadata payloads not available. Please refer to the installation chapter for details.
 10/06 21:42:30,452 WARNING [547365036480] (hdr-merge.cpp:38) HDR Merge filter cannot process frames because relevant metadata params are missing

what about meta params missed?

@AndreV84
Copy link

realsense-viewer 
 10/06 21:46:16,981 ERROR [548145164736] (http-downloader.cpp:141) Download error from URL: http://realsense-hw-public.s3-eu-west-1.amazonaws.com/Releases/rs_versions_db.json, error info: HTTP response code said error
 10/06 21:46:34,477 WARNING [548145164736] (ds5-timestamp.cpp:69) UVC metadata payloads not available. Please refer to the installation chapter for details.
 10/06 21:46:35,188 WARNING [548136772032] (hdr-merge.cpp:38) HDR Merge filter cannot process frames because relevant metadata params are missing
 10/06 21:46:42,514 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:42,515 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:44,398 ERROR [548195520960] (error-handling.cpp:76) Error during polling error handler: get_xu(id=7) failed! Last Error: Input/output error
 10/06 21:46:44,404 ERROR [548291658176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Input/output error
 10/06 21:46:47,518 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:47,519 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:50,539 ERROR [548195520960] (error-handling.cpp:76) Error during polling error handler: get_xu(id=7) failed! Last Error: Input/output error
 10/06 21:46:50,545 ERROR [548291658176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Input/output error
 10/06 21:46:52,520 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:52,521 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:56,684 ERROR [548195520960] (error-handling.cpp:76) Error during polling error handler: get_xu(id=7) failed! Last Error: Input/output error
 10/06 21:46:56,689 ERROR [548291658176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Input/output error
 10/06 21:46:57,522 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:46:57,523 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:47:02,526 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:47:02,527 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:47:02,842 ERROR [548291658176] (global_timestamp_reader.cpp:239) Error during time_diff_keeper polling: set_xu(ctrl=1) failed! Last Error: Input/output error
 10/06 21:47:02,848 ERROR [548195520960] (error-handling.cpp:76) Error during polling error handler: get_xu(id=7) failed! Last Error: Input/output error
 10/06 21:47:07,529 WARNING [548119990720] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds
 10/06 21:47:07,530 WARNING [548145164736] (backend-v4l2.cpp:1122) Frames didn't arrived within 5 seconds

@AndreV84
Copy link

so it got broken instantly

@AndreV84
Copy link

after reboot it won't discover the camera at all

@AndreV84
Copy link

after powering off/on the unit
we got realsense viewer somehow working but for this error

Per-frame metadata is not enabled at the OS level; Please follow the installation guide for details

@AndreV84
Copy link

Screenshot from 2021-06-11 01-41-53

@AndreV84
Copy link

how to patch this in the OS for the apt installed realsense-sdk?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 11, 2021

@AndreV84 If the current problem is that you need to add per-frame metadata on a librealsense build installed from packages on Jetson: my understanding is that metadata support is provided by default when using the regular Debian packages for x86/x64, but I am not certain the same is true for the Jetson packages, and people on the JetsonHacks and Nvidia forums have reported problems with metadata support.

Installing via RSUSB or kernel-patching a source-code build should provide metadata support, but I understand that doing so has problems for your slamcore compatibility.

@AndreV84
Copy link

AndreV84 commented Jun 11, 2021

@MartyG-RealSense could you extend what is exactly

 kernel-patching a source-code build 

please?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 11, 2021

Kernel-patching a source code build on Jetson is covered in the Building from Source using Native Backend section of the official Jetson installation instructions:

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_jetson.md#building-from-source-using-native-backend

@AndreV84
Copy link

@MartyG-RealSense does the apt version work after applying the patch kernel modules script or it still will need to be sdk built from sources?
rsusb did not work for ttheir software according to my tyests it worked for realsense viewer though

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Jun 11, 2021

On x86/x64 systems at least, package and source code installs are two separate installations. If you perform both on the same computer then you can end up with a Multiple udev rules warning and have to manually remove one of the rules to make the warning disappear.

@AndreV84
Copy link

@MartyG-RealSense
could you extend on patching, please? Which exactly modules need to be build for the kernel?
the patch scriopt seems failed

Apply Librealsense Kernel Patches
patching file drivers/media/usb/uvc/Makefile
patching file drivers/media/usb/uvc/uvc_driver.c
Hunk #1 succeeded at 254 with fuzz 2 (offset 85 lines).
patching file drivers/media/usb/uvc/uvcvideo.h
Hunk #1 succeeded at 122 with fuzz 2 (offset -16 lines).
patching file drivers/media/v4l2-core/v4l2-ioctl.c
Hunk #1 FAILED at 1262.
1 out of 1 hunk FAILED -- saving rejects to file drivers/media/v4l2-core/v4l2-ioctl.c.rej
patching file include/uapi/linux/videodev2.h
Hunk #1 FAILED at 641.
1 out of 1 hunk FAILED -- saving rejects to file include/uapi/linux/videodev2.h.rej

@MartyG-RealSense
Copy link
Collaborator

It looks like the same hunk errors that were reported by dlee640 at the beginning of this case. The key difference in your situation is the need to use a camera configuration that is compatible with SLAMcore, and that dlee640 used RSUSB (which will not work with your SLAMcore setup).

A case in the link below that experienced these errors on Jetson AGX when kernel patching offers advice for resolving the problem.

#7987

@AndreV84
Copy link

AndreV84 commented Jun 11, 2021

seems somehow patched

Copying the patched modules to (~/) 

Move the modified modules into the modules tree
Insert the modified kernel modules
Replacing uvcvideo  -
Module is resident, unloading ...  succeeded. 
	Applying the patched module ...  succeeded

Script has completed. Please consult the installation guide for further instruction.

so it should work with apt installed realsense, right?

@MartyG-RealSense
Copy link
Collaborator

It looks as though the patching process completed. And you still do not have metadata support?

@AndreV84
Copy link

realsense viewer [ 2.41 built from sources with te instruction from patch doesnt see the camera
realsense viewer from apt repository do not show meta frameerror after patching
but the intended application seems won't work either

@AndreV84
Copy link

seems we got the thing working on JP 4.4.1
trying to figure out if we could have newest Jetpack 4.5.1

@MartyG-RealSense
Copy link
Collaborator

Great to hear that you made significant progress @AndreV84 :)

@AndreV84
Copy link

it seems we shoud be able to repeat on newest Jetpack release as I was able to patch Jetson with Jetpack 4.5.1

opying the patched modules to (~/) 
/home/nvidia/librealsense
Move the modified modules into the modules tree
Insert the modified kernel modules
Replacing uvcvideo  -
	Applying the patched module ...  succeeded

Script has completed. Please consult the installation guide for further instruction.

if it was as easy at countries borders with visas as it works with realsense cameras it would be a relief

@AndreV84
Copy link

while patch applied to JP 4 5 1 uses version of patch 4.4.1 it would return that it suceeded, bu twhen trying to run the software it wouldn't be able to access usb hid devices; so maybe librealsense patch doesn't support 4.5.1 Jetpack actually?

@AndreV84
Copy link

@MartyG-RealSense Could you help out figuring out solution for Jetpack 4.5.1, please?
With using the default installed from apt repository realsense 2.48.0 package we are getting the metaframe missed situation

@AndreV84
Copy link

it applies patch of version 4.4.1 to version 4.5.1 that won't work

Jetson Board (proc/device-tree/model): NVIDIA Jetson Xavier NX Developer Kit
Jetson L4T version: 32.5.1
L4T 32.5.1 to use patches revision 4.4.1

@AndreV84
Copy link

@MartyG-RealSense how we could get new release of the patch for newer version?

@AndreV84
Copy link

@MartyG-RealSense
Please disregard;
all worked

@MartyG-RealSense
Copy link
Collaborator

Thanks @AndreV84 :)

@AndreV84
Copy link

@MartyG-RealSense
Intel is said to shut down the realsense
What's next?

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Sep 2, 2021

Hi @AndreV84 My apologies for the delay in responding to your question about the future of RealSense. I am now able to provide information as an 'End of Life' notice has been released by Intel. Although some RealSense models are being discontinued, the 400 Series stereo depth cameras are continuing. The official PDF of the End of Life notice explaining the changes is attached below.

PCN118463-00.pdf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants