ELP camera (on Jetson TX2)

[agasser]

The installation documentation suggests PhotonVision will run on an NVidia Jetson, but it does not work.

First, the installation script requires curl, but it is not pre-installed on the Jetson. An easy workaround is to manually install it first:

sudo apt update
sudo apt install curl

Once PhotonVision is running successfully, it is not able to use the attached USB camera. The following output prints repeatedly:

CS: WARNING: vi-output,_ov5693_2-0036: invalid JPEG image received from camera (UsbCameraImpl.cpp:528)

I attempted this with a Jetson TX2 using a USB ELP camera.

[mcm001]

If you could send log files that would be helpful, but based on this it sounds like CSCore doesn't like the USB camera you used? Do you have any others you can try?

[agasser]

I think you're onto something. It looks like it's actually the camera onboard the Jetson TX2 that cscore doesn't like. When I try using cscore with python and that camera I get the same message.

However, it doesn't matter which camera I select, I am unable to get a stream to appear.

Here is the log output:

[2022-10-05 22:04:51] [General - Main] [ERROR] Failed to parse command-line options!
[2022-10-05 22:04:51] [General - Main] [INFO] Logging initialized in debug mode.
[2022-10-05 22:04:51] [General - Main] [INFO] Starting PhotonVision version v2022.2.0 on Unknown Platform. OS: Linux, Architecture: aarch64
[2022-10-05 22:04:51] [General - Main] [INFO] Native libraries loaded.
[2022-10-05 22:04:51] [General - ConfigManager] [INFO] Loading settings...
[2022-10-05 22:04:51] [General - ConfigManager] [INFO] Hardware config does not exist! Loading defaults
[2022-10-05 22:04:51] [General - ConfigManager] [INFO] Hardware settings does not exist! Loading defaults
[2022-10-05 22:04:51] [General - ConfigManager] [INFO] Network config file does not exist! Loading defaults
[2022-10-05 22:04:51] [General - ConfigManager] [DEBUG] Cameras config folder did not exist. Created!
[2022-10-05 22:04:51] [General - NetworkTablesManager] [INFO] Starting NT Client
[2022-10-05 22:04:51] [General - NetworkTablesManager] [ERROR] [NetworkTablesManager] Could not connect to the robot! Will retry in the background...
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [DEBUG] Matching 3 new cameras!
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Adding local video device - "H264 USB Camera" at "/dev/video0"
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Adding local video device - "H264 USB Camera" at "/dev/video1"
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Adding local video device - "vi-output, ov5693 2-0036" at "/dev/video2"
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [DEBUG] After matching loaded configs 3 cameras were unmatched.
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Creating a new camera config for camera H264_USB_Camera
[2022-10-05 22:04:52] [Camera - CameraConfiguration] [DEBUG] Creating USB camera configuration for UsbCameraH264 USB Camera (AKA H264_USB_Camera) at /dev/video0
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Creating a new camera config for camera H264_USB_Camera (1)
[2022-10-05 22:04:52] [Camera - CameraConfiguration] [DEBUG] Creating USB camera configuration for UsbCameraH264 USB Camera (AKA H264_USB_Camera (1)) at /dev/video1
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [INFO] Creating a new camera config for camera vi-output,_ov5693_2-0036
[2022-10-05 22:04:52] [Camera - CameraConfiguration] [DEBUG] Creating USB camera configuration for UsbCameravi-output, ov5693 2-0036 (AKA vi-output,_ov5693_2-0036) at /dev/video2
[2022-10-05 22:04:52] [Camera - VisionSourceManager] [DEBUG] Matched or created 3 camera configs!
[2022-10-05 22:04:52] [WebServer - DataChangeService] [DEBUG] Added subscriber - Sources: DCS_WEBSOCKET, DCS_HTTP, DCS_NETWORKTABLES, DCS_VISIONMODULE, DCS_OTHER, Destinations: DCD_UI
[2022-10-05 22:04:52] [WebServer - DataChangeService] [DEBUG] Added subscriber - Sources: DCS_WEBSOCKET, Destinations: DCD_GENSETTINGS
[2022-10-05 22:04:53] [Camera - USBCameraSource - H264_USB_Camera (1)] [ERROR] Got a null video mode! Doing nothing...
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] Got a null video mode! Doing nothing...
[2022-10-05 22:04:53] [Camera - VisionSourceManager] [DEBUG] Matched config for camera "USBFrameProvider - opencv_H264_USB_Camera" and loaded 0 pipelines
[2022-10-05 22:04:53] [Camera - VisionSourceManager] [DEBUG] Matched config for camera "USBFrameProvider - opencv_H264_USB_Camera (1)" and loaded 0 pipelines
[2022-10-05 22:04:53] [Camera - VisionSourceManager] [DEBUG] Matched config for camera "USBFrameProvider - opencv_vi-output,_ov5693_2-0036" and loaded 0 pipelines
[2022-10-05 22:04:53] [Camera - VisionSourceManager] [INFO] Adding 3 configs to VMM.
[2022-10-05 22:04:53] [General - NetworkTablesManager] [ERROR] NT Connection has failed! Will retry in background.
[2022-10-05 22:04:53] [WebServer - DataChangeService] [DEBUG] Added subscriber - Sources: DCS_WEBSOCKET, DCS_HTTP, DCS_NETWORKTABLES, DCS_VISIONMODULE, DCS_OTHER, Destinations: DCD_ACTIVEMODULE, DCD_ACTIVEPIPELINESETTINGS, DCD_GENSETTINGS, DCD_UI, DCD_OTHER
[2022-10-05 22:04:53] [VisionModule - VisionModule - H264_USB_Camera] [INFO] Setting pipeline to 0
[2022-10-05 22:04:53] [Camera - USBCameraSource - H264_USB_Camera] [DEBUG] Setting camera exposure to 50
[2022-10-05 22:04:53] [WebServer - DataChangeService] [DEBUG] Added subscriber - Sources: DCS_WEBSOCKET, DCS_HTTP, DCS_NETWORKTABLES, DCS_VISIONMODULE, DCS_OTHER, Destinations: DCD_ACTIVEMODULE, DCD_ACTIVEPIPELINESETTINGS, DCD_GENSETTINGS, DCD_UI, DCD_OTHER
[2022-10-05 22:04:53] [VisionModule - VisionModule - H264_USB_Camera (1)] [INFO] Setting pipeline to 0
[2022-10-05 22:04:53] [Camera - USBCameraSource - H264_USB_Camera (1)] [ERROR] Got a null video mode! Doing nothing...
[2022-10-05 22:04:53] [Camera - USBCameraSource - H264_USB_Camera (1)] [DEBUG] Setting camera exposure to 50
[2022-10-05 22:04:53] [WebServer - DataChangeService] [DEBUG] Added subscriber - Sources: DCS_WEBSOCKET, DCS_HTTP, DCS_NETWORKTABLES, DCS_VISIONMODULE, DCS_OTHER, Destinations: DCD_ACTIVEMODULE, DCD_ACTIVEPIPELINESETTINGS, DCD_GENSETTINGS, DCD_UI, DCD_OTHER
[2022-10-05 22:04:53] [VisionModule - VisionModule - vi-output,_ov5693_2-0036] [INFO] Setting pipeline to 0
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] Got a null video mode! Doing nothing...
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] Failed to set camera brightness!
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] VideoException [edu.wpi.first.cscore.VideoException: property write failed]
        at edu.wpi.first.cscore.CameraServerJNI.setCameraBrightness(Native Method)
        at edu.wpi.first.cscore.VideoCamera.setBrightness(VideoCamera.java:27)
        at org.photonvision.vision.camera.USBCameraSource$USBCameraSettables.setBrightness(USBCameraSource.java:135)
        at org.photonvision.vision.processes.VisionModule.setPipeline(VisionModule.java:394)
        at org.photonvision.vision.processes.VisionModule.<init>(VisionModule.java:149)
        at org.photonvision.vision.processes.VisionModuleManager.addSources(VisionModuleManager.java:59)
        at org.photonvision.vision.processes.VisionSourceManager.tryMatchUSBCams(VisionSourceManager.java:83)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [DEBUG] Setting camera exposure to 50
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] Failed to set camera exposure!
[2022-10-05 22:04:53] [Camera - USBCameraSource - vi-output,_ov5693_2-0036] [ERROR] VideoException [edu.wpi.first.cscore.VideoException: property write failed]
        at edu.wpi.first.cscore.CameraServerJNI.setCameraExposureManual(Native Method)
        at edu.wpi.first.cscore.VideoCamera.setExposureManual(VideoCamera.java:74)
        at org.photonvision.vision.camera.USBCameraSource$USBCameraSettables.setExposure(USBCameraSource.java:124)
        at org.photonvision.vision.processes.VisionModule.setPipeline(VisionModule.java:395)
        at org.photonvision.vision.processes.VisionModule.<init>(VisionModule.java:149)
        at org.photonvision.vision.processes.VisionModuleManager.addSources(VisionModuleManager.java:59)
        at org.photonvision.vision.processes.VisionSourceManager.tryMatchUSBCams(VisionSourceManager.java:83)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

[2022-10-05 22:04:53] [WebServer - SocketHandler] [INFO] New websocket connection from /192.168.1.215:52707
[2022-10-05 22:04:54] [General - HardwareManager] [INFO] Shutting down LEDs...

[mcm001]

Does the H264_USB_Camera work? And do you get an image if you try using v4l to say capture a frame from /dev/video0 or video1 or video2? Something like this:

ffmpeg -f video4linux2 -s 640x480 -i /dev/video0 -ss 0:0:2 -frames 1 out.jpg

[agasser]

Do you have any other [cameras] you can try?

I tried a Logitech C525 webcam and it does work on the Jetson TX2. The onboard camera still does not work, but that's fine if it is not compatible. This does appear to be an issue with the USB ELP camera.

I also tried using the ELP camera with a Raspberry Pi 4 (by running the PhotonVision JAR, not the full image), and it does not work there either. On the Raspberry Pi the camera also shows up as two cameras, but neither of them will display a stream in the PhotonVision web UI.

As I noted earlier though, I am able to use this camera from Python with cscore on the Jetson.

Does the H264_USB_Camera work? And do you get an image if you try using v4l to say capture a frame from /dev/video0 or video1 or video2?

I am able to capture a frame from video0 and video1 (the ELP camera.)
I am not able to capture from video2 (the onboard csi camera,) but I think it is expected and fine that this camera does not work with v4l or cscore.

[mdurrani808]

@agasser https://www.chiefdelphi.com/t/issues-with-viewing-a-usb-camera/343005/1
Please read above w/rt the ELP camera error.

[mdurrani808]

See note about logitech cameras here: https://docs.photonvision.org/en/latest/docs/hardware/supportedhardware.html#supported-cameras

[agasser]

@agasser https://www.chiefdelphi.com/t/issues-with-viewing-a-usb-camera/343005/1
Please read above w/rt the ELP camera error.

This looks like guidance for correcting the error in the PhotonVision code, not something I can do as a user, unless I'm missing something. The ELP camera does work on the Jetson with cscore.

This python cscore code works with the ELP camera. It is setting the mode and resolution/fps like Peter suggested.

frontCam = cscore.UsbCamera("DriverCam0", "/dev/v4l/by-path/platform-70090000.xusb-usb-0:3.2:1.0-video-index0")
frontCam.setPixelFormat(cscore.VideoMode.PixelFormat.kMJPEG)
frontCam.setResolution(320, 180)
frontCam.setFPS(20)

As a user PhotonVision user, is there something I can do?

[agasser]

See note about logitech cameras here: https://docs.photonvision.org/en/latest/docs/hardware/supportedhardware.html#supported-cameras

Yes, that is a problem. In conflict with the link you shared, my experience is that the Logitech C525 does work, the ELP camera does not.

[mdurrani808]

See note about logitech cameras here: https://docs.photonvision.org/en/latest/docs/hardware/supportedhardware.html#supported-cameras

Yes, that is a problem. In conflict with the link you shared, my experience is that the Logitech C525 does work, the ELP camera does not.

Ahh, I was misreading what was above. Sorry about that.

[mdurrani808]

Still doing some digging into this. Have a link to the ELP camera?

[mcm001]

If the MJpeg format is a known fix, the best solution may be adding a camera quirk in photon code

[agasser]

Still doing some digging into this. Have a link to the ELP camera?

Here it is https://a.co/bgCAFAK

[mdurrani808]

Are you able to print out all the current video modes using your Python code? Wondering if all of them just register as kUnknown and the list ends up null.
https://robotpy.readthedocs.io/projects/cscore/en/stable/_modules/cscore/cameraserver.html

[agasser]

I added this code to the USBCameraSettables constructor and this is what came out.

            System.out.println(
                    "--------CAMERA: "
                            + configuration.baseName
                            + " "
                            + configuration.path
                            + " modes:"
                            + videoModes.size());
            for (VideoMode vm : videoModes.values()) {
                System.out.println("h: " + vm.height);
                System.out.println("w: " + vm.width);
                System.out.println("fps:" + vm.fps);
                System.out.println("fmt: " + vm.pixelFormat.name());
                System.out.println("--------");
            }

It sees the USB camera as two cameras, /dev/video0 and /dev/video1. /dev/video2 is the onboard csi camera. /dev/video0 does have 70 modes.

In the PhotonVision UI I see two cameras "H264_USB_Camera" and "H264_USB_Camera (1)". Both cameras seem to list all of the modes. Neither camera will create a stream in the UI.