Black feed previews when input feed is from IP Camera

[ing-norante]

Hello, I'm testing smart-social-distancing using a jetson nano and a D-Link DCS-5222LB IP Camera.
The frontend is running locally on my laptop, the Processor is running on the Jetson and the Dlink camera are all on the same subnet.

Laptop: 192.168.188.20
Dlink camera: 192.168.188.23
Jetson: 192.168.188.81

Here are my configurations:

config-frontend.ini

[App]
Host: 0.0.0.0
Port: 8000

[Processor]
Host: 192.168.188.81
Port: 8000

config-jetson.ini

[App]
VideoPath = rtsp://username:password@192.168.188.23/live1.sdp
Resolution = 640,360

Encoder: videoconvert ! video/x-raw,format=I420 ! x264enc speed-preset=ultrafast

[API]
Host = 0.0.0.0
Port = 8000

[CORE]
Host = 0.0.0.0
QueuePort = 8010
QueueAuthKey = shibalba

I'm running the Processor on the Jetson with this command:

docker build -f jetson-nano.Dockerfile -t "neuralet/smart-social-distancing:latest-jetson-nano" .
docker run -it --runtime nvidia --privileged -p 8000:8000 -v "$PWD/data":/repo/data neuralet/smart-social-distancing:latest-jetson-nano

and this is the output I get on the console:

ok video is going to be processed
[TensorRT] ERROR: Could not register plugin creator:  FlattenConcat_TRT in namespace:
INFO:libs.detectors.jetson.mobilenet_ssd_v2:allocated buffers
Device is:  Jetson
Detector is:  ssd_mobilenet_v2_pedestrian_softbio
image size:  [300, 300, 3]
INFO:libs.distancing:opened video rtsp://admin:racthenwyothtb@192.168.188.23/live2.sdp
error: XDG_RUNTIME_DIR not set in the environment.
0:00:02.002293373    68   0x55a7901730 ERROR                default gstvaapi.c:254:plugin_init: Cannot create a VA display
INFO:libs.distancing:processed frame 1 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 11 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 21 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 31 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 41 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 51 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 61 for rtsp://username:password@192.168.188.23/live1.sdp
INFO:libs.distancing:processed frame 71 for rtsp://username:password@192.168.188.23/live1.sdp

[...] and so on

I'm running the webapp locally on my laptop with this command:

docker build -f frontend.Dockerfile -t "neuralet/smart-social-distancing:latest-frontend" .
docker build -f web-gui.Dockerfile -t "neuralet/smart-social-distancing:latest-web-gui" .
docker run -it -p 8000:8000 --rm neuralet/smart-social-distancing:latest-web-gui 

This is the output I get from the console:

Successfully built 0047284f1577
Successfully tagged neuralet/smart-social-distancing:latest-web-gui
INFO:     Started server process [1]
INFO:uvicorn.error:Started server process [1]
INFO:     Waiting for application startup.
INFO:uvicorn.error:Waiting for application startup.
INFO:     Application startup complete.
INFO:uvicorn.error:Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:uvicorn.error:Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

When I browse the frontend locally with the latest Chrome on http://0.0.0.0:8000/panel/#/live I see the Camera Feed and the Bird's View box completely black. See the screenshot below:

The plot underneath the camera feeds is working because if I step back and forth in front of the camera seems like it's recognizing me.

If I open Chrome's inspector I can see that there are two errors in the console.

And if I try performing a GET to that URL (or any similar path) I get a 404.

and a "Not Found" response.

Has anyone of you experienced something similar when working with IP Cams? Any idea on how to fix this?

Thank you very much and big ups for the beautiful project!

Cheers

[mats-claassen]

Hi @eppemela I did get it to work with IP cams, but there could be several issues. Have you checked whether there is a playlist.m3u8 file in the folder you are trying to access? (data/processor/static/gstreamer/default)

[ing-norante]

Hello @mats-claassen! There's no playlist file in that directory but there's a video.ts file

ls -alh data/processor/static/gstreamer/default

total 796K
drwxr-xr-x 2 root root 4,0K set 14 15:30 .
drwxr-xr-x 4 root root 4,0K set 14 15:30 ..
-rw-r--r-- 1 root root 788K set 14 15:33 video_00000.ts

[mats-claassen]

I have seen this issue when it seems that no playlist is created. In my case it happened together with a lot of other messages which point to a bad network connection to the cameras. It was fixed a few days later. However I don't see those error messages for you. I am not completely sure what provokes this error but you could try restarting the container on the Jetson a few times and see whether it creates the playlist file, if you haven't already

[ing-norante]

Yeah, I tried several times to restart the Processor but I can see that the playlist.m3u8 file never gets created. I've even tried to chown the directory data/processor/static/gstreamer/default because I noticed that was owned by root but I get no playlist file at all.
Any other idea?

Thanks!

[renzodgc]

Hey, we just got around checking these old issues, sorry about the delay.

I believe we never were able to reproduce this issue, so perhaps with the new iterations in the codebase, it now works for your case.

As a side note, we deprecated the old frontend code and implemented a new one hosted in https://www.beta.lanthorn.ai/, the README has more information regarding this.

Feel free to try to reproduce it if you are still interested and report back If the issue remains. Elsewise I'll close this issue in a week :)

[ing-norante]

This can be closed because, with the new iterations in the codebase, it works fine now.
Thanks!