Play video with Jetson-inference #104

curiouser001 · 2017-07-12T08:45:26Z

Hi.
How can we use the video file instead of the image we took from the camera to implement DetectNet?

dusty-nv · 2017-07-12T12:09:13Z

Hi, we would need to replace gstCamera class with similar gstreamer class that read video file instead of camera... On Jul 12, 2017 4:45 AM, curiouser001 <notifications@github.com> wrote: Hi. How can we use the video file instead of the image we took from the camera to implement DetectNet? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#104>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AOpDKwDQqwUeXy7CDy3Ff8WU4XtsbFNRks5sNIengaJpZM4OVT3V>.

…

----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -----------------------------------------------------------------------------------

curiouser001 · 2017-07-12T18:33:43Z

Thank you, I will try.

bmaione · 2017-07-14T12:51:38Z

I was able to do this by passing in an additional command line argument with a path to a video file and then using a filesrc instead of the nvcamera src. You'll have to modify the create, init and buildlauhchstr functions in the gstCamera class to accept a string in addition to the other parameters. Bit of a hack but it works.

dkopljar · 2017-07-19T19:37:00Z

@bmaione Would you be so kind and send me that modified code?

curiouser001 · 2017-08-02T06:30:39Z

Thank you for your answer. If you submit the modified code, it will be appreciated.

dawnhowe-sync · 2017-09-13T18:23:16Z

@bmaione - can you share your launch string? I have been pulling my hair out on this one. I am trying to use a raw video file (more details on the file at the bottom)

This is my gstreamer pipeline (modification I put gstCamera::buildLaunchStr(). Unfortunately, this pipeline finds frames, but it drops a whole lot of them:
filesrc location=myfilename.yuv blocksize=6220800 ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12 ! appsink name=mysink

I was guessing at blocksize = 3 * height * width. Without the blocksize, the screen is green. Other googling led me to the videoparse gstreamer plugin, but I can't seem to install it. It should be a "bad" plugin, and I installed on the TX2 (Jetpack 3.1) them with:
sudo apt-get install libgstreamer-plugins-bad1.0-dev
but the plugin isn't found.

I took some video with my video camera. I converted it to a raw video file with this command:
ffmpeg -i 00090.MTS -vcodec rawvideo -pix_fmt nv12 myfilename.yuv

It plays fine with:
ffplay -f rawvideo -pix_fmt nv12 -s 1920x1080 myfilename.yuv

Any help would really be appreciated!

dawnhowe-sync · 2017-09-14T14:10:01Z

I finally found a solution using raw video in a file. Gstreamer "bad" plugins need to be installed to use videoparse. The pipeline that worked for me is:

filesrc location=myfilename.yuv ! videoparse width=1920 height=1080 framerate=30/1 format=2 ! appsink name=mysink

NOTE: to install all the gstreamer plugins, use this command:

sudo apt-get install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav
gstreamer1.0-doc gstreamer1.0-tools

curiouser001 · 2017-09-25T17:08:03Z

@bmaione Hi,
How did you do it. I have worked hard, but I have failed.

curiouser001 · 2017-09-26T08:40:55Z

Hi @ @dawnhowe-sync
I did what you said. But I am getting the following error.

gst-camera:  successfully initialized video device
    width:  640
   height:  480
    depth:  12 (bpp)
default X screen 0:   1920 x 1080
[OpenGL]  glDisplay display window initialized
[OpenGL]   creating 640x480 texture
[gstreamer] gstreamer transitioning pipeline to GST_STATE_PLAYING
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> videoparse0
[gstreamer] gstreamer changed state from NULL to READY ==> filesrc0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer stream status CREATE ==> sink
[gstreamer] gstreamer changed state from READY to PAUSED ==> videoparse0
[gstreamer] gstreamer changed state from READY to PAUSED ==> filesrc0
[gstreamer] gstreamer decoder onPreroll
[gstreamer] gstreamer stream status ENTER ==> sink
[gstreamer] gstreamer msg stream-start ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer msg async-done ==> pipeline0
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoparse0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> filesrc0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0

gst-camera:  camera open for streaming
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208200000 GPU 0x10208200000
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208254600 GPU 0x10208254600
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102082a8c00 GPU 0x102082a8c00
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102082fd200 GPU 0x102082fd200
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208351800 GPU 0x10208351800
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102083a5e00 GPU 0x102083a5e00
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208400000 GPU 0x10208400000
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208454600 GPU 0x10208454600
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102084a8c00 GPU 0x102084a8c00
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102084fd200 GPU 0x102084fd200
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208551800 GPU 0x10208551800
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102085a5e00 GPU 0x102085a5e00
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208600000 GPU 0x10208600000
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x10208654600 GPU 0x10208654600
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102086a8c00 GPU 0x102086a8c00
[cuda]  cudaAllocMapped 345600 bytes, CPU 0x102086fd200 GPU 0x102086fd200
[cuda]   gstreamer camera -- allocated 16 ringbuffers, 345600 bytes each
[cuda]   gstreamer camera -- allocated 16 RGBA ringbuffers

[cuda]   registered 4915200 byte openGL texture for interop access (640x480)
[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

[gstreamer] gstreamer decoder onEOS
[cuda]   cudaGetLastError()
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/cudaRGB.cu:44
[cuda]   cudaRGBToRGBAf((uchar3*)input, (float4*)mRGBA[mLatestRGBA], mWidth, mHeight)
[cuda]      invalid argument (error 11) (hex 0x0B)
[cuda]      /home/monster/Desktop/gstCamera/gstCamera.cpp:90
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

gst-camera:  failed to capture frame
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

gst-camera:  failed to capture frame
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

gst-camera:  failed to capture frame
gst-camera:  failed to convert from NV12 to RGBA
[cuda]   cudaNormalizeRGBA((float4*)imgRGBA, make_float2(0.0f, 255.0f), (float4*)imgRGBA, make_float2(0.0f, 1.0f), camera->GetWidth(), camera->GetHeight())
[cuda]      invalid device pointer (error 17) (hex 0x11)
[cuda]      /home/monster/Desktop/gstCamera/camera/gst-camera.cpp:120

Error continue...

dawnhowe-sync · 2017-09-26T13:07:12Z

@curiouser001 - there should be 2 sets of mallocs for the video frames. One for the incoming NV12 frames (in gstCamera::checkBuffer) and one set for the RGBA frames (in gstCamera::ConvertRGBA). I don't see a malloc in the log for the incoming frames. It should be 640 x 480 x 1.5 = 460800. (The 1.5 is because it has a depth of 12 bits, which is 1.5 bytes). The RGBA frames need 16 bytes per frame. It should be 640 x 480 x 16 = 4915200 because you need 4 floats per pixel). Your log says that you created 345600 bytes for the incoming frames. It is impossible for me to say what the problem is because you have modified the code so much. I can't figure out how you could possibly get 345600.

DennisFaucher · 2019-11-01T21:08:23Z

Please attach a copy of a modified gstCamera.cpp that works. I have been trying different options all day with no luck. Thank you.

dany-mns · 2020-03-16T11:36:17Z

Full code gstCamera.cpp
https://pastebin.com/3cAykMYW

You need to change location with your filename video and paste code after:
else if( src == GST_SOURCE_NVARGUS )
ss << "filesrc location=filename.mp4 ! qtdemux name=demux demux.video_0 ! h264parse ! omxh264dec ! "; ss << "nvvidconv ! video/x-raw,format=NV12 ! appsink name=mysink";

dusty-nv closed this as completed Apr 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Play video with Jetson-inference #104

Play video with Jetson-inference #104

curiouser001 commented Jul 12, 2017

dusty-nv commented Jul 12, 2017 via email

curiouser001 commented Jul 12, 2017

bmaione commented Jul 14, 2017

dkopljar commented Jul 19, 2017 •

edited

Loading

curiouser001 commented Aug 2, 2017

dawnhowe-sync commented Sep 13, 2017

dawnhowe-sync commented Sep 14, 2017 •

edited

Loading

curiouser001 commented Sep 25, 2017

curiouser001 commented Sep 26, 2017

dawnhowe-sync commented Sep 26, 2017

DennisFaucher commented Nov 1, 2019

dany-mns commented Mar 16, 2020 •

edited

Loading

Play video with Jetson-inference #104

Play video with Jetson-inference #104

Comments

curiouser001 commented Jul 12, 2017

dusty-nv commented Jul 12, 2017 via email

curiouser001 commented Jul 12, 2017

bmaione commented Jul 14, 2017

dkopljar commented Jul 19, 2017 • edited Loading

curiouser001 commented Aug 2, 2017

dawnhowe-sync commented Sep 13, 2017

dawnhowe-sync commented Sep 14, 2017 • edited Loading

curiouser001 commented Sep 25, 2017

curiouser001 commented Sep 26, 2017

dawnhowe-sync commented Sep 26, 2017

DennisFaucher commented Nov 1, 2019

dany-mns commented Mar 16, 2020 • edited Loading

dkopljar commented Jul 19, 2017 •

edited

Loading

dawnhowe-sync commented Sep 14, 2017 •

edited

Loading

dany-mns commented Mar 16, 2020 •

edited

Loading