cuda::videoWriter and VideoWriter error

[opentld]

environment：

• OpenCV => 4.5.3 build with ffmpeg cuda nvcuvid
• Operating System / Platform => Windows 64 Bit
• Compiler => Visual Studio 2019

---

cuda::videoReader is running normally, but cuda::videoWriter has throw_no_cuda error. Try to use VideoWriter to store the image decoded by cuda::videoReader. The stored file is only 1kb. But the video files obtained by VideoCapture are stored correctly by VideoWriter. why?
The following is the source code：

---

#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/cudacodec.hpp>

using namespace std;
using namespace cv;

int main()
{
    string video_name = "test2.mp4";

    VideoWriter writer;
    int fourcc = VideoWriter::fourcc('X', '2', '6', '4'); 

    cout << "cuda device count: " << cuda::getCudaEnabledDeviceCount() << endl;
    cuda::GpuMat d_frame;
    Ptr<cudacodec::VideoReader> d_reader = cudacodec::createVideoReader(video_name);

    Mat mat;

    writer.open("out2.mp4", fourcc, 25, Size(d_reader->format().width, d_reader->format().height), true);

    if (!writer.isOpened())
    {
        cout << "error opening writer" << endl;
        return -1;
    }

    double time1 = static_cast<double>(getTickCount());
    while(true) 
    {
        if (!d_reader->nextFrame(d_frame))
            break;

        d_frame.download(mat);
        writer << mat;
    }

    double time2 = (static_cast<double>(getTickCount()) - time1) / getTickFrequency();
    cout << "time: " << time2;

    writer.release();
    return 0;
}

[opentld]

Two problems have been found that may cause errors in cv::VideoWriter:

1. After decoding by cudacodec::VideoReader, the size of the video has changed from (1920,1080) to (1920,1088)
2. The decoded image is in RGBA format, while VideoWriter needs BGR format.

   while(true) 
    {
        if (!d_reader->nextFrame(d_frame))
            break;

        cuda::cvtColor(d_frame, d_mat, cv::COLOR_RGBA2BGR);
        
        d_mat.download(mat);

        if (!mat.empty())
            writer.write(mat);
    }

However, there is still a problem, why cudacodec::VideoReader's decoding is normal, but cudacodec::VideoWriter throw_no_cuda error?

The error message:

OpenCV(4.5.3-pre) Error: The function/feature is not implemented (The called functionality is disabled for current build or platform) in throw_no_cuda, file E:\Tools\opencv-4.5.3\modules\core\include\opencv2/core/private.cuda.hpp, line 112

private.cuda.hpp line 112:
static inline CV_NORETURN void throw_no_cuda() { CV_Error(cv::Error::StsNotImplemented, "The called functionality is disabled for current build or platform"); }

[alalek]

/cc @cudawarped

[cudawarped]

1. After decoding by cudacodec::VideoReader, the size of the video has changed from (1920,1080) to (1920,1088)

Depending on the backend which is used for the video source cudacodec::VideoReader can output the coded size instead of the display size, see opencv_contib/#3001. Unfortunately the true display size may not be known until the first call to nextFrame() see this test for details. To store the video I would first crop the redundant lines from the bottom as

FormatInfo fmt = videoReader->format();
GpuMat frameCropped = frameDecoded(fmt.displayArea);

However, there is still a problem, why cudacodec::VideoReader's decoding is normal, but cudacodec::VideoWriter throw_no_cuda error?

cudacodec::VideoWriter hasn't worked for a long time. Currently it uses the the old redundant Nvidia decoding SDK. What is your use case for encoding the video that you have just read?

[opentld]

cudacodec::VideoWriter hasn't worked for a long time. Currently it uses the the old redundant Nvidia decoding SDK. What is your use case for encoding the video that you have just read?

I read the video in, decode it, and then transcode it for storage.
Cudacodec decodes fast, but cv::VideoWriter encodes and stores very slowly, so I want to use cudacodec::VideoWriter to complete this function. Unfortunately, I tried to use the previous version of nvcuvenc.dll, but it crashed in this code:
cudacodec/src/video_writer.cpp:

void VideoWriterImpl::createHWEncoder()
    {
        int err;

        // Create the NVIDIA HW resources for Encoding on NVIDIA hardware
        err = NVCreateHWEncoder(encoder_);
        CV_Assert(err == 0);

    }

Is there any way to encode with GPU?

[cudawarped]

Is there any way to encode with GPU?

Inside OpenCV at the moment I would say no. It may be possible to use cv::VideoWriter with hardware acceleration but I am not sure as this is a relatively new feature and you would still need to download the frames to the host. I also do not know what the performace of it is like.

Outside OpenCV you could probably just use most of the code from the transcoding example (AppTrans) from the Nvidia Video Codec SDK samples.

[opentld]

Inside OpenCV at the moment I would say no. It may be possible to use cv::VideoWriter with hardware acceleration but I am not sure as this is a relatively new feature and you would still need to download the frames to the host. I also do not know what the performace of it is like.

Outside OpenCV you could probably just use most of the code from the transcoding example (AppTrans) from the Nvidia Video Codec SDK samples.

Well, I will try to use the Nvidia Video Codec SDK to solve this problem, thanks a lot!

[opentld]

I tried the Nvidia Video Codec SDK, but the AppTrans in it also cannot support NVEnc, because CUDA no longer supports NVEnc. However, I found an open source algorithm that may be helpful for this. https://github.com/rigaya/NVEnc

If someone implements the Nvidia Video Codec SDK that supports NVEnc, can you tell me how to do it?

[cudawarped]

I am not sure what you are trying to do and what you mean by NVEnc and that CUDA no longer supports NVEnc?

The AppTrans application decodes and encodes video using an Nvidia GPU. On my system I can see it using both the video encode and decode hardware units for most of the processing.

What hardware and CUDA version are you using?

[opentld]

I am not sure what you are trying to do and what you mean by NVEnc and that CUDA no longer supports NVEnc?

The AppTrans application decodes and encodes video using an Nvidia GPU. On my system I can see it using both the video encode and decode hardware units for most of the processing.

What hardware and CUDA version are you using?

RTX2080, CUDA 10.2
"The called functionality is disabled for current build or platform"

[cudawarped]

"The called functionality is disabled for current build or platform"

That's an opencv error not an AppTrans error. What error do you get from AppTrans which indicates that GPU hardware decoding is not supported?

[khanhha]

the displayArea field no longer holds the correct video size in the version 4.7. In my case, the correct size of video frame is 1440x1080 but displayArea is 1440x1088. I have to use VideoReader to get the correct size first for cropping

[cudawarped]

the displayArea field no longer holds the correct video size in the version 4.7

Have you read a frame and checked that the format is valid?

FormatInfo fmt = videoReader->format();
fmt.valid

I am in the process of changing this so it will be unecessary to check in the future.

[asmorkalov]

@cudawarped

1. The first issue with frame height is fixed in contrib.
2. The BGR-RGBA issue is covered by ColorFormat properties in decoder and encoder.
3. CUDA 10.2 might be too old for RTX2080 and should be updated.

Are there open items in the ticket. Could it be closed?

[cudawarped]

1. The first issue with frame height is fixed in contrib.

Fixed in opencv/opencv_contrib#3470.

2. The BGR-RGBA issue is covered by ColorFormat properties in decoder and encoder.

Exactly it wasn't an issue the default format for cudacodec::VideoReader is BGRA, however it can now be changed (opencv/opencv_contrib#3198) so even if it was an issue its fixed.

3. CUDA 10.2 might be too old for RTX2080 and should be updated.

This was related to cudacodec::VideoWriter being disabled because it used the old redundant Nvidia decoding SDK. Fixed in opencv/opencv_contrib#3360.

Are there open items in the ticket. Could it be closed?

Not that I can see, lets close it so we don't have to read through it again!