Codec errors fails the whole import process

[evekeen]

Steps:

1. Create a project backup
2. Import backup back to CVAT

Some of the video files might cause the encoder error.
Error during import leads to the whole import being unavailable, there is no information on which file caused it

2022-03-28 16:16:37,646 DEBG 'rqworker_default_1' stderr output:
WARNING:libav.libopenh264:[OpenH264] this = 0x0x5587c44d70, Warning:WelsEncoderEncodeExt()MinCr Checking,codec bitstream size is larger than Level limitation

2022-03-28 16:16:57,981 DEBG 'rqworker_default_1' stderr output:
WARNING:libav.libopenh264:[OpenH264] this = 0x0x5587c44d70, Warning:[MT] CWelsSliceEncodingTask ExecuteTask(), WriteSliceBs not successful: coding_idx 0, um_iSliceIdx 3

2022-03-28 16:16:57,983 DEBG 'rqworker_default_1' stderr output:
ERROR:libav.libopenh264:[OpenH264] this = 0x0x5587c44d70, Error:WelsEncoderEncodeExt(), multi-slice (mode 1) encoding error!

2022-03-28 16:16:57,985 DEBG 'rqworker_default_1' stderr output:
ERROR:libav.libopenh264:[OpenH264] this = 0x0x5587c44d70, Error:CWelsH264SVCEncoder::EncodeFrame(), kiEncoderReturn 3

2022-03-28 16:16:57,986 DEBG 'rqworker_default_1' stderr output:
ERROR:libav.libopenh264:EncodeFrame failed

2022-03-28 16:16:58,032 DEBG 'rqworker_default_1' stderr output:
DEBUG:rq.worker:Handling failed execution of job eevert@/api/projects/e96cd475-091b-4c40-a38f-d204cab2401b/import

2022-03-28 16:16:58,054 DEBG 'rqworker_default_1' stderr output:
ERROR:rq.worker:Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/engine/backup.py", line 652, in _import_project
    db_project = project_importer.import_project()
  File "/home/django/cvat/apps/engine/backup.py", line 644, in import_project
    self._import_tasks()
  File "/home/django/cvat/apps/engine/backup.py", line 634, in _import_tasks
    TaskImporter(
  File "/home/django/cvat/apps/engine/backup.py", line 527, in import_task
    self._import_task()
  File "/home/django/cvat/apps/engine/backup.py", line 510, in _import_task
    _create_thread(self._db_task.pk, data.copy(), True)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/engine/task.py", line 617, in _create_thread
    img_sizes = compressed_chunk_writer.save_as_chunk(chunk_data, compressed_chunk_path)
  File "/home/django/cvat/apps/engine/media_extractors.py", line 698, in save_as_chunk
    self._encode_images(images, output_container, output_v_stream)
  File "/home/django/cvat/apps/engine/media_extractors.py", line 659, in _encode_images
    for packet in stream.encode(frame):
  File "av/stream.pyx", line 152, in av.stream.Stream.encode
  File "av/codec/context.pyx", line 476, in av.codec.context.CodecContext.encode
  File "av/codec/context.pyx", line 391, in av.codec.context.CodecContext._send_frame_and_recv
  File "av/error.pyx", line 336, in av.error.err_check
av.error.UnknownError: [Errno 1313558101] Unknown error occurred; last error log: [libopenh264] EncodeFrame failed
Traceback (most recent call last):
  File "/opt/venv/lib/python3.8/site-packages/rq/worker.py", line 936, in perform_job
    rv = job.perform()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 684, in perform
    self._result = self._execute()
  File "/opt/venv/lib/python3.8/site-packages/rq/job.py", line 690, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/engine/backup.py", line 652, in _import_project
    db_project = project_importer.import_project()
  File "/home/django/cvat/apps/engine/backup.py", line 644, in import_project
    self._import_tasks()
  File "/home/django/cvat/apps/engine/backup.py", line 634, in _import_tasks
    TaskImporter(
  File "/home/django/cvat/apps/engine/backup.py", line 527, in import_task
    self._import_task()
  File "/home/django/cvat/apps/engine/backup.py", line 510, in _import_task
    _create_thread(self._db_task.pk, data.copy(), True)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/django/cvat/apps/engine/task.py", line 617, in _create_thread
    img_sizes = compressed_chunk_writer.save_as_chunk(chunk_data, compressed_chunk_path)
  File "/home/django/cvat/apps/engine/media_extractors.py", line 698, in save_as_chunk
    self._encode_images(images, output_container, output_v_stream)
  File "/home/django/cvat/apps/engine/media_extractors.py", line 659, in _encode_images
    for packet in stream.encode(frame):
  File "av/stream.pyx", line 152, in av.stream.Stream.encode
  File "av/codec/context.pyx", line 476, in av.codec.context.CodecContext.encode
  File "av/codec/context.pyx", line 391, in av.codec.context.CodecContext._send_frame_and_recv
  File "av/error.pyx", line 336, in av.error.err_check
av.error.UnknownError: [Errno 1313558101] Unknown error occurred; last error log: [libopenh264] EncodeFrame failed

[maxtju]

Any answer on this issue ? Was it solved ? Cause I have the same problem

[nmanovic]

@maxtju , need videos to reproduce the issue. Could you please provide? How did you create the backup?

[maxtju]

Hi @nmanovic
Thank you very much for fast answer I've found other thread of similar question.
The problem that you moved to https://github.com/cisco/openh264
Currently they are limited with maximal resolution of 3Kx2K pixels.
That's was my problem - I tried to make it with 4Kx3K. The error is not much informative, but scaling down video solves the problem.

Regards
Max

[nmanovic]

@maxtju , thanks for the information.

[apiszcz]

I'm seeing a similar issue with 2.3.0 (<2.3 worked fine) the input file is < 300 pixels x 300 pixels
status=Failed, message=av.error.UnknownError: [Errno 1313558101] Unknown error occurred; last error log: [libopenh264] Initialize failed