mkver / FFmpeg

Up until now, the bitmasks used to initially filter out when one needs
to take a closer look and search for startcodes were rather primitive:
If a block (of four or eight bytes, depending on the system) contained a
zero, it was treated as a target for closer inspection.
This can be improved: Using the same technique of bitmasking as before,
one can test whether an arbitrary continuous range of bits consists of
zeros alone (and one can or the results of tests for non-overlapping
ranges of bits). A simple improvement would consist in simply testing
for whether every even byte does not vanish. But even better masks are
possible, in particular for big endian systems.
If all of the bits called 'a' or all of the bits called 'b' in the
uint32_t aaaa aaaa aaaa aaax bbbb bbbb bbbb bbby vanish, then this is
a candidate for closer inspection. If not, then it follows that
the three least serious bytes can't be the position of the 0x01 byte of
a 0x00 0x00 0x01 startcode. And if the most significant byte is the
position of the 0x01 of a startcode and if the same test is performed on
each block of four bytes in sequence, then this would have all the b
bits (as well as the y bit) of the previous uint32_t read would vanish
and it would be considered a candidate for closer inspection. In other
words: All startcodes can be found with this test. The amount of
candidates is thereby reduced by a factor of about 256 (from about 4/2^8
to about 2/2^15). For eight bytes at a time, the patterns are simply
applied to the high and low 32 bits at the same time.
Unfortunately, not so much is possible for little-endian systems because
continuous blocks of bits in the input byte array won't be continuous
blocks in the integer read. But one can nevertheless even improve upon
the "check every even/odd byte" test using the following mask:
aaaa aaaa aaaa bbbb bbbb bbbb cccc cccc
If any of these three blocks of bits vanishes, the block is a candidate
for further inspection. In the byte array, this mask corresponds to the
following situation:
cccc cccc bbbb bbbb aaaa bbbb aaaa aaaa
If a startcode's 0x01 is at the second or third position, the c block
vanishes; if it is at the fourth position, the b block vanishes and if
it is at the first position, the a block of the previous four-byte block
vanishes. (This derivation just made use of the fact that there must be
two vanishing bytes in sequence; I couldn't find a way to utilize that
the third byte also has nearly no bit set.) So all startcodes can be
found with this mask. Doubling this mask yields a mask for eight bytes
at a time. I haven't found a better mask for eight bytes than this one.
Using this, one can detect reduce the amount of false positives by about
72% compared to the earlier check whether one of the bytes vanishes.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Until now the function ff_startcode_find_candidate_c did not really
search for startcodes (the startcode 0x00 0x00 0x01 (used in
MPEG-1/2/4, VC-1 and H.264/5) is the only startcode meant here). Instead
it searched for zero bytes and returned the earliest position of a zero
byte. This of course led to lots of false positives - millions per GB
of video.
This has been changed: The first position of the buffer that
may be part of a four-byte startcode is now returned. This excludes zero
bytes that are known not to belong to a startcode, but zero bytes at the
end of a buffer that might be part of a startcode whose second part is in
the next buffer are also returned. This is in accordance with the
expectations of the current callers of ff_startcode_find_candidate_c,
namely the H.264 parser and the VC-1 parser.
Getting rid of lots of function calls with its accompanying overhead of
course brings a little speed-up with it.

Up until now ff_startcode_find_candidate_c could overread; it relied on
zero-padding after the buffer in order to function correctly. This has
been changed: No overreads occur any more.
The ultimate goal behind all this is to create a high-performance
function for searching of startcodes that can be applied even in
scenarios where the buffer is not padded.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

ff_startcode_find_candidate_c already checks multiple bytes for zeros at
once if HAVE_FAST_UNALIGNED is true; up until now the other case checked
all bytes in sequence. This has been modified: A few bytes are checked
until alignment is reached from which point on several bytes can be
checked at once.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The reasons are cosmetics and preparation for future patches that will
have even more cases.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes crash when writting trailer without any previous packets.

…mpeg2_read_slice_header()

cbs_mpeg2_free_slice() calls av_buffer_unref() on extra_information_ref,
meaning allocating with av_malloc() was not the intention.

Signed-off-by: James Almer <jamrial@gmail.com>

Up until now, things that are merely unsupported by cbs_mpeg2 have been
declared to be invalid input. This has been changed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The frame_centre_horizontal/vertical_offset values contained in picture
display extensions are actually signed values (i.e. it is possible to
indicate that the display device should add black bars/pillars).

The files sony-ct3.bs and tcela-6.bits (which are both used in fate
tests for mpeg2_metadata) contain picture display extensions; the former
even contains a negative frame_centre_vertical_offset. Fortunately, the
old code did not damage the picture display extensions when one did a
cycle of reading and writing. For the same reason the fate tests needn't
be updated either.

Furthermore these fields now use the trace output for matrices.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

MPEG-2 contains several elements that mustn't be zero according to the
specifications: horizontal/vertical_size_value, aspect_ratio_information,
frame_rate_code, the quantiser matrices, the colour_description
elements, picture_coding_type, the f_code[r][s] values and
quantiser_scale_code. It is now checked that the invalid values don't
occur.

The colour_description elements are treated specially in this regard:
Given that there are files in the wild which use illegal values for the
colour_description elements (some of them created by mpeg2_metadata),
they will be corrected to the value meaning "unknown" (namely 2) during
reading. This has been done in such a way that trace_headers will
nevertheless report the original value, together with a message about
the fixup.

Furthermore, the trace_headers output of user_data has been beautified.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

At present, if the outer stream extradata is empty but first packet
has extradata as a side data element, then only the first segment's
muxer instance may be able to extract this side data and use it.
For all other segments, extradata in packet side data could be missing
and generated segments may be invalid or unplayable in some apps
e.g. for an ADTS AAC stream segmented to MP4, the adtstoasc BSF will
add extradata to the first packet. The MOV muxer for the first segment
will add this to codecpar for the inner stream and write
Decoder Specific Information within the esds box. For other segments,
their esds' will not have this decSpecificInfo and they can't be opened
in Quicktime player or by services like nginx-vod-module.

native and tensorflow is exclusive, so change the type from
flags to int.

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

sr didn't enable the slice threading, so remove the flag

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Document eld_v2 option for libfdk_aac encoder.

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Document checkheaders/alltools and consistency fixes

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

…k_motion_data()

Fixes: Timeout (54sec -> 188ms)
Fixes: 14585/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5649933052411904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: signed integer overflow: -1877966852 + -469491713 cannot be represented in type 'int'
Fixes: 14561/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5167608359288832

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (14sec -> 9sec)
Fixes: 13398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFV1_fuzzer-5664106709778432

decode_line() becomes 1% faster for fate/vsynth2-ffv1.avi
for another fate sample there is a 0.5% speedup
the effect should be bigger for files with "flat" colored areas
the new faster branch is used in 97-100% of the cases in fate samples
compared to the older more complex (which i tested)

vsynth3-ffv1-v3-bgr0.avi had the lowest percentual useage of about 97%

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: James Almer <jamrial@gmail.com>

Explain that the default Lanczos filter parameter is 3 and that it can be
changed by the param0 option.

Signed-off-by: Werner Robitza <werner.robitza@gmail.com>

…ferencing its data

To ensure the custom allocator is effectively used.

Signed-off-by: James Almer <jamrial@gmail.com>

Signed-off-by: James Almer <jamrial@gmail.com>

Slightly modified by Marton Balint to produce valid json as well.

Signed-off-by: Marton Balint <cus@passwd.hu>

In function ff_dct_unquantize_mpeg2_intra_mmi,
addr0 shoudn't be changed before storage operation.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>