avcenc produces run-to-run bit compare mismatches.

[tmjohns1]

Sporadically seeing sha1sum mismatches from run-to-run of avcenc (see below). Reproduced with Big Buck Bunny clip, as well as 176x144-25_1882_seq4.yuv. Pengfei has reproduced this issue.

Typical invocation:
avcenc 1920 1080 big_buck_bunny_1080p24_frames_11858_12100.yuv bbb_1080p24-cqp10-1.264 --qp=10 --mode=2

46668c54572db0e02024ed8f1663f892d4cbe966 bbb_1080p24-cqp10-1.264
0fe29c90b887403e52c66fc7d834b3802c82182e bbb_1080p24-cqp10-2.264
ef6b2e0ca63d1ca6f4e7d309671658e0b48d4c03 bbb_1080p24-cqp10-3.264
4758d7207c7365a579ca524ba60332b8e05d5386 bbb_1080p24-cqp10-4.264
cd83106cd2253e7897358ef2c495b9a6f80a57a5 bbb_1080p24-cqp10-5.264

ed064d19d758d23c64ac30bd6a20e4f541658adb bbb_1080p24-cqp20-1.264
ed064d19d758d23c64ac30bd6a20e4f541658adb bbb_1080p24-cqp20-2.264
ed064d19d758d23c64ac30bd6a20e4f541658adb bbb_1080p24-cqp20-3.264
ed064d19d758d23c64ac30bd6a20e4f541658adb bbb_1080p24-cqp20-4.264
ed064d19d758d23c64ac30bd6a20e4f541658adb bbb_1080p24-cqp20-5.264

c797c1b0bceee677ecdb6be740c7c6984f56414d bbb_1080p24-cqp25-1.264
c797c1b0bceee677ecdb6be740c7c6984f56414d bbb_1080p24-cqp25-2.264
c797c1b0bceee677ecdb6be740c7c6984f56414d bbb_1080p24-cqp25-3.264
c797c1b0bceee677ecdb6be740c7c6984f56414d bbb_1080p24-cqp25-4.264
c797c1b0bceee677ecdb6be740c7c6984f56414d bbb_1080p24-cqp25-5.264

d65807d292bbf4d1e29d8b3a828183515e276099 bbb_1080p24-cqp30-1.264
d65807d292bbf4d1e29d8b3a828183515e276099 bbb_1080p24-cqp30-2.264
663f722b754cb471cbb4d797bc068fe487d8fa48 bbb_1080p24-cqp30-3.264
d65807d292bbf4d1e29d8b3a828183515e276099 bbb_1080p24-cqp30-4.264
d65807d292bbf4d1e29d8b3a828183515e276099 bbb_1080p24-cqp30-5.264

3bb1f52ce294901d8fecb5133c624e914cf79c81 bbb_1080p24-cqp40-1.264
3bb1f52ce294901d8fecb5133c624e914cf79c81 bbb_1080p24-cqp40-2.264
3bb1f52ce294901d8fecb5133c624e914cf79c81 bbb_1080p24-cqp40-3.264
3bb1f52ce294901d8fecb5133c624e914cf79c81 bbb_1080p24-cqp40-4.264
3bb1f52ce294901d8fecb5133c624e914cf79c81 bbb_1080p24-cqp40-5.264

[chivakker]

have you tried libyami https://github.com/01org/libyami and https://github.com/01org/libyami-utils for the same use cases? Those libraries offer also md5sum comparisons on intel-vaapi-driver

[dvrogozh]

I can see this issue with:
yamitranscode -i input_1920x1080p.264 -o out.h264 -W 1920 -H 1080
-c AVC -ipperiod 2 -intraperiod 30
--rcmode CBR -b 5000

Looking into mismatches of 2 consequent runs, I think that issue is with bit2bit mismatches in B-frames. Looks like frames either match exactly or mismatch, mismatches do not spread on next frames. First few mismatch frames are B.

Hope this helps.

[chivakker]

thanks, we will take a look

[sreerenjb]

I think B-frame encoding is broken in current master, because my gstreamer-vaapi based test cases showing clear artifacts and macroblock decoding errors. And this is reproducible only for streams encoded with B frames.

[xhaihao]

Could you provide the platform information, such as graphics PCI id? It will help us to narrow down the issue.

[dvrogozh]

I see issue at SKL Skull Canyon NUC6i7KYK, CPU i7-6770HQ, GPU dev id is 0x193b.

[sreerenjb]

B frame encode while enabling High profile is the usage scenario which can reproduce the issue. Till now it was only reproducible in SKL. But with today's git master(c0ea9ac), the issue is reproducible in KBL also, because we started using the same codepath(with new media kernel) of SKL in KBL too.

[tmjohns1]

With patch to disable dct 8x8 ( Commit 6ec853) still seeing the r2r issue.

avcenc 176 144 176x144-25_1882_seq4.yuv 176x144-25_1882_seq4-1.264 --fb=500 --mode=2

7dd31a573bd61bca949b036b8391f515cbb9cee0 176x144-25_1882_seq4-1.264
7dd31a573bd61bca949b036b8391f515cbb9cee0 176x144-25_1882_seq4-2.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-3.264
5c1a2cd0d18af01e655bc7ab85fcaf1819d5ede1 176x144-25_1882_seq4-4.264
7e52f1e42272ab14f51a70105acd0265a409371c 176x144-25_1882_seq4-5.264
280f4c23aeec2bacd5de2db478a68ca28eca851b 176x144-25_1882_seq4-6.264
1a693eb3d49e9b50462db1ffb33bc76093df326b 176x144-25_1882_seq4-7.264
7dd31a573bd61bca949b036b8391f515cbb9cee0 176x144-25_1882_seq4-8.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-9.264
2bf40379504692a9314c29b883f2a45885730167 176x144-25_1882_seq4-10.264
1e2a1fca3ac16b9dbbeddc7209a9cc45fb6b0b2d 176x144-25_1882_seq4-20.264
ec299f0543e8783585354deea27feffc6dda6865 176x144-25_1882_seq4-21.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-22.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-23.264
afdd7d48a04f4aa94121babd8cbc260963004ed6 176x144-25_1882_seq4-24.264
679c51a1a3bdeacea709f84f5fbf2d7ce7d80a00 176x144-25_1882_seq4-25.264

[sreerenjb]

Commit 6ec853 was intended for fixing a different issue, not the r2r mismatch.

[QuPengfei]

@tmjohns1 . patch to disable dct 8x8 ( Commit 6ec853) fix the #97. that issue cause the B frame quality drop.

[QuPengfei]

it is related to direct_spatial_mv_pred_flag. it should be 1. it is the avcenc issue or middleware issue
intel/libva-utils#29

[QuPengfei]

@xhaihao would you please close it. the fix patch is in libva-utils

[tmjohns1]

Patch to enable direct_spatial_mv_pred_flag seems to fix r2r issues.
Typical test invocation:
avcenc 176 144 176x144-25_1882_seq4.yuv 176x144-25_1882_seq4-1.264 --fb=500 --mode=2

With the encoded streams showing:
transform_8x8_mode_flag = 0
direct_spatial_mv_pred_flag = 0

sha1sum output:
60995b82fb7d4694caa2b9b98847b8647c936363 176x144-25_1882_seq4-1.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-2.264
22fc7f350654164d8a4141017ee80bd96db3f9a8 176x144-25_1882_seq4-3.264
e0299cad0d36780b16f44c46aafd8439d35c2e6d 176x144-25_1882_seq4-4.264
885adb12206d23b8121aef1649e57839dd603bf0 176x144-25_1882_seq4-5.264
0fcf36609b6c16a3db21295460dfdd061cd70029 176x144-25_1882_seq4-6.264
c062960fdee66820a146eebe16253a286a06bf6e 176x144-25_1882_seq4-7.264
30368262965c69af552176047786de598c7afba5 176x144-25_1882_seq4-8.264
baed4eeeac2f2ab410045b152688a25a0f0a8e24 176x144-25_1882_seq4-9.264

After patching with Pengfei’s commit 215c04.

The encoded stream shows:
transform_8x8_mode_flag = 0
direct_spatial_mv_pred_flag = 1
Code changes (in encode/avcenc.c:avcenc_update_slice_parameter):
slice_param->direct_spatial_mv_pred_flag = 1;
slice_param->cabac_init_idc = 1;

sha1sum output:
d387b985d84eefabbdbf160e00d35d12bac17525 176x144-25_1882_seq4-1.264
d387b985d84eefabbdbf160e00d35d12bac17525 176x144-25_1882_seq4-2.264
d387b985d84eefabbdbf160e00d35d12bac17525 176x144-25_1882_seq4-3.264
d387b985d84eefabbdbf160e00d35d12bac17525 176x144-25_1882_seq4-4.264
d387b985d84eefabbdbf160e00d35d12bac17525 176x144-25_1882_seq4-5.264

[wangzj0601]

This issue can not be duplicated on skl platform with the latest libva-utils 1.8.3.pre1
env:
platform : SKL
linux kernel: 4.12.0-rc2
Libva :1.8.3.pre1
Libva- intel-driver :1.8.3.pre1
libva-utils : 1.8.3.pre1
Typical invocation:
avcenc 1280 720 720p5994_parkrun_ter.yuv 720p5994_parkrun_ter.yuv_7500_1.h264 --fb=7500 --mode 2
avcenc 1920 1080 1080p_blue_sky.yuv 1080p_blue_sky.yuv_21_1.h264 --qp=21 --mode 2

[FocusLuo]

@wangzj0601 this issue has been fixed, and it can't be reproduced now, can you help to close this issue?

[FocusLuo]

The issue had been fixed and verified on QA side, so closed it.