[question] How to parse H. 264/AVC bitstream?

[Freed-Wu]

https://github.com/mirror/x264 will input a YUV and output a file such as out.264. we can use ffplay of ffmpeg to play it. How can we use fq to parse its information? I try fq -d avc_* . out.264 however cannot work.

TIA!

[wader]

HI, looks like it's anneb? this seems to work:

$ ffmpeg -y -f lavfi -i testsrc -pix_fmt yuv420p -t 100ms test.y4m
$ x264 -o test.264 test.y4m
$ fq -d avc_annexb  . test.264
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│.[0:12]: test.264 (avc_annexb)
0x000│00 00 00 01                                    │....            │  [0]: raw bits
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0x000│            67 64 00 0d ac d9 41 41 fb 01 10 00│    gd....AA....│  [1]{}: nalu (avc_nalu)
0x010│00 03 00 10 00 00 03 03 28 f1 42 99 60         │........(.B.`   │
0x010│                                       00 00 00│             ...│  [2]: raw bits
0x020│01                                             │.               │
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0x020│   68 eb e3 cb 22 c0                           │ h...".         │  [3]{}: nalu (avc_nalu)
0x020│                     00 00 01                  │       ...      │  [4]: raw bits
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0x020│                              06 05 ff ff aa dc│          ......│  [5]{}: nalu (avc_nalu)
0x030│45 e9 bd e6 d9 48 b7 96 2c d8 20 d9 23 ee ef 78│E....H..,. .#..x│
*    │until 0x2d7.7 (686)                            │                │
0x2d0│                        00 00 01               │        ...     │  [6]: raw bits
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0x2d0│                                 65 88 84 00 33│           e...3│  [7]{}: nalu (avc_nalu)
0x2e0│ff fe f6 ec be 05 36 14 c8 50 94 a0 40 07 bb 44│......6..P..@..D│
*    │until 0xcea.7 (2576)                           │                │
0xce0│                                 00 00 00 01   │           .... │  [8]: raw bits
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0xce0│                                             41│               A│  [9]{}: nalu (avc_nalu)
0xcf0│9a 22 6c 42 bf fe 38 85 de b9 a0 e0 2e c9 3f 6d│."lB..8.......?m│
*    │until 0xdca.7 (220)                            │                │
0xdc0│                                 00 00 00 01   │           .... │  [10]: raw bits
     │00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f│0123456789abcdef│
0xdc0│                                             01│               .│  [11]{}: nalu (avc_nalu)
0xdd0│9e 41 79 0a ff 0b ea bd aa b9 c5 10 39 bf 7d b8│.Ay.........9.}.│
*    │until 0xdfc.7 (end) (46)                       │                │

As it work now you can't use wildcard with the -d option, needs to be an exact name or group. Don't remember why i didn't add avc_annexb to the "probe"-group, maybe because miss-detected a bit too much, maybe could be improved.

[Freed-Wu]

LGTM!

maybe could be improved

Expect!

[wader]

👍

[Freed-Wu]

BTW, is it possible to display bits?

    |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|.[1].sps{}: (avc_sps)
0x00|42                                                                           |B                         |  profile_idc: "baseline_profile" (66)
0x00|   c0                                                                        | .                        |  constraint_set0_flag: true
0x00|   c0                                                                        | .                        |  constraint_set1_flag: true
0x00|   c0                                                                        | .                        |  constraint_set2_flag: false
0x00|   c0                                                                        | .                        |  constraint_set3_flag: false
0x00|   c0                                                                        | .                        |  constraint_set4_flag: false
0x00|   c0                                                                        | .                        |  constraint_set5_flag: false
0x00|   c0                                                                        | .                        |  reserved_zero_2bits: 0

We know 0xc0 is 0b11000000

    |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|.[1].sps{}: (avc_sps)
0x00|42                                                                           |B                         |  profile_idc: "baseline_profile" (66)
0x00|   c0(1)                                                                        | .                        |  constraint_set0_flag: true
0x00|   c0(1)                                                                        | .                        |  constraint_set1_flag: true
0x00|   c0(0)                                                                        | .                        |  constraint_set2_flag: false
0x00|   c0(0)                                                                        | .                        |  constraint_set3_flag: false
0x00|   c0(0)                                                                        | .                        |  constraint_set4_flag: false
0x00|   c0(0)                                                                        | .                        |  constraint_set5_flag: false
0x00|   c0(00)                                                                        | .                        |  reserved_zero_2bits: 0

The latter will be more clear, right?

Or other better solution? Just my 2c 😄

[wader]

Yeap i've experimented a bit with making the "dump" view a bit more configurable, ex what columns to show and bit/byte format. There is an example how base 2 could look like here #703 (comment) to do it would want to refactor the code as it's quite messy as it is and also come up with some good way how a user would configure it, ex display({columns: [{type: "offset", base: 16}, {type: "data", base: 2, width: ...}, ...]}) or some compact text foramat display({columns: "offset,hex,field"})

You can also do some half ugly hack like this:

$ fq -r -d avc_annexb '.[1].sps | to_entries[] | "\(.key): \(.value | tobits | explode | join(""))"' test.264'profile_idc: 01100100
constraint_set0_flag: 0
constraint_set1_flag: 0
constraint_set2_flag: 0
constraint_set3_flag: 0
constraint_set4_flag: 0
constraint_set5_flag: 0
reserved_zero_2bits: 00
level_idc: 00001101
seq_parameter_set_id: 1
chroma_format_idc: 010
bit_depth_luma: 1
bit_depth_chroma: 1
qpprime_y_zero_transform_bypass_flag: 0
seq_scaling_matrix_present_flag: 0
log2_max_frame_num: 1
pic_order_cnt_type: 1
log2_max_pic_order_cnt_lsb: 011
max_num_ref_frames: 00101
gaps_in_frame_num_value_allowed_flag: 0
pic_width_in_mbs: 000010100
pic_height_in_map_units: 0001111
frame_mbs_only_flag: 1
direct_8x8_inference_flag: 1
frame_cropping_flag: 0
vui_parameters_present_flag: 1
vui_parameters: 10000000100010000000000000000000000000000000100000000000000000000000000110010100011110001010000101001100101
rbsp_trailing_bits: 100000

[Freed-Wu]

Maybe we can only display bits in necessary situation. such as:

    |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|.[1].sps{}: (avc_sps)
0x00|42                                                                           |B                         |  profile_idc: "baseline_profile" (66)
0x00|   c0                                                                        | .                        |  constraint_set0_flag: true
0x00|   c0                                                                        | .                        |  constraint_set1_flag: true
0x00|   c0                                                                        | .                        |  constraint_set2_flag: false
0x00|   c0                                                                        | .                        |  constraint_set3_flag: false
0x00|   c0                                                                        | .                        |  constraint_set4_flag: false
0x00|   c0                                                                        | .                        |  constraint_set5_flag: false
0x00|   c0                                                                        | .                        |  reserved_zero_2bits: 0
0x00|      0d                                                                     |  .                       |  level_idc: "1.3" (13)
0x00|         da                                                                  |   .                      |  seq_parameter_set_id: 0
0x00|         da                                                                  |   .                      |  log2_max_frame_num: 4
0x00|         da                                                                  |   .                      |  pic_order_cnt_type: 2
0x00|         da                                                                  |   .                      |  max_num_ref_frames: 1
0x00|            05                                                               |    .                     |  gaps_in_frame_num_value_allowed_flag: false
0x00|            05 82                                                            |    ..                    |  pic_width_in_mbs: 22
0x00|               82 5a                                                         |     .Z                   |  pic_height_in_map_units: 18
0x00|                  5a                                                         |      Z                   |  frame_mbs_only_flag: true
0x00|                  5a                                                         |      Z                   |  direct_8x8_inference_flag: true
0x00|                  5a                                                         |      Z                   |  frame_cropping_flag: false
0x00|                  5a                                                         |      Z                   |  vui_parameters_present_flag: true
0x00|                  5a 10 00 00 00 10 00 00 03 28 f1 42 aa|                    |      Z........(.B.|      |  vui_parameters{}:
0x00|                                                      aa|                    |                  .|      |  rbsp_trailing_bits: raw bits
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|42                                                                           |B                         |.[1].sps.profile_idc: "baseline_profile" (66)
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set0_flag: true
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set1_flag: true
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set2_flag: false
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set3_flag: false
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set4_flag: false
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.constraint_set5_flag: false
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|   c0                                                                        | .                        |.[1].sps.reserved_zero_2bits: 0

We can only display bits for the below lines:

    |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|.[1].sps{}: (avc_sps)
0x00|42                                                                           |B                         |  profile_idc: "baseline_profile" (66)
0x00|   c0                                                                        | .                        |  constraint_set0_flag: true
0x00|   c0                                                                        | .                        |  constraint_set1_flag: true
0x00|   c0                                                                        | .                        |  constraint_set2_flag: false
0x00|   c0                                                                        | .                        |  constraint_set3_flag: false
0x00|   c0                                                                        | .                        |  constraint_set4_flag: false
0x00|   c0                                                                        | .                        |  constraint_set5_flag: false
0x00|   c0                                                                        | .                        |  reserved_zero_2bits: 0
0x00|      0d                                                                     |  .                       |  level_idc: "1.3" (13)
0x00|         da                                                                  |   .                      |  seq_parameter_set_id: 0
0x00|         da                                                                  |   .                      |  log2_max_frame_num: 4
0x00|         da                                                                  |   .                      |  pic_order_cnt_type: 2
0x00|         da                                                                  |   .                      |  max_num_ref_frames: 1
0x00|            05                                                               |    .                     |  gaps_in_frame_num_value_allowed_flag: false
0x00|            05 82                                                            |    ..                    |  pic_width_in_mbs: 22
0x00|               82 5a                                                         |     .Z                   |  pic_height_in_map_units: 18
0x00|                  5a                                                         |      Z                   |  frame_mbs_only_flag: true
0x00|                  5a                                                         |      Z                   |  direct_8x8_inference_flag: true
0x00|                  5a                                                         |      Z                   |  frame_cropping_flag: false
0x00|                  5a                                                         |      Z                   |  vui_parameters_present_flag: true
0x00|                  5a 10 00 00 00 10 00 00 03 28 f1 42 aa|                    |      Z........(.B.|      |  vui_parameters{}:
0x00|                                                      aa|                    |                  .|      |  rbsp_trailing_bits: raw bits
   |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19|0123456789abcdef0123456789|
0x0|42                                                                           |B                         |.[1].sps.profile_idc: "baseline_profile" (66)
      |01234567|
0x01.0|1       |.[1].sps.constraint_set0_flag: true
      |01234567|
0x01.1| 1      |.[1].sps.constraint_set1_flag: true
      |01234567|
0x01.2|  0     |.[1].sps.constraint_set2_flag: false
      |01234567|
0x01.3|   0    |.[1].sps.constraint_set3_flag: false
      |01234567|
0x01.4|    0   |.[1].sps.constraint_set4_flag: false
      |01234567|
0x01.5|     0  |.[1].sps.constraint_set5_flag: false
      |01234567|
0x01.6|      00|.[1].sps.reserved_zero_2bits: 00

[wader]

You mean if a user displays a single field? guess it could be done, but then i think the format decoder somehow needs to somehow add a hint to those fields how it should behave in different display modes 🤔