Creating JPEG file from .heic files

[agharib]

Thanks for reporting your issue. Please make sure these boxes are checked before submitting your issue - thank you!

• I looked for a similar issue and couldn't find any.
• I tried with the latest version of GPAC. Installers available at http://gpac.io/downloads/gpac-nightly-builds/
• I give enough information for contributors to reproduce my issue (meaningful title, github labels, platform and compiler, command-line ...). I can share files anonymously with this dropbox: https://www.mediafire.com/filedrop/filedrop_hosted.php?drop=eec9e058a9486fe4e99c33021481d9e1826ca9dbc242a6cfaab0fe95da5e5d95

Detailed guidelines: http://gpac.io/2013/07/16/how-to-file-a-bug-properly/

Hi there,

I see on the GPAC post that there is an easy way to create HEIF files using mp4box... Apologies if this is clearly stated somewhere, but is there a way to get jpeg's from .heic files using mp4box? Any documentation/references would be greatly appreciated.

Thanks!

-Ahmed

[knopa]

Hey, have same issue

No chance to get how to convert from heif to jpg?

[rbouqueau]

You guys miss an important point: MP4Box can only manipulate the container. JPEG and HEVC are different codecs so you'll need to re-encode.

The easiest way to do the conversion is to use the Nokia tool to decode your HEIF content to RAW -formats to be precised). Then use a JPEG encoder to perform the RAW -> JPEG conversion.

[ericcj]

@rbouqueau It seemed like the below invocation of MP4Box is supposed to extract the raw tiles (in HEVC format) from the HEIC container, but as you can see they don't seem to be valid HEVC when I get them out even though they do work properly for the same test file when extracting them with the nokia library. Am I misunderstanding the MP4Box options or what exactly gpac supports? Thanks

$ ~/gpac/bin/gcc/MP4Box -info ~/test_001.heic
ICC colour profile not supported
ICC colour profile not supported
Root Meta type: "pict" - 52 resource item(s)
Primary Item - ID 49
Item #1 - ID 1 - Name:
Item #2 - ID 2 - Name:
...
Item #51 - ID 51 - Name:
File has no movie (moov) - static data container

$ ~/gpac/bin/gcc/MP4Box -dump-item 1:path=item1.hevc ~/test_001.heic

$ ffmpeg -i item1.hevc -frames:v 1 -vsync vfr -q:v 1 -y -an item1.bmp
item1.hevc: Invalid data found when processing input

[rbouqueau]

@ericcj I don't know if it is supposed to be HEVC compliant but I could reproduce your issue:
CC @cconcolato

$ wget http://nokiatech.github.io/heif/content/images/crowd_1440x960.heic
$ MP4Box -info crowd_1440x960.heic
Root Meta type: "pict" - 3 resource item(s)
Primary Item - ID 20002
Item #1 - ID 20002 - Name: HEVC Image
Item #2 - ID 20003 - Name: HEVC Image
File has no movie (moov) - static data container

$ MP4Box -dump-item 1:path=item1.hevc crowd_1440x960.heic

        Error: Bad Parameter

$ MP4Box -dump-item 20002:path=item1.hevc crowd_1440x960.heic

$ MP4Box -add item1.hevc item1.mp4
Cannot find HEVC start code
Error importing item1.hevc: BitStream Not Compliant

[cconcolato]

The extraction code is very generic. It takes the bytes of the item and outputs them. This works well when the resource is stored as is in the item. For HEIC, it's a bit tricky. The payload of the item is only the VCL NALU (IIRC), the SPS/PPS should be in an item property (hvcC). It's highly likely that the extraction does not extract the hvcC content because the notion of item property did not exist when we first wrote the item extraction code. Also the start code vs. length fields and maybe the emulation prevention bytes processing have to be added.

[avibrazil]

Apparently, MP4Box does not support HEIF/HEIC as generated by iOS 11 devices. Here are samples I took with my iPhone 7+ myself, useful for tests and debug.

https://avi.alkalay.net/clipboard/HEIF-iOS-11.zip

Zip file contains videos, regular photos, photos with depth and photo bursts.

I'm particularly interested in MP4Box support for iOS 11 HEIF tags and metadata.

This is what I get with GPAC 0.6.1 and 0.7.1 on Fedora Linux:

$ MP4Box -info IMG_4453.HEIC
[iso file] Read Box "infe" failed (Invalid IsoMedia File)
[iso file] Read Box "iinf" failed (Invalid IsoMedia File)
[iso file] Read Box "meta" failed (Invalid IsoMedia File)
Error opening file IMG_4453.HEIC: Invalid IsoMedia File

[jeanlf]

This is now fixed on git, thanks for the report

[deweydb]

@jeanlf is it possible that this commit while fixing it for most cases missed an edge case? I've got a file (IMG_5915.HEIC attached in the zip) that when i do:
MP4Box -info IMG_5915.HEIC
I get:

prof colour profile not supported 
prof colour profile not supported 
Root Meta type: "pict" - 52 resource item(s)
Primary Item - ID 49
Item #1 - ID 1 - Name: 
Item #2 - ID 2 - Name: 
Item #3 - ID 3 - Name: 
Item #4 - ID 4 - Name: 
Item #5 - ID 5 - Name: 
Item #6 - ID 6 - Name: 
Item #7 - ID 7 - Name: 
Item #8 - ID 8 - Name: 
Item #9 - ID 9 - Name: 
Item #10 - ID 10 - Name: 
Item #11 - ID 11 - Name: 
Item #12 - ID 12 - Name: 
Item #13 - ID 13 - Name: 
Item #14 - ID 14 - Name: 
Item #15 - ID 15 - Name: 
Item #16 - ID 16 - Name: 
Item #17 - ID 17 - Name: 
Item #18 - ID 18 - Name: 
Item #19 - ID 19 - Name: 
Item #20 - ID 20 - Name: 
Item #21 - ID 21 - Name: 
Item #22 - ID 22 - Name: 
Item #23 - ID 23 - Name: 
Item #24 - ID 24 - Name: 
Item #25 - ID 25 - Name: 
Item #26 - ID 26 - Name: 
Item #27 - ID 27 - Name: 
Item #28 - ID 28 - Name: 
Item #29 - ID 29 - Name: 
Item #30 - ID 30 - Name: 
Item #31 - ID 31 - Name: 
Item #32 - ID 32 - Name: 
Item #33 - ID 33 - Name: 
Item #34 - ID 34 - Name: 
Item #35 - ID 35 - Name: 
Item #36 - ID 36 - Name: 
Item #37 - ID 37 - Name: 
Item #38 - ID 38 - Name: 
Item #39 - ID 39 - Name: 
Item #40 - ID 40 - Name: 
Item #41 - ID 41 - Name: 
Item #42 - ID 42 - Name: 
Item #43 - ID 43 - Name: 
Item #44 - ID 44 - Name: 
Item #45 - ID 45 - Name: 
Item #46 - ID 46 - Name: 
Item #47 - ID 47 - Name: 
Item #48 - ID 48 - Name: 
Item #49 - ID 49 - Name: 
Item #50 - ID 50 - Name: 
Item #51 - ID 51 - Name: 
File has no movie (moov) - static data container

Then if I extract all of these:

for i in `seq 51`; do MP4Box -dump-item $i:path=item$i.hevc IMG_5915.HEIC; done

I get 51 files, item1.hevc through item51.hevc, for every extraction i get this warning:
prof colour profile not supported

Then if i continue, and convert these to PNG with ffmpeg nearly all of them convert successfully, but 2 fail:
ffmpeg -i item49.hevc -frames:v 1 -vsync vfr -q:v 1 -an image49.png�
item49.hevc: Invalid data found when processing input
ffmpeg -i item51.hevc -frames:v 1 -vsync vfr -q:v 1 -an image51.png
item51.hevc: Invalid data found when processing input

Although, it seems that image 48 is the bottom right corner of the image (because of the "letterboxing" visible). And that image 50 is some sort of thumbnail stored in an extra tile. So perhaps image 49 and image 51 are some other superfluous information? metadata stored in a "tile"?
HEIC Conversion.zip

[cconcolato]

item 49 and item 51 are not HEVC. 49 is a 'grid' item. 51 is EXIF metadata.

[deweydb]

@cconcolato Awesome! thanks for the help! So i guess maybe there would be some way of grabbing the EXIF metadata from 51 and applying it to the final image once the tiles are recombined?
Also, is there any way to tell from the metadata how many items there are per row in the image tile set? for example the one i posted above has 8 tiles per row.

[deweydb]

Ok. for anyone wanting to do this, here's an example cobbled together python script it had a TON of dependancies but it will convert HEIC files to PNG with a single command:
python heic.py inputfile.heic

Its late now, so just dropping the file here, but i'll put it on github as a separate project tomorrow.
heic.py.zip

[avibrazil]

Very useful discussion and command line and script samples.

What about the inverse: conversion of a lossless DNG or lossless JPEG 2000 file, containing EXIF and XMP metadata, to lossless HEVC still into a HEIF container with metadata ?

I first tried PNG to lossless HEVC but stopped with a x265 error message related to wrong image dimensions.

Anyone knows how to do this ?

[deweydb]

@cconcolato Can we discuss this tile 49 (grid item) a bit further? I've been trying to follow the way it is extracted in the Nokia implementation:
https://github.com/nokiatech/heif/blob/c2877578d62481c4bfc6c906e7aef550046ab591/Srcs/common/imagegrid.cpp#L46

ImageGrid parseImageGrid(BitStream& input)
{
    ImageGrid grid;

    input.read8Bits(); // discard version
    bool read32BitFields = input.read8Bits() & 1; // flags

    grid.rowsMinusOne = input.read8Bits();
    grid.columnsMinusOne = input.read8Bits();

    if (read32BitFields)
    {
        grid.outputWidth = input.read32Bits();
        grid.outputHeight = input.read32Bits();
    }
    else
    {
        grid.outputWidth = input.read16Bits();
        grid.outputHeight = input.read16Bits();
    }

    return grid;
}

If i understand correctly, this code takes the first 8 bits and discards it. Then takes the next 8 bits, and treats is as a flag to check if we are using either 32 bit or 16 bit ints to describe the image size. Then takes the next 16 bits, 8 bits at a time to get the number of rows and columns, finally getting the image dimensions.

The file attached above has the following binary representation:

$ xxd -b tile49.hevc 
00000000: 00000000 00000000 00000000 00011000 01100110 01110100  ....ft
00000006: 01111001 01110000                                      yp

So from this I get:
Version = 0
Flag = 0 (so we are using 16 bit ints for size, which is correct and corresponds with the 8 byte file)
RowsMinusOne = 0
ColumnsMinusOne = 24
Width = 29798
Height = 28793

The rows is clearly incorrect, and so is the columns, the actual image was made up of 8 columns and 6 rows. The width and height are also incorrect, the image is 4096px by 3072px.

Is it possible that MP4Box did not properly extract this data when i did:
MP4Box -dump-item 49:path=tile49.hevc IMG_5921.HEIC
Perhaps i need to use different MP4Box flags for extracting this tile?

My python implementation of extracting this data for reference:

import numpy
import struct
f = open("tile49.hevc", "rb")
version, = struct.unpack('b', f.read(1))
print version
flag, = struct.unpack('?', f.read(1))
print flag
rowsMinusOne, = struct.unpack('b', f.read(1))
print rowsMinusOne
columnsMinusOne, = struct.unpack('b', f.read(1))
print columnsMinusOne
if flag:
	width, = struct.unpack('I', f.read(4))
	height, = struct.unpack('I', f.read(4))	
else:
	width, = struct.unpack('H', f.read(2))
	height, = struct.unpack('H', f.read(2))
print width
print height

Logfile of dump in question:

MP4Box -v -dump-item 49:path=tile49.hevc IMG_5921.HEIC������
[iso file] Starting to parse a top-level box at position 0
[iso file] Read Box type ftyp size 24 start 0
[iso file] Starting to parse a top-level box at position 24
[iso file] Read Box type meta size 3956 start 24
[iso file] Read Box type hdlr size 34 start 36
[iso file] Read Box type dinf size 36 start 70
[iso file] Read Box type dref size 28 start 78
[iso file] Read Box type url  size 12 start 94
[iso file] Read Box type pitm size 14 start 106
[iso file] Read Box type iinf size 1085 start 120
[iso file] Read Box type infe size 21 start 134
[iso file] Read Box type infe size 21 start 155
[iso file] Read Box type infe size 21 start 176
[iso file] Read Box type infe size 21 start 197
[iso file] Read Box type infe size 21 start 218
[iso file] Read Box type infe size 21 start 239
[iso file] Read Box type infe size 21 start 260
[iso file] Read Box type infe size 21 start 281
[iso file] Read Box type infe size 21 start 302
[iso file] Read Box type infe size 21 start 323
[iso file] Read Box type infe size 21 start 344
[iso file] Read Box type infe size 21 start 365
[iso file] Read Box type infe size 21 start 386
[iso file] Read Box type infe size 21 start 407
[iso file] Read Box type infe size 21 start 428
[iso file] Read Box type infe size 21 start 449
[iso file] Read Box type infe size 21 start 470
[iso file] Read Box type infe size 21 start 491
[iso file] Read Box type infe size 21 start 512
[iso file] Read Box type infe size 21 start 533
[iso file] Read Box type infe size 21 start 554
[iso file] Read Box type infe size 21 start 575
[iso file] Read Box type infe size 21 start 596
[iso file] Read Box type infe size 21 start 617
[iso file] Read Box type infe size 21 start 638
[iso file] Read Box type infe size 21 start 659
[iso file] Read Box type infe size 21 start 680
[iso file] Read Box type infe size 21 start 701
[iso file] Read Box type infe size 21 start 722
[iso file] Read Box type infe size 21 start 743
[iso file] Read Box type infe size 21 start 764
[iso file] Read Box type infe size 21 start 785
[iso file] Read Box type infe size 21 start 806
[iso file] Read Box type infe size 21 start 827
[iso file] Read Box type infe size 21 start 848
[iso file] Read Box type infe size 21 start 869
[iso file] Read Box type infe size 21 start 890
[iso file] Read Box type infe size 21 start 911
[iso file] Read Box type infe size 21 start 932
[iso file] Read Box type infe size 21 start 953
[iso file] Read Box type infe size 21 start 974
[iso file] Read Box type infe size 21 start 995
[iso file] Read Box type infe size 21 start 1016
[iso file] Read Box type infe size 21 start 1037
[iso file] Read Box type infe size 21 start 1058
[iso file] Read Box type infe size 21 start 1079
[iso file] Read Box type infe size 21 start 1100
[iso file] Read Box type infe size 21 start 1121
[iso file] Read Box type infe size 21 start 1142
[iso file] Read Box type infe size 21 start 1163
[iso file] Read Box type infe size 21 start 1184
[iso file] Read Box type iref size 148 start 1205
[iso file] Read Box type dimg size 108 start 1217
[iso file] Read Box type thmb size 14 start 1325
[iso file] Read Box type cdsc size 14 start 1339
[iso file] Read Box type iprp size 1779 start 1353
[iso file] Read Box type ipco size 1453 start 1361
[iso file] Read Box type colr size 560 start 1369
prof colour profile not supported 
[iso file] Read Box type hvcC size 112 start 1929
[iso file] Read Box type ispe size 20 start 2041
[iso file] Read Box type ispe size 20 start 2061
[iso file] Read Box type irot size 9 start 2081
[iso file] Read Box type pixi size 16 start 2090
[iso file] Read Box type colr size 560 start 2106
prof colour profile not supported 
[iso file] Read Box type hvcC size 112 start 2666
[iso file] Read Box type ispe size 20 start 2778
[iso file] Read Box type pixi size 16 start 2798
[iso file] Read Box type ipma size 318 start 2814
[iso file] Read Box type idat size 16 start 3132
[iso file] Read Box type iloc size 832 start 3148
[iso file] Starting to parse a top-level box at position 3980
[iso file] Read Box type mdat size 978000 start 3980

[deweydb]

Actually @cconcolato @jeanlf the output of dumping the grid is literally just the first 8 bytes of the HEIC file, is this correct?

$ MP4Box -quiet -dump-item 49:path=grid.hevc IMG_5921.HEIC

$ xxd -b IMG_5921.HEIC | head -n 2
00000000: 00000000 00000000 00000000 00011000 01100110 01110100  ....ft
00000006: 01111001 01110000 01101000 01100101 01101001 01100011  ypheic

$ xxd -b grid.hevc 
00000000: 00000000 00000000 00000000 00011000 01100110 01110100  ....ft
00000006: 01111001 01110000                                      yp

[aureliendavid]

@deweydb you're right, there is a bug in the item extraction

this is the location info (in the iloc box) of item 48 (a normal image) :

0030 0000 0000 0001 000b fee7 0000 36ed

It says that the item is at offset 0x0bfee7 in the file with a length of 0x36ed. So far so good.

this is the location info of item 49:

0031 0001 0000 0001 0000 0000 0000 0008

If we interpret it the same way it looks like the data is at offset 0 and length 8. That's why we extract the header of the file, but that's incorrect.

The fourth byte, which is different for these two items, is the construction_method flag. It says that in the case of item 49, the offset is from the start of the idat box and not from the start of the file.

If I look at the 8 bytes at the start of the idat box I get :

0000 0507 0fc0 0bd0

meaning

00 version
00 flags
05 rows-1
07 cols-1
0fc0 width=4032
0bd0 height=3024

which looks a lot better.

I'll look into the extraction bug in mp4box.

[avibrazil]

I'm glad to see this is evolving and bugs are getting destroyed.
Isn't the subject changing in a way that it should be moved to #905 ?

Let me know if you need more sample files.

[deweydb]

@aureliendavid thank you for solving that mystery, I was really pulling my hair out yesterday trying to figure out what i was doing wrong!

Interesting about the height and width, I must also have a bug somewhere in my reconstruction code because i should not have ended up with 4096px by 3072px because this is wrong. I just looked for some sample photos from an iPhone 7 and they are all 4032x3024 as you suggested.

[aureliendavid]

@deweydb you can test with the latest commit, the grid should be extracted properly now

[deweydb]

@aureliendavid awesome! seems fixed! so far getting correct values in all my tests!

As a side question that also touches on what @avibrazil mentioned above, would it be possible with mp4box to do the reverse of -dump-item. i.e. Can i take a folder with all the tiles, grid item, thumbnail, and exif item, and recombine them into an heic with mp4box?

Actually nevermind, there is another way to do what i was asking:
http://jpgtoheif.com/

[Koryantii]

I am using these two converters and it works just fine.

[MarcelMitzkus]

@deweydb Did you upload your python script to github or is the attached version the latest?

[rbouqueau]

FYI in case it can help some of you: for a customer that contacted the GPAC Licensing professional services, we have developed an HEIC to JPEG conversion tool that is very fast.