Legacy Blackberry JPEGs with embedded TIFFs report "Premature end of JPEG file"

[BrianInglis]

Have you searched the existing issues (both open and closed) in the libjpeg-turbo issue tracker to ensure that this bug report is not a duplicate?
YES

Does this bug report describe one of the two known and unsolvable issues with the JPEG format?
NO

Clear and concise description of the bug:
Processing legacy Blackberry JPEG images with embedded TIFFs reports "Premature end of JPEG file".
This causes failure of some downstream products specifically Google libwebp - which points fingers upstream: Issue 562 in webp: Blackberry photo in jpeg with embedded TIFF image fails to convert to webp

Steps to reproduce the bug (using only libjpeg-turbo):

$ file ~/Downloads/Sample.jpg
/home/BWI/Downloads/Sample.jpg: JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=10, manufacturer=Research In Motion, model=BlackBerry 9790, orientation=upper-left, xresolution=170, yresolution=178, resolutionunit=2, software=Rim Exif Version1.00a, datetime=2012:07:24 18:59:04], baseline, precision 8, 2592x1944, components 3
$ jpegtran -verbose -outfile ~/Downloads/SampleJ.jpg ~/Downloads/Sample.jpg
libjpeg-turbo version 2.1.2 (build 20211123)
Copyright (C) 2009-2021 D. R. Commander
Copyright (C) 2015, 2020 Google, Inc.
Copyright (C) 2019-2020 Arm Limited
Copyright (C) 2015-2016, 2018 Matthieu Darbois
Copyright (C) 2011-2016 Siarhei Siamashka
Copyright (C) 2015 Intel Corporation
Copyright (C) 2013-2014 Linaro Limited
Copyright (C) 2013-2014 MIPS Technologies, Inc.
Copyright (C) 2009, 2012 Pierre Ossman for Cendio AB
Copyright (C) 2009-2011 Nokia Corporation and/or its subsidiary(-ies)
Copyright (C) 1999-2006 MIYASAKA Masaru
Copyright (C) 1991-2020 Thomas G. Lane, Guido Vollbeding

Emulating The Independent JPEG Group's software, version 8d  15-Jan-2012

Start of Image
Miscellaneous marker 0xe1, length 408
Miscellaneous marker 0xe2, length 11
Define Quantization Table 0  precision 0
Define Quantization Table 1  precision 0
Define Restart Interval 50
Start Of Frame 0xc0: width=2592, height=1944, components=3
    Component 1: 2hx2v q=0
    Component 2: 1hx1v q=1
    Component 3: 1hx1v q=1
Define Huffman Table 0x00
Define Huffman Table 0x10
Define Huffman Table 0x01
Define Huffman Table 0x11
Start Of Scan: 3 components
    Component 1: dc=0 ac=0
    Component 2: dc=1 ac=1
    Component 3: dc=1 ac=1
  Ss=0, Se=63, Ah=0, Al=0
Premature end of JPEG file
End Of Image

Image(s) needed in order to reproduce the bug (if applicable):

Others available if required, preferably via private share or DM.

Expected behavior:
End Of Image
report normal end of file.

Observed behavior:

Premature end of JPEG file
End Of Image

and presumably equivalent in the library and downstream products, some of which save the resulting files, others just quit processing.

Platform(s) (compiler version, operating system version, CPU) on which the bug was observed:

$ apt-cyg la '^(libwebp|libjpeg.|libturbojpeg.|cmake|binutils|gcc-core|cygwin)$'
binutils 2.37-2 x86_64
cmake 3.20.0-1 x86_64
cygwin 3.3.4-2 x86_64
gcc-core 11.2.0-1 x86_64
libjpeg8 2.1.2-1 x86_64
libturbojpeg0 2.1.2-1 x86_64
libwebp 1.2.2-1 x86_64
$ jpegtran -verbose -outfile ~/Downloads/SampleJ.jpg ~/Downloads/Sample.jpg
libjpeg-turbo version 2.1.2 (build 20211123)
...
$ gcc --version
gcc (GCC) 11.2.0
...
$ uname -srvmo
CYGWIN_NT-10.0 3.3.4(0.341/5/3) 2022-01-31 19:35 x86_64 Cygwin
$ win-ver.sh
Windows 10 Home Client Core Multiprocessor Free 6.3 21H2 2009 vb_release 10 0 19044 1586

newlib 4.3 (Cygwin libc)
Windows 10 Home 21H2/[20]2009/10.0.19044.1586

$ head -n26 /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 101
model name      : AMD A10-9700 RADEON R7, 10 COMPUTE CORES 4C+6G
stepping        : 1
microcode       : 0x6006118
cpu MHz         : 3500.000
cache size      : 1024 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good acc_power nopl tsc_reliable nonstop_tsc cpuid aperfmperf pni pclmuldq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid_msr tbm perfctr_core perfctr_nb bpext ptsc mwaitx cpb hw_pstate fsgsbase bmi1 avx2 smep bmi2 xsaveopt arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decode_assists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov
bogomips        : 7000.00
TLB size        : 1536 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb eff_freq_ro acc_power

libjpeg-turbo release(s), commit(s), or branch(es) in which the bug was observed (always test the tip of the main branch or the latest stable pre-release to verify that the bug hasn't already been fixed):
libjpeg-turbo 2.1.2 (latest not quite yet available on Cygwin)

If the bug is a regression, the specific commit that introduced the regression (use git bisect to determine this):

Additional information:

[dcommander]

That libjpeg warning means exactly what it says: the decoder ran out of JPEG data to decode. All versions of libjpeg behave exactly the same way with the test image, so I fail to see why this is our bug.

[dcommander]

Generally speaking, libjpeg-turbo should handle or at least ignore TIFF data stored in EXIF, but if the encoder is storing the data as TIFF/JPEG, then libjpeg-turbo can't handle it. The IJG README file says as much:

libjpeg-turbo/README.ijg

Lines 221 to 222 in df3c3dc

	Although IJG's own code does not support TIFF/JPEG, the free libtiff library
	uses our library to implement TIFF/JPEG per the Note.

[BrianInglis]

I am not a graphics guy and don't know the details, but would not mind better understanding, why you are returning this status in the case of these files?
Yours and other utilities seem to perform the conversion successfully and save the output without any issues.
Is the situation really a Premature end of JPEG file, or in this case, perhaps because of the embedded TIFF, is it just an exceptional End of Image, and could and should that exception be made in the case of these kinds of files?
Or should downstreams really just report and ignore that status, as your and other utilities do?

[dcommander]

I haven't looked at it closely, but it appears as if libjpeg-turbo (and libjpeg) legitimately cannot decode a section of data between the Start of Scan and End of Image markers. That is also where the valid image data resides, so I don't know how libjpeg-turbo would distinguish between the two. It would be safe to ignore the warning in this case, but ignoring libjpeg API warnings isn't necessarily a safe practice for applications, such as browsers, that have to handle arbitrary data.

Unfortunately, I can't comment more intelligently without understanding exactly what the encoder is doing. If it is implementing some form of TIFF/JPEG encoding, then libjpeg-turbo won't be able to read that (but libtiff might.) It appears as if jpegtran -copy all gets rid of the unwanted TIFF data without damaging the valid data.