RangeError #50
Comments
|
I can take a look at this next week.
…On Thu, Jun 15, 2017, 10:49 AM Matthew Barbour ***@***.***> wrote:
I get this error with a few images (very few out of thousands, but it
keeps coming up), none of which appear to have any issues when opened in
other applications:
RangeError (index): Invalid value: Only valid value is 0: 1
#0 List.[] (dart:core-patch/growable_array.dart:151)
#1 JpegScan._decodeHuffman (package:image/src/formats/jpeg/jpeg_scan.dart:134:18)
#2 JpegScan._decodeACSuccessive (package:image/src/formats/jpeg/jpeg_scan.dart:238:20)
#3 JpegScan._decodeBlock (package:image/src/formats/jpeg/jpeg_scan.dart:308:13)
#4 JpegScan.decode (package:image/src/formats/jpeg/jpeg_scan.dart:78:11)
#5 JpegData._readSOS (package:image/src/formats/jpeg/jpeg_data.dart:568:55)
#6 JpegData._read (package:image/src/formats/jpeg/jpeg_data.dart:338:11)
#7 JpegData.read (package:image/src/formats/jpeg/jpeg_data.dart:94:5)
#8 JpegDecoder.decodeImage (package:image/src/formats/jpeg_decoder.dart:46:10)
#9 decodeImage (package:image/src/formats/formats.dart:60:18)
Here is one such image:
[image: test]
<https://user-images.githubusercontent.com/1385789/27200352-09d883f8-51df-11e7-8982-df440a43149a.jpg>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#50>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADeS81yRJ72b2CUB9kuLdMwJfR4uVMsaks5sEZOZgaJpZM4N7s5n>
.
|
|
I started taking a look at the problem. Might take a bit to work out. I pushed a fix that keeps it from crashing, but doesn't solve the underlying problem and the decoded image has a lot of block artifacts. I checked the image against several jpeg libraries. libjpeg, the standard library most programs use, does decode the image correctly. It's also one of the more complex decoders. Other decoders, such as the one I ported the dart version from, fail for this image. I'll have to figure out what libjpeg's secret ingredient is which will take a little time. |
|
I tried opening the image in GIMP and when I did it said that the image has embedded color profile "c2", and asks if I want to convert it to the RGB working space. I don't know if that might have something to do with the odd decoding. |
|
It has to do with being progressive encoded. I thought those were being
handled by apparently not. I hope to be able to finish working on this
soon, work and family have just been keeping me extra busy so carving out
time to finish it has been slow going.
…On Wed, Jun 28, 2017, 7:09 AM Matthew Barbour ***@***.***> wrote:
I tried opening the image in GIMP and when I did it said that the image
has embedded color profile "c2", and asks if I want to convert it to the
RGB working space. I don't know if that might have something to do with the
odd decoding.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#50 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADeS8zYDCVB5DeqfKF6krQBEiW_jvMczks5sIl6BgaJpZM4N7s5n>
.
|
|
I rewrote the JPEG decoder and it's mostly finished, and it decodes this image correctly now. It should just be another day or so before I finalize all of the testing and push the new version. |
|
Wow, I'm impressed. I really appreciate the work you put into this. Thanks a lot for tackling this. |
|
I pushed the new jpeg decoder. I did a lot of testing and it seems to be ok, but if you wouldn't mind testing the git version with your usage, I'd appreciate it. |
|
I've actually got about 200,000 images that I've previously successfully used with this library, I'm going to go ahead and run them all through the decoder to help make sure everything's okay. |
|
Hit an error:
This image previously decoded fine. A few other images also output a similar error with this same stack trace, haven't encountered a different error yet. |
|
Found one more error: It's happened on several images which were not decoding before. I would make a new ticket, but since we're testing the re-written jpeg decoder I figured I'd just keep it all here. Here's one of the images:
|
|
Thanks a lot for the tests! The challenge in porting a C library to Dart is that Dart doesn't have pointers and many C libraries do a lot of pointer manipulation, and Dart and C have different integer representations, and C libraries like to do integer bit manipulation. There are a few code paths in the new version that I haven't fully tested yet due to lack of images that reach those parts of the code, so I appreciate the examples. |
|
I completely understand, I was converting some compression-related C-code for work to a scripted language for work a little while back, and it was a quite the ordeal to convert all the fancy memory tricks it would do. I've only tested a small fraction of my images so far, should I go ahead and test everything and post all the results here, or wait for these results to be addressed first? |
|
I'm almost done with these tests, the first one was because Dart and C seem to have a different order of operations for + and >>, which caused the index range error. Looking at why the second one fails now. |
|
Okay, I'll just wait for your next push then. |
|
I pushed the fixes for those images. |
|
I did find another test case that fails, CMYK encoded jpegs. I don't think those are very common, the library I ported doesn't support them. Hopefully your images don't have any. |
|
Got another: This one previously worked.
|
|
I know I have at least one CMYK image, which you may remember from #25 CMYK encoding was working before, maybe it would be beneficial to keep the old decoder around and branch to it when CMYK is detected? |
|
That's right. I'll just copy the cmyk code from the other decoder.
…On Fri, Jul 7, 2017, 11:03 AM Matthew Barbour ***@***.***> wrote:
I know I have at least one CMYK image, which you may remember from #25
<#25>
CMYK encoding was working before, maybe it would be beneficial to keep the
old decoder around and branch to it when CMYK is detected?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#50 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADeS89NHKizrpFIV2E4VuB7YJI4gwv2Mks5sLnLQgaJpZM4N7s5n>
.
|
|
Picking away at it. I got 1:1 CMYK working, which handles the #25 image, and pushed that fix. But I still need to fix it for other scaling factors of CMYK, as well as non 1:1 grayscale images, which is the problem with the last image you attached. Fortunately once I get the non 1:1 grayscale images working, it'll be easy enough to apply that to the CMYK case (or vise versa), so I should be able to get these working soon. The only thing that may slow down getting a fix out is I will be on vacation starting tomorrow, so hopefully I can get through the issues today (neglecting my day job so I get through these issues sooner than later :-)), otherwise it'll be another week before I can get back to it. |
|
Well geeze, thanks for putting in all that effort. I hope your vacation goes well! |
|
The last grayscale image you posted seems to be working, just 2:2 cmyk format left, afaik. I don't know how common that format is vs 1:1 cmyk. Almost all open source decoders don't implement cmyk at all. Hope to get it figured out tonight, if not then it'll have to wait. |
|
Sounds like you get bragging rights for being one of the only open source decoders to implement CMYK. Congratulations! |
|
I know you're on vacation, but my test of all my images finished and I got a few to report. We can look into them when you get back. |
|
ImageException: JPEG ERROR: -227
|
|
ImageException: JPEG ERROR: -206 The image for this error is rather large, let me know if you need it. |
|
ImageException: JPEG ERROR: -226
|
|
I am back working on this, here and there as time permits. I decided to merge that new jpeg decoder, which correctly handled the huffman decoding but not the cmyk/grayscale color spaces, with the original one, which worked in all cases but had the original huffman decoding bug that caused the range error. Hopefully I'll finish this hybrid decoder in the next day or two. |
|
Welcome back. Hope your vacation went well. |
|
Thanks, it was nice being away from the computer and with the family for a bit :-) |
|
I pushed another change. I probably wasted some time porting that other decoder, it was incomplete and I just don't have the time/ability to fix up the missing pieces. So I reverted back to the original decoder, which was actually doing a pretty good job, and found/fixed the original problem which turned out to be simpler than I anticipated. All of the images you have sent me so far are decoding now. One caveat is that the first image in this bug doesn't seem to be as smooth as other decoders, so there are some subtle blocking artifacts in the dart version. I'm not sure yet what's missing. I verified the original decoder that I had ported, which happens to be the same one Firefox uses for decoding PDF files, has the same results as this dart version, so any further improvements will require more brain power. |
|
Hi. Thanks for the library @brendan-duncan. Im also having a similar issue: |
|
Do you have an example I can test against? |
I get this error with a few images (very few out of thousands, but it keeps coming up), none of which appear to have any issues when opened in other applications: It occurs when I try to read the file.
Here is one such image:
The text was updated successfully, but these errors were encountered: