Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to read large files #915

Open
pillowfication opened this issue Jul 24, 2020 · 30 comments
Open

Failing to read large files #915

pillowfication opened this issue Jul 24, 2020 · 30 comments
Labels
bug there is a bug in the way jimp behaves help wanted solution in issue

Comments

@pillowfication
Copy link

pillowfication commented Jul 24, 2020

I'm trying to use Jimp to process images that are about 70-100 MB in size. However, I get the following error

// image is 87.7 MB
Jimp.read('big-image', (err, img) => {
  console.log(err)
})
> Error: maxMemoryUsageInMB limit exceeded by at least 226MB

Is there any way around this?

@IanStewart-Celayix
Copy link

I get this issue in 0.16.0 but not 0.9.8.

@ghost
Copy link

ghost commented Sep 7, 2020

same issue heare (0.16.1)

@gregholland
Copy link

Same issue with 0.16.1

@kde12327
Copy link

kde12327 commented Sep 9, 2020

I have same problem. So I tried fix something.
[(https://developer.aliyun.com/mirror/npm/package/jpeg-js)]
On this site, there are maxMemoryUsageInMB option on decode function.

And I change \node_modules@jimp\core\dist\utils\image-bitmap.js:196 line.

this.bitmap = this.constructor.decoders[_mime](data);
to
this.bitmap = this.constructor.decoders[_mime](data, {maxMemoryUsageInMB: 2000});

Then I can read the large image. Try this one.

@master117
Copy link

master117 commented Sep 21, 2020

Having the same issue, 10mb jpgs turn into 1.1gb+ ram usage and crash with Error: maxMemoryUsageInMB limit exceeded
Happens on newest jimp and the newest windows server

@bergiusj
Copy link

I'm having the same issue.

Our problem has only been reported from users with a specific phone model (Huawei P30).
We're using JIMP to compress and resize images taken from a native mobile application.

Running version:
"jimp": "^0.16.0"

@joelcoronah
Copy link

Same problem here, any update?

@ajitStephen
Copy link

I get this issue in 0.16.0 but not 0.9.8.

worked for me.

@chibuike-vincent
Copy link

I have same problem. So I tried fix something.
[(https://developer.aliyun.com/mirror/npm/package/jpeg-js)]
On this site, there are maxMemoryUsageInMB option on decode function.

And I change \node_modules@jimp\core\dist\utils\image-bitmap.js:196 line.

this.bitmap = this.constructor.decoders[_mime](data);
to
this.bitmap = this.constructor.decoders[_mime](data, {maxMemoryUsageInMB: 2000});

Then I can read the large image. Try this one.

This works locally but not working with the live app. How do I fix it

@githubBrandon
Copy link

Downgrading to 0.9.8 worked for me as well.

@friddle
Copy link

friddle commented Dec 23, 2020

if error is maxResolutionInMP limit exceeded by 36MP

 this.bitmap = this.constructor.decoders[_mime](data, {
        maxResolutionInMP:400
      });

@hisham
Copy link

hisham commented Feb 8, 2021

You can fix by overriding the jpeg-js decoder jimp uses as follows:

Jimp.decoders['image/jpeg'] = (data: Buffer) => JPEG.decode(data, { maxMemoryUsageInMB: 1024 });

@ianwieds
Copy link

ianwieds commented Mar 9, 2021

@hisham where do we put this code? I converted it to regular js like so

Jimp.decoders['image/jpeg'] = (data) => JPEG.decode(data, { maxMemoryUsageInMB: 1024 })

But I get JPEG undefined. Can you help? Thank you

@hisham
Copy link

hisham commented Mar 9, 2021

You put it before you use any Jimp operations - i.e.:

import JPEG from 'jpeg-js';

constructor() {     
Jimp.decoders['image/jpeg'] = (data: Buffer) => JPEG.decode(data, { maxMemoryUsageInMB: 1024 });
 }

myMethod(image: Buffer) {
    const jimpImage = await Jimp.read(jpegBuffer);
}

@1Map
Copy link

1Map commented Mar 16, 2021

Just a question. How will you handle memory for 'image/tiff' images ?

Is the following correct for NodeJS ?

const jimpJPEG = require('jpeg-js')
const jimpTIFF = require('utif')

jimp.decoders['image/jpeg'] = (data) => {
	return jimpJPEG.decode(data, {
		maxMemoryUsageInMB: 1024
	})
}
jimp.decoders['image/tiff'] = (data) => {
	var ifds = jimpTIFF.decode(data, {
		maxMemoryUsageInMB: 1024
	})
	var page = ifds[0]
	jimpTIFF.decodeImages(data, ifds)
	var rgba = jimpTIFF.toRGBA8(page)
	return {
		data: Buffer.from(rgba),
		width: page.t256[0],
		height: page.t257[0]
	}
}

@isikhi
Copy link

isikhi commented Mar 29, 2021

Any update?
problem is here: node_modules/@jimp/core/dist/utils/image-bitmap.js -> node_modules/jpeg-js/lib/decoder.js

@koji98
Copy link

koji98 commented Apr 20, 2021

Having this same problem too. Is there any major difference between the 0.9 and 0.16 versions?

@tomprogers
Copy link

tomprogers commented Sep 6, 2021

The solutions described here weren't working for me. I tried 1024, 2048, even 4096. In every case, the error said the limit has been exceeded by 466MP:

throw new Error(`maxResolutionInMP limit exceeded by ${exceededAmount}MP`);
                    ^

Error: maxResolutionInMP limit exceeded by 466MP
    at constructor.parse (/Users/me/projects/my-app/node_modules/jpeg-js/lib/decoder.js:738:21)
    at Object.decode (/Users/me/projects/my-app/node_modules/jpeg-js/lib/decoder.js:1109:11)
    at Object.Jimp.decoders.image/jpeg (/Users/me/projects/my-app/src/ingest-images.js:11:14)
    at Jimp.parseBitmap (/Users/me/projects/my-app/node_modules/@jimp/core/dist/utils/image-bitmap.js:196:53)
    at Jimp.parseBitmap (/Users/me/projects/my-app/node_modules/@jimp/core/dist/index.js:431:32)
    at /Users/me/projects/my-app/node_modules/@jimp/core/dist/index.js:373:15

The solution was to override the maxResolutionInMP value in addition to the memory value. This works for me:

const Jimp = require('jimp')

// add support for very large image files
const JPEG = require('jpeg-js')
Jimp.decoders['image/jpeg'] = (data) => JPEG.decode(data, {
	maxMemoryUsageInMB: 6144,
	maxResolutionInMP: 600
})

@smirea
Copy link

smirea commented Sep 8, 2021

same problem, still happening for 16mb jpegs

tomprogers hack worked great for me, but having jimp use 200mb of memory for a 10mb jpeg is not ideal

@Dakuan
Copy link

Dakuan commented Oct 4, 2021

this is lots of fun with 100mb pro photos

@kane-mason
Copy link

i found the solutions above would only partially work and further transforms like crop would error

This is what worked for me

const cachedJpegDecoder = Jimp.decoders['image/jpeg']
Jimp.decoders['image/jpeg'] = (data) => {
  const userOpts = { maxMemoryUsageInMB: 1024 }
  return cachedJpegDecoder(data, userOpts)
}

@kahilkubilay
Copy link

What should be the maximum size range (px or size) of an image? How can I calculate the range of image sizes?

@rcupic
Copy link

rcupic commented Jun 24, 2022

I have tried to read buffer of an 2.5MB image and got this error.

@ajitStephen
Copy link

try using https://github.com/lovell/sharp

@rcupic
Copy link

rcupic commented Jun 24, 2022

try using https://github.com/lovell/sharp

Yup, I have migrated to https://sharp.pixelplumbing.com/

@hipstersmoothie hipstersmoothie added bug there is a bug in the way jimp behaves solution in issue labels Feb 6, 2023
@AnjulaS
Copy link

AnjulaS commented Apr 26, 2023

I just faced this today as well , is this still open or was a solution presented?

@generated
Copy link

i found the solutions above would only partially work and further transforms like crop would error

This is what worked for me

const cachedJpegDecoder = Jimp.decoders['image/jpeg']
Jimp.decoders['image/jpeg'] = (data) => {
  const userOpts = { maxMemoryUsageInMB: 1024 }
  return cachedJpegDecoder(data, userOpts)
}

@kane-mason's solution if you're using typescript you might have to add <any> pre to jimp.decoders["image/jpeg"] as the default constructor only expects one argument.

const cachedJpegDecoder = <any>jimp.decoders['image/jpeg']

askmeaboutlo0m added a commit to askmeaboutlo0m/postybirb-plus that referenced this issue Aug 12, 2023
It's capped at some fixed value, which botches loading images of larger
resolutions for no good reasons This disables the limit in the deferred
image manipualtion worker. Presumably the same manipulation isn't
necessary in the non-deferred variant, since those images are tiny.

Upstream issue: jimp-dev/jimp#915
askmeaboutlo0m added a commit to askmeaboutlo0m/postybirb-plus that referenced this issue Aug 21, 2023
It's capped at some fixed value, which botches loading images of larger
resolutions for no good reasons This disables the limit in the deferred
image manipualtion worker. Presumably the same manipulation isn't
necessary in the non-deferred variant, since those images are tiny.

Upstream issue: jimp-dev/jimp#915
mvdicarlo pushed a commit to mvdicarlo/postybirb-plus that referenced this issue Aug 25, 2023
It's capped at some fixed value, which botches loading images of larger
resolutions for no good reasons This disables the limit in the deferred
image manipualtion worker. Presumably the same manipulation isn't
necessary in the non-deferred variant, since those images are tiny.

Upstream issue: jimp-dev/jimp#915
@DONGXUyyds
Copy link

Have you solved your problem? I have the same problem

@lopugit
Copy link

lopugit commented Oct 26, 2023

@kane-mason genius 💯

@tamtmh
Copy link

tamtmh commented Jan 11, 2024

Have you solved your problem? I have the same problem

const cachedJpegDecoder = Jimp.decoders['image/jpeg']
Jimp.decoders['image/jpeg'] = (data) => {
  const userOpts = { maxMemoryUsageInMB: 1024 }
  return cachedJpegDecoder(data, userOpts)
}

worked for me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug there is a bug in the way jimp behaves help wanted solution in issue
Projects
None yet
Development

No branches or pull requests