New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libtiff denying saving of really large images #148

Closed
zebastian opened this Issue Nov 1, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@zebastian
Collaborator

zebastian commented Nov 1, 2016

see also problem reported here by smurfboard: http://www.fractalforums.com/bug-reporting-b231/2-09-2-'zlib-cannot-deal-with-buffers/
program is crashing when trying to write 16bit tiff files, message: "zlib cannot deal with buffers this size" , image size is 16000x28000 pixels, i have 32gb memory.

Maybe tiff saving can be leveraged in memory consumption or saving can be "streamed" instead of total allocation.

@zebastian zebastian added the BUG label Nov 1, 2016

@zebastian zebastian self-assigned this Nov 1, 2016

@buddhi1980

This comment has been minimized.

Show comment
Hide comment
@buddhi1980

buddhi1980 Nov 1, 2016

Owner

Here is the answer: http://www.awaresystems.be/imaging/tiff/faq.html#q8

The format uses 32bit offsets, and as such, it is limited to 4 gigabytes. Many implementations handle these offsets using signed integers, and thus support files of up to 2 gigabytes, but the only real limit resulting from the format specification is 4 gigabytes.

Owner

buddhi1980 commented Nov 1, 2016

Here is the answer: http://www.awaresystems.be/imaging/tiff/faq.html#q8

The format uses 32bit offsets, and as such, it is limited to 4 gigabytes. Many implementations handle these offsets using signed integers, and thus support files of up to 2 gigabytes, but the only real limit resulting from the format specification is 4 gigabytes.

@buddhi1980

This comment has been minimized.

Show comment
Hide comment
@buddhi1980

buddhi1980 Nov 1, 2016

Owner

Fixed by commit cfe4ed0

I'm going to release 2.09-3 with this bugfix

Owner

buddhi1980 commented Nov 1, 2016

Fixed by commit cfe4ed0

I'm going to release 2.09-3 with this bugfix

@mclarekin

This comment has been minimized.

Show comment
Hide comment
@mclarekin

mclarekin Nov 1, 2016

Collaborator

can you include this replacement text with V2.09.3

transformCommon.benesiT1EnabledFalse = container->Get("transf_
benesi_T1_enabled_false");
transformCommon.benesiT1MEnabledFalse = container->Get("transf_
benesi_T1M_enabled_false");

I had them switched

On Wed, Nov 2, 2016 at 9:43 AM, Krzysztof Marczak notifications@github.com
wrote:

Fixed by commit cfe4ed0
cfe4ed0

I'm going to release 2.09-3 with this bugfix


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#148 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMLZHNqqq1cNtBKw94Usb-5TQIZIEIx-ks5q56R1gaJpZM4KmZ_y
.

Collaborator

mclarekin commented Nov 1, 2016

can you include this replacement text with V2.09.3

transformCommon.benesiT1EnabledFalse = container->Get("transf_
benesi_T1_enabled_false");
transformCommon.benesiT1MEnabledFalse = container->Get("transf_
benesi_T1M_enabled_false");

I had them switched

On Wed, Nov 2, 2016 at 9:43 AM, Krzysztof Marczak notifications@github.com
wrote:

Fixed by commit cfe4ed0
cfe4ed0

I'm going to release 2.09-3 with this bugfix


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#148 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMLZHNqqq1cNtBKw94Usb-5TQIZIEIx-ks5q56R1gaJpZM4KmZ_y
.

@buddhi1980

This comment has been minimized.

Show comment
Hide comment
@buddhi1980

buddhi1980 Nov 2, 2016

Owner

@mclarekin , I can add this change to 2.09-3

Owner

buddhi1980 commented Nov 2, 2016

@mclarekin , I can add this change to 2.09-3

@buddhi1980

This comment has been minimized.

Show comment
Hide comment
@buddhi1980

buddhi1980 Nov 2, 2016

Owner

@mclarekin , I'm not sure what I should change here. In 2.09-2 transformCommon.benesiT1MEnabledFalse didn't exist.
You have added TransformScaleVaryVCLIteration() formula just 7 days ago, so it's not included in 2.09 at all.
Here is your change: 38e14a9#diff-048e74ab08be3a31d1abc425824bb60d

Owner

buddhi1980 commented Nov 2, 2016

@mclarekin , I'm not sure what I should change here. In 2.09-2 transformCommon.benesiT1MEnabledFalse didn't exist.
You have added TransformScaleVaryVCLIteration() formula just 7 days ago, so it's not included in 2.09 at all.
Here is your change: 38e14a9#diff-048e74ab08be3a31d1abc425824bb60d

@mclarekin

This comment has been minimized.

Show comment
Hide comment
@mclarekin

mclarekin Nov 2, 2016

Collaborator

my mistake, I thought I had made the change before V2.09 release, but I
now see it was done after. :)

On Thu, Nov 3, 2016 at 6:20 AM, Krzysztof Marczak notifications@github.com
wrote:

@mclarekin https://github.com/mclarekin , I'm not sure what I should
change here. In 2.09-2 transformCommon.benesiT1MEnabledFalse didn't
exist.
You have added TransformScaleVaryVCLIteration() formula just 7 days ago,
so it's not included in 2.09 at all.
Here is your change: 38e14a9#diff-048e74ab08be3a31d1abc425824bb60d
38e14a9#diff-048e74ab08be3a31d1abc425824bb60d


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#148 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMLZHEn6OfnuE3YTpRkDuetvXw7PiGdcks5q6MZpgaJpZM4KmZ_y
.

Collaborator

mclarekin commented Nov 2, 2016

my mistake, I thought I had made the change before V2.09 release, but I
now see it was done after. :)

On Thu, Nov 3, 2016 at 6:20 AM, Krzysztof Marczak notifications@github.com
wrote:

@mclarekin https://github.com/mclarekin , I'm not sure what I should
change here. In 2.09-2 transformCommon.benesiT1MEnabledFalse didn't
exist.
You have added TransformScaleVaryVCLIteration() formula just 7 days ago,
so it's not included in 2.09 at all.
Here is your change: 38e14a9#diff-048e74ab08be3a31d1abc425824bb60d
38e14a9#diff-048e74ab08be3a31d1abc425824bb60d


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#148 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMLZHEn6OfnuE3YTpRkDuetvXw7PiGdcks5q6MZpgaJpZM4KmZ_y
.

@buddhi1980

This comment has been minimized.

Show comment
Hide comment
@buddhi1980

buddhi1980 Nov 3, 2016

Owner

I had one surprise here. When program was compiled with mingw32, then 'long int' type was only 32-bit size (PNG and TIFF failed under Windows10).
There is always needed to use int64_t or uint64_t to be sure that variable will have 64-bit size.
Latest commit solved this problem (63a14a6)
The program was tested under Win10 and everything seams to work properly (tested image of resolution 16000x28000)
Tomorrow I'm going to publish new packages.

Owner

buddhi1980 commented Nov 3, 2016

I had one surprise here. When program was compiled with mingw32, then 'long int' type was only 32-bit size (PNG and TIFF failed under Windows10).
There is always needed to use int64_t or uint64_t to be sure that variable will have 64-bit size.
Latest commit solved this problem (63a14a6)
The program was tested under Win10 and everything seams to work properly (tested image of resolution 16000x28000)
Tomorrow I'm going to publish new packages.

@buddhi1980 buddhi1980 closed this Nov 5, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment