Can't generate zip file of images #151

Closed
Kwbmm opened this Issue Jul 9, 2014 · 18 comments

Projects

None yet

4 participants

@Kwbmm
Kwbmm commented Jul 9, 2014

I'm developing a script that fetches a series of images ( that can be either png or jpg ), encodes them in base64 and zip them.
Everything seems to work as expected except for the generate function that throws an error:
TypeError: transform[inputType] is undefined
After some digging I saw that at some point the function exports.transformTo receives as input and empty Uint8Array that when passed to exports.getTypeOf from var inputType = exports.getTypeOf(input); doesn't return anything ( for reasons I can't get ).
This is the code in JSZip that creates issues:

exports.transformTo = function(outputType, input) {
    if (!input) {
        // undefined, null, etc
        // an empty string won't harm.
        input = "";
    }
    if (!outputType) {
        return input;
    }
    exports.checkSupport(outputType);
    var inputType = exports.getTypeOf(input);
    var result = transform[inputType][outputType](input);
    return result;
};

/**
 * Return the type of the input.
 * The type will be in a format valid for JSZip.utils.transformTo : string, array, uint8array, arraybuffer.
 * @param {Object} input the input to identify.
 * @return {String} the (lowercase) type of the input.
 */
exports.getTypeOf = function(input) {
    if (typeof input === "string") {
        return "string";
    }
    if (Object.prototype.toString.call(input) === "[object Array]") {
        return "array";
    }
    if (support.nodebuffer && nodeBuffer.test(input)) {
        return "nodebuffer";
    }
    if (support.uint8array && input instanceof Uint8Array) {
        return "uint8array";
    }
    if (support.arraybuffer && input instanceof ArrayBuffer) {
        return "arraybuffer";
    }
};

While this one is part of the code I wrote:

var chapterImgs = new Array();
function convertImgToBase64(image, callback, outputFormat){
    var canvas = document.createElement('CANVAS');
    var ctx = canvas.getContext('2d');
    var img = new Image;
    img.src = image[0];
    img.width = image[1];
    img.height = image[2];
    canvas.height = img.height;
    canvas.width = img.width;
    ctx.drawImage(img,0,0);
    var dataURL = canvas.toDataURL(outputFormat || 'image/png');
    callback.call(this, dataURL);
    canvas = null;
}
function getChapterImgs(result){
    $.each(result,function(i,property){
        chapterImgs.push(new Array("http://cdn.mangaeden.com/mangasimg/"+property[1],property[2],property[3]));
    });
    chapterImgs.reverse();
}
$(document).ready(function(){

    //Here some other code goes on...

    $.getJSON("//www.mangaeden.com/api/chapter/"+manga[4]+'/').done(function(pages){
        getChapterImgs(pages.images);
        $.each(chapterImgs,function(i,image){
            var format = image[0].split('.');
            if(format[3] === 'jpg')
                convertImgToBase64(image,function(data){
                    var imgBase64 = data.split('base64,');
                    zip.file(i+'.jpg',imgBase64[1],{base64: true});
            },'image/jpeg');
            else
                convertImgToBase64(image,function(data){
                    var imgBase64 = data.split('base64,');
                    zip.file(i+'.png',imgBase64[1],{base64: true});
                });
        });
        if(JSZip.support.blob){
            try {
                // console.dir(zip);
                var blob = zip.generate({type: "blob"});
                saveAs(blob,"chapter.zip");
            } catch(e){
                console.log("Exception!!");
                console.log(e);
            }
        }
    });     
});

Would love some help!

@dduponchel
Collaborator

I don't see any obvious issue... Could you check the type of imgBase64[1] ? (I have troubles testing canvas.toDataURL on my side).

@dduponchel
Collaborator

at some point the function exports.transformTo receives as input and empty Uint8Array that when passed to exports.getTypeOf from var inputType = exports.getTypeOf(input); doesn't return anything ( for reasons I can't get ).

If transformTo gets an empty Uint8Array and can't find its type, that would mean that (support.uint8array && input instanceof Uint8Array) is false. What browser do you use ? Do you have a Uint8Array shim on your page ?

@Kwbmm
Kwbmm commented Jul 10, 2014

imgBase64[1] is string.
So I modified the method exports.getTypeOf in this way to be sure that support.uint8array is true:

exports.getTypeOf = function(input) {
    if (typeof input === "string") {
        return "string";
    }
    if (Object.prototype.toString.call(input) === "[object Array]") {
        return "array";
    }
    if (support.nodebuffer && nodeBuffer.test(input)) {
        return "nodebuffer";
    }
    if (support.uint8array && input instanceof Uint8Array) {
        return "uint8array";
    }
    if (support.arraybuffer && input instanceof ArrayBuffer) {
        return "arraybuffer";
    }
    if(support.uint8array)
        console.log(input);
};

Look at the last 2 lines...
Now this is the output on the console:
errors
The first line above is given by console.log(input) found in exports.getTypeOf
This is the structure of the resulting Uint8Array:
structure

I'm using Firefox 30.0

@dduponchel dduponchel added a commit to dduponchel/jszip that referenced this issue Jul 10, 2014
@dduponchel dduponchel add a test page for the issue #151 0f604e6
@dduponchel
Collaborator

So we have a Uint8Array which doesn't pass the test input instanceof Uint8Array...
I've created a test page with your code (I've slightly modified convertImgToBase64) : http://dduponchel.github.io/jszip/issue_151.html
This page works (no exception, the download is triggered and the zip file is valid) on my machine in Firefox 30 (windows and linux). Could you check it on your side ?

If it works for you too, do you have an example page to reproduce your issue ?

@Kwbmm
Kwbmm commented Jul 10, 2014

Yes it works.
I did some tests and came up with a possible culprit: Scriptish ( Firefox Extension ). I'm running my script inside it so that it can be loaded on the needed page.
I'm saying Scriptish might be the culprit because a script I wrote, which is pretty much the same as yours in the test page here above, isn't working if used inside Scriptish when in fact it should..
Now I'm wondering what should I do.. I don't have such a deep knowledge of JS to understand if my issue can be solved or if the project i'm working on is good to be dropped due to this unsolvable issues..
Or should I go ask Scriptish devs?
I'm totally lost...

@dduponchel
Collaborator

I reproduced your error in scriptish, I'll investigate to find out why JSZip doesn't work in this environment.

@Kwbmm
Kwbmm commented Jul 11, 2014

All right! I'll wait

@dduponchel
Collaborator

This seems to come from TextEncoder and TextDecoder objects (used by JSZip to do utf8 conversion).

The following test case logs isU8= true in a <script> and isU8= false in Scriptish.

var encoder = new TextEncoder("utf-8");
var testU8 = encoder.encode("test");

var isU8 = testU8 instanceof Uint8Array;

console.log("testU8=", testU8);
console.log("isU8=", isU8);
@Kwbmm
Kwbmm commented Jul 11, 2014

So.. I'm quite confused.. Is JSZip to blame or Scriptish?

@dduponchel
Collaborator

I don't know yet if this comes from Scriptish or from Firefox itself (but it's not JSZip). I'll dig a bit more and report the issue.

I don't see any workaround without modifying JSZip source code. If you're ok with a custom file (until a fix for this issue) you can comment these lines in utf8.js and object.js.

@dduponchel
Collaborator

I reported the bug to Mozilla here. I'll also fix it in JSZip (you can blame JSZip for using an API flagged as experimental :p)

@Kwbmm
Kwbmm commented Jul 12, 2014

Good!
In the meantime I'll try your fix and see if that works!
I'll report back asap

@Kwbmm
Kwbmm commented Jul 13, 2014

I did some testing with the fixed version of JSZip.
Now I'm able to download the zip file BUT images are either white or black.
I thought this was due to the image not being rendered as zip.file(i+'.png',imgBase64[1],{base64: true}); was called. So I used window.setTimeout to allow at least some images to render completely and see if that was the issue.
Also I append the images into a hidden div at the bottom of the page.
Still no luck..

$.getJSON("//www.mangaeden.com/api/chapter/"+manga[4]+'/').done(function(pages){
    getChapterImgs(pages.images);
    $.each(chapterImgs,function(i,image){
        $('#images').append('<img src="'+image[0]+'"/>');
        var format = image[0].split('.');
        if(format[3] === 'jpg')
            convertImgToBase64(image,function(data){
                window.setTimeout(function(){
                    var imgBase64 = data.split('base64,');
                    zip.file(i+'.png',imgBase64[1],{base64: true});                                 
                },5000);
            },'image/jpeg');
        else
            convertImgToBase64(image,function(data){
                window.setTimeout(function(){
                    var imgBase64 = data.split('base64,');
                    zip.file(i+'.png',imgBase64[1],{base64: true});                                 
                },5000);
            });
        $(that).siblings('.dwn').removeAttr('disabled');
    });
});

Is there something we could do?

@Mithgol
Contributor
Mithgol commented Jul 14, 2014

Something's not right: in the format[3] === 'jpg' case you still add '.png' as extension.

Could it be that some files are JPEG pictures and thus they are rendered incorrectly (“either white or black”) when they are given '.png' extension?

@dduponchel
Collaborator

You could also use an ajax request (with xhr.responseType = 'arraybuffer') to get the image content. I tested it in scriptish on a local server and I got no corruption.

@Kwbmm
Kwbmm commented Jul 14, 2014

@Mithgol Yeah, this is a mistake. It should be .jpgbut I don't think this changes the outcome
@dduponchel Do you mean that after I fetch the list of links to images I should make an AJAX request to each one of them? Like:

$.getJSON("//www.mangaeden.com/api/chapter/"+manga[4]+'/').done(function(pages){
    getChapterImgs(pages.images);
    $.each(chapterImgs,function(i,image){
        $.get(image[0]).done(function(result){
            //Do something
        });
    });
});
@dduponchel
Collaborator

@Kwbmm almost :)
You will get encoding issues with $.get (it expects a text response, not an image). jQuery can't (yet) handle binary data, you need to create your xhr object yourself (or use a library which support responseType).
You can take a look at this example which does something similar to your use case.

@dduponchel dduponchel added the bug label Jul 14, 2014
@dduponchel dduponchel added a commit to dduponchel/jszip that referenced this issue Jul 21, 2014
@dduponchel dduponchel Remove usage of Text{En,De}coder.
The goal was to provide a faster utf8 encoding / decoding. This API is only
available on Firefox and it doesn't work well in a Firefox addon context
(the generated Uint8Array and the available Uint8Array class come from
different contexts, leading to bugs and performance issues).

Instead of adding more conditions to (try to) detect if the Uint8Array
is from the same context or not, I think it's better to remove this
optimization for now.

Fix #151.
8e25faa
@Stuk Stuk closed this in #159 Jul 22, 2014
@tarikbenmerar

For information, for those interrested in arraybuffer for jquery ajax, I have developed a plugin here : https://github.com/acigna/jquery-ajax-native

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment