decompressFromEncodeURIComponent fails

[dgstpierre]

Using the 1.4.4 version of your javascript file, after I compress a string with LZString.compressToEncodedURIComponent(), and then call LZString.decompressFromEncodedURIComponent() on it, it fails 3 quarters of the way through not being able to find an entry in the dictionary and returns null:
} else {
if (c === dictSize) {
entry = w + w.charAt(0);
} else {
return null;
}
}

Any idea why? I'm testing with Internet Explorer 11. The decompress call is below...

    var test = LZString.decompressFromEncodedURIComponent("N4IgpgliBc4G4EYC0CQBoQCcCOVY4QGpUMBnSGEcxAEhJAGMHKBVAZXSzyzFIAd6pbvTgAXSgHsAiiwASAdwBqAJgCuADgAW62QDMAwgHFCAMQBsAEQA2AEwBymALJg2 + 0QGY7ZgEYBWOLJsLABKDACeAFIAHgDmALycfNzKAAxoaWnKAOzKAD6 + aAhoACwIymXpuWAAdgCG3lZgNrkIRRmVuZ1duZw2fJQAggC2YJgQDLUA9HZg8gD6AJoSmADWucq + ZikAGgjFxSnrxUjKxbmimKpguQDqENXueQjqAJwvuY9nuYcAKpqqAAIIrVqgDlOoAQh3NBfAhoAcAQBRNg - MEpBBmD4pQ41JDsc6Xa4XK75GIALWUMUcVgssikCH0 + mKUhWtXUcCsAAU2CZMCwJBFisETEIADIREx2CwxBhkgDSLGKcVJFKpNLpDKZLLZHM5P0mpCkb3cFj4tQQUTM2AA8j8bmSFikYvJlbldLUrOROGB + rAWp1DocNlsWu51Fl1m6srV3GYXspamANt53Lpii9SuozLpsylfC8UgcGC9dN8y3kg2WA5w4L7QGAhpQbLU4BAbAABYm8WpDPiNAB0DAkjYwfE0MGqqisVjIojAlFhANFYQk1VIAJutTCnAY3Dsy1EmgBA2qNgkcFGnFI4lgjgGiWYsHR6mKvl6jdgXdIPb7YEHw84aoPxACwWzbAE2FEfsAU5CBRkwecMHkUhKE0URRD4aBJkmeRcP7L8fwHIdGwAXwwGxHwAbSKZQAF0MAuGAuxIoA");

[rquadling]

var test = LZString.compressToEncodedURIComponent("This is a string of very long length that seems to be having a lot of suggestions from the Chrome console as to what I will type next. And so I need to add a LOT more text!!!!!!!!!!!!!!!!!!!!!!!! Added!!!!");
LZString.decompressFromEncodedURIComponent(test);

outputs

This is a string of very long length that seems to be having a lot of suggestions from the Chrome console as to what I will type next. And so I need to add a LOT more text!!!!!!!!!!!!!!!!!!!!!!!! Added!!!!

In your example, the spaces around the + seem the issue.

"{"ei":"ev1-1","rqi":"rq1+1","sei":"sev1$1","cc":"US","ri":"resp1","si":"1","vt":"oQUHwV2u8h8HfCG+F6DldNrMeSCt3N6b5vHSURcyJxg=","pi":"20,0,0,272|5,1,41212,0|enabled|11,0,0,0||||||||","dp":"America/New_York|2560X1440|24-24|true|Win32|1899|324||0|Thu Jan 28 13:51:40 EST 2016|300|en-US|true|true|5gZ2gMlDHQ1CC4Qka8vlPSFrUoJ4RFsiLJFNDgcZKU4=|5gZ2gMlDHQ1CC4Qka8vlPT/sQ993Dpa1x6qOTWZY0gw=||false","ep":"|1||0|0|2560|1387|2|f7a3692ae25b3f494186f6f059040c9f|0|0|2|0|2|0|||0","vp":{"em":"david@truesample.com","ph":null,"ste":"51 Lyons Way","ci":"North Andover","st":"MA","pc":"01845","dm":"truesample.com","nm":"David St. Pierre","ws":"http://www.truesample.com"},"dc":[1,2],"tr":true}" is what I get with the spaces removed.

[dgstpierre]

The result you get is correct, thanks for the quick response. I’m trying to pass json strings over a rest api using jsonp, which requires passing the data as a query string parameter. Do I need to do something to the json string or is there a fix to the javascript code to support it?

Thanks,
David

From: Richard Quadling [mailto:notifications@github.com]
Sent: Friday, January 29, 2016 9:04 AM
To: pieroxy/lz-string lz-string@noreply.github.com
Cc: David St. Pierre david@truesample.com
Subject: Re: [lz-string] decompressFromEncodeURIComponent fails (#72)

var test = LZString.compressToEncodedURIComponent("This is a string of very long length that seems to be having a lot of suggestions from the Chrome console as to what I will type next. And so I need to add a LOT more text!!!!!!!!!!!!!!!!!!!!!!!! Added!!!!");

LZString.decompressFromEncodedURIComponent(test);

outputs

This is a string of very long length that seems to be having a lot of suggestions from the Chrome console as to what I will type next. And so I need to add a LOT more text!!!!!!!!!!!!!!!!!!!!!!!! Added!!!!

In your example, the spaces around the + seem the issue.

"{"ei":"ev1-1","rqi":"rq1+1","sei":"sev1$1","cc":"US","ri":"resp1","si":"1","vt":"oQUHwV2u8h8HfCG+F6DldNrMeSCt3N6b5vHSURcyJxg=","pi":"20,0,0,272|5,1,41212,0|enabled|11,0,0,0||||||||","dp":"America/New_York|2560X1440|24-24|true|Win32|1899|324||0|Thu Jan 28 13:51:40 EST 2016|300|en-US|true|true|5gZ2gMlDHQ1CC4Qka8vlPSFrUoJ4RFsiLJFNDgcZKU4=|5gZ2gMlDHQ1CC4Qka8vlPT/sQ993Dpa1x6qOTWZY0gw=||false","ep":"|1||0|0|2560|1387|2|f7a3692ae25b3f494186f6f059040c9f|0|0|2|0|2|0|||0","vp":{"em":"david@truesample.commailto:david@truesample.com","ph":null,"ste":"51 Lyons Way","ci":"North Andover","st":"MA","pc":"01845","dm":"truesample.com","nm":"David St. Pierre","ws":"http://www.truesample.com"},"dc":[1,2],"tr":true}" is what I get with the spaces removed.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/72#issuecomment-176767449.

[rquadling]

If you are passing it as a query string, then the URI escaping of the spaces is getting in the way. It may be better to pass it in the body of the POST rather than in the URL.

[dgstpierre]

I can’t pass it in the body with a jsonp call, it’s not supported (not officially, and it doesn’t work on all browsers if you attempt it). And with this being a cross-domain api call it has to be either jsonp or cors (which does support posts), but cors isn’t supported on older browsers.

If I don’t compress it and simply call encodeURIComponent then it works… but then I don’t get the compression… not sure why spaces are a problem for the compression?

Thanks

From: Richard Quadling [mailto:notifications@github.com]
Sent: Friday, January 29, 2016 9:15 AM
To: pieroxy/lz-string lz-string@noreply.github.com
Cc: David St. Pierre david@truesample.com
Subject: Re: [lz-string] decompressFromEncodeURIComponent fails (#72)

If you are passing it as a query string, then the URI escaping of the spaces is getting in the way. It may be better to pass it in the body of the POST rather than in the URL.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/72#issuecomment-176776035.

[pieroxy]

The URI escaping is just (If I understand the issue correctly) converting '+' to spaces. A quick fix for you might be to convert all spaces to '+' before trying to decompress the String.

As this is a common problem, I might include this behavior in the lib. I'm just waiting for you to confirm that it would solve the issue.

[dgstpierre]

You already perform this in the decompress method right before the _decompress call or am I misunderstanding?

//decompress from an output of compressToEncodedURIComponent
decompressFromEncodedURIComponent:function (input) {
if (input == null) return "";
if (input == "") return null;
input = input.replace(/ /g, "+");
return LZString._decompress(input.length, 32, function(index) { return getBaseValue(keyStrUriSafe, input.charAt(index)); });
},

From: pieroxy [mailto:notifications@github.com]
Sent: Saturday, January 30, 2016 11:07 AM
To: pieroxy/lz-string lz-string@noreply.github.com
Cc: David St. Pierre david@truesample.com
Subject: Re: [lz-string] decompressFromEncodeURIComponent fails (#72)

The URI escaping is just (If I understand the issue correctly) converting '+' to spaces. A quick fix for you might be to convert all spaces to '+' before trying to decompress the String.

As this is a common problem, I might include this behavior in the lib. I'm just waiting for you to confirm that it would solve the issue.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/72#issuecomment-177220613.

[pieroxy]

In the example you posted, you have in effect " + " in some part of your input. This doesn't seem right. The space character is not used in the compressToEncodedURIComponent method. They could appear through URL encoding, but then they would replace the "+" character which would have been converted. In the input you provide to LZString.decompressFromEncodedURIComponentthere are spaces AND + characters. I would bet this is not what came out of compressToEncodedURIComponent, hence the inability for the decompressor to decompress it. Can you try to understand where those spaces come from?

[dgstpierre]

Thanks for your help on this. You are correct, I made a mistake and grabbed the string after it came over to the server. When I retested just on the client side it worked. Thanks again!

From: pieroxy [mailto:notifications@github.com]
Sent: Sunday, January 31, 2016 3:24 AM
To: pieroxy/lz-string lz-string@noreply.github.com
Cc: David St. Pierre david@truesample.com
Subject: Re: [lz-string] decompressFromEncodeURIComponent fails (#72)

In the example you posted, you have in effect " + " in some part of your input. This doesn't seem right. The space character is not used in the compressToEncodedURIComponent method. They could appear through URL encoding, but then they would replace the "+" character which would have been converted. In the input you provide to LZString.decompressFromEncodedURIComponentthere are spaces AND + characters. I would bet this is not what came out of compressToEncodedURIComponent, hence the inability for the decompressor to decompress it. Can you try to understand where those spaces come from?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/72#issuecomment-177430127.