New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization (or preprocessing) strategies for fromJS #514

Closed
psaia opened this Issue Jun 20, 2015 · 2 comments

Comments

Projects
None yet
2 participants
@psaia

psaia commented Jun 20, 2015

We're dealing with a huge JSON object which gets delivered by an ajax request - around 4mb with nested objects and arrays (one has a length of 10,000+). Upon receiving, we're running it through Immutable.fromJS which takes about 7.8 seconds. This 7 seconds is obviously a huge bottleneck for the browser.

A feature for preprocessing the data on the server before it hits the client's fromJS would be nice. Something like a curry which preps the object for immutability. Or is there another recommended strategy for handling this sort of thing?

Update:

I've isolated the issue into a jsfiddle and get much better performance without any other javascript executing. Though, it does still take about 1.5 seconds of grinding.

http://jsfiddle.net/jL596cny/1/

@leebyron

This comment has been minimized.

Show comment
Hide comment
@leebyron

leebyron Jun 22, 2015

Collaborator

Wow, that's huge data.

Have you tried Transit.js? It can help with moving rich data from server to client. I would be curious if that is at all helpful.

http://swannodette.github.io/2014/07/30/hijacking-json/ Explains how to set this up.

I'm curious to know if that's helpful on the initial data construction

Collaborator

leebyron commented Jun 22, 2015

Wow, that's huge data.

Have you tried Transit.js? It can help with moving rich data from server to client. I would be curious if that is at all helpful.

http://swannodette.github.io/2014/07/30/hijacking-json/ Explains how to set this up.

I'm curious to know if that's helpful on the initial data construction

@psaia

This comment has been minimized.

Show comment
Hide comment
@psaia

psaia Jun 22, 2015

That did work! Thanks for the speedy reply as well. There is definitely a noticeable difference in performance. It's worth noting that I had to change the example given in that article slightly to use List instead of Vector.

import { List, Map } from 'immutable';
import transit from 'transit-js';

let rdr = transit.reader("json", {
  arrayBuilder: {
    init: function(node) { return List().asMutable(); },
    add: function(ret, val, node) { return ret.push(val); },
    finalize: function(ret, node) { return ret.asImmutable(); },
    fromArray: function(arr, node) { return List(arr); }
  },
  mapBuilder: {
    init: function(node) { return Map().asMutable(); },
    add: function(ret, key, val, node) { return ret.set(key, val);  },
    finalize: function(ret, node) { return ret.asImmutable(); }
  }
});

Having done this, I actually decided to take another approach and not immediately transform all the data upon receiving it (especially that huge array). It just wasn't necessary. Feel free to close!

psaia commented Jun 22, 2015

That did work! Thanks for the speedy reply as well. There is definitely a noticeable difference in performance. It's worth noting that I had to change the example given in that article slightly to use List instead of Vector.

import { List, Map } from 'immutable';
import transit from 'transit-js';

let rdr = transit.reader("json", {
  arrayBuilder: {
    init: function(node) { return List().asMutable(); },
    add: function(ret, val, node) { return ret.push(val); },
    finalize: function(ret, node) { return ret.asImmutable(); },
    fromArray: function(arr, node) { return List(arr); }
  },
  mapBuilder: {
    init: function(node) { return Map().asMutable(); },
    add: function(ret, key, val, node) { return ret.set(key, val);  },
    finalize: function(ret, node) { return ret.asImmutable(); }
  }
});

Having done this, I actually decided to take another approach and not immediately transform all the data upon receiving it (especially that huge array). It just wasn't necessary. Feel free to close!

@leebyron leebyron closed this Jun 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment