You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Very large data (~100 mb+) sets cause severe performance issues, and may not render at all.
Details
After the data is loaded, the codebook shouldn't need huge amounts of time to summarize the data and render the page. After doing some basic profiling, I'm 99% sure issues are due to inefficient data handling in various places in the code.
The text was updated successfully, but these errors were encountered:
determineType takes a significant amount of that time. Avoid checking every variable value; rather loop through values until a value identifies the variable's type.
coerce to numeric once here, and only here
define value arrays once
figure out how to preserve record index, used when clicking bars; defining an array of objects that is the same size as the input data array is computationally intensive
draw takes ~30% of load time
makeTitle takes a lot of time
lots of garbage collection at the bottom of makeHist
determineType takes a significant amount of that time. Avoid checking every variable value; rather loop through values until a value identifies the variable's type.
Could also recommend the user provide type for each column and avoid this altogether in large data sets. We could just pass in the R column types in datadigest.
determineType takes a significant amount of that time. Avoid checking every variable value; rather loop through values until a value identifies the variable's type.
Could also recommend the user provide type for each column and avoid this altogether in large data sets. We could just pass in the R column types in datadigest.
Having R determine the data types would definitely take some load off the browser.
Summary
Very large data (~100 mb+) sets cause severe performance issues, and may not render at all.
Details
After the data is loaded, the codebook shouldn't need huge amounts of time to summarize the data and render the page. After doing some basic profiling, I'm 99% sure issues are due to inefficient data handling in various places in the code.
The text was updated successfully, but these errors were encountered: