Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON import not working #70

Closed
jmvanel opened this issue Mar 12, 2014 · 6 comments
Closed

JSON import not working #70

jmvanel opened this issue Mar 12, 2014 · 6 comments

Comments

@jmvanel
Copy link

jmvanel commented Mar 12, 2014

I'm using Karma for the first time,
and I'm assuming that it's OK to import a JSON with no model pre-existing.

I tried from the web UI (gihub clone) this JSON URL:
http://data.iledefrance.fr/api/datasets/1.0/search?rows=100
I raised the memory to 3Gb ; but it's real slow, and it takes 100% of both CPU's,
and it ends up in "OutOfMemoryError: GC overhead limit exceeded" anyway.

But the data, after download with wget, is not huge:
% wc data.iledefrance.fr_search_rows_100.json
0 19320 206512 data.iledefrance.fr_search_rows_100.json

Then I retarted, and tried this smaller JSON URL:
http://data.iledefrance.fr/api/datasets/1.0/search?rows=20

Now no CPU used, no special message on the console, and running forever in the browser.

@szeke
Copy link
Member

szeke commented Mar 12, 2014

Hi, we are looking at your problem. I was able to load the large file in the development version of Karma on a Mac and Chrome. Did you change the default memory allocation for maven:
export MAVEN_OPTS=-Xmx4096m

The default is very small.
github-70-large-screenshot

@szeke
Copy link
Member

szeke commented Mar 12, 2014

We found the problem. We can load the JSON as a file after we download it, but somehow doing it directly from the service is not working. We'll look into it, but we have a few urgent tasks to complete before we get to this one. Hopefully you can still work by downloading the files first. Not ideal, but a temporary workaround.

@jmvanel
Copy link
Author

jmvanel commented Mar 12, 2014

Thanks

Indeed it works from the file.
And not with memory as large as 5Gb.

Also I understood from the doc, although it's not stressed, that one must first import a small file, then m

@jmvanel
Copy link
Author

jmvanel commented Mar 12, 2014

Thanks

Indeed it works from the file.
And not with memory as large as 5Gb.

1 similar comment
@jmvanel
Copy link
Author

jmvanel commented Mar 12, 2014

Thanks

Indeed it works from the file.
And not with memory as large as 5Gb.

@jmvanel jmvanel closed this as completed Mar 12, 2014
@jmvanel
Copy link
Author

jmvanel commented Mar 12, 2014

Thanks

Indeed it works from the file.
And not with memory as large as 5Gb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants