Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auxiliary function for load file #17

Closed
arademaker opened this issue Apr 30, 2015 · 8 comments
Closed

auxiliary function for load file #17

arademaker opened this issue Apr 30, 2015 · 8 comments

Comments

@arademaker
Copy link

Does this library has any auxiliary function to read from a file?

@rudolph-miller
Copy link
Owner

Currently, no, it can only handle strings.
I plan to support pathnames and steams.

(parse (alexandria:read-file-into-string "pathname"))

@arademaker
Copy link
Author

With a 150MB file:

Heap exhausted (no more space for allocation).
395870208 bytes available, 536870944 requested.

PROCEED WITH CAUTION.
[Condition of type SB-KERNEL::HEAP-EXHAUSTED-ERROR]

Restarts:
0: [RETRY] Retry SLIME REPL evaluation request.
1: [*ABORT] Return to SLIME's top level.
2: [ABORT] abort thread (#<THREAD "repl-thread" RUNNING {1004BC8003}>)

(jonathan:parse (alexandria:read-file-into-string "onepetro.json"))

The problem is the reading of the text file, not the parse!

@arademaker
Copy link
Author

BTW, mlisp (Allegro Modern Lisp):

cl-user> (time (with-open-file (in #P"onepetro.json")
         (length (st-json:read-json in))))
; cpu time (non-gc) 12.014951 sec user, 0.453510 sec system
; cpu time (gc)     6.919787 sec user, 0.259170 sec system
; cpu time (total)  18.934738 sec user, 0.712680 sec system
; cpu time (thread) 11.999369 sec user, 0.439226 sec system
; real time  20.168150 sec (97.42%)
; space allocation:
;  12,748,321 cons cells, 646,417,616 other bytes, 0 static bytes
; Page Faults: major: 489 (gc: 113026), minor: 227839 (gc: 113026)
166942

In SBCL it killed the SBCL proc.

@rudolph-miller
Copy link
Owner

Hmm, it can be solved by using streams, and it can be time to add support of streams.

Before then, you can avoid the problem with using like http://www.sbcl.org/manual/#Runtime-Options (SBCL).

@arademaker
Copy link
Author

Jonathan didn't work in mlisp! Thanks, I will try.

@rudolph-miller
Copy link
Owner

Thank you for reporting of your use case : )

@arademaker
Copy link
Author

Not sure which parameter to try. With the command below I still got error:

sbcl --control-stack-size 10 --dynamic-space-size 5000

@rudolph-miller
Copy link
Owner

--dynastic-space-sizeis the correct option.
Didn't it make some change to the error message?
Like ~ bytes available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants