OutOfMemoryException when building holy-lambda 0.2.3

[FieryCod]

Related build:
https://cljdoc.org/builds/46616

Discussion from Slack:

lread 6 hours ago
So v0.2.3 failed, but v0.2.2 worked. I just hit the rebuild button on v0.2.2 and it has now failed

lread 6 hours ago
Seems to be failing on the clone? I can take a look at this sometime later today if nobody else beats me to it.

lread 2 hours ago
Ok, I’ll try to reproduce locally on my dev box shortly.

lread 2 hours ago
It ingested fine on my dev box, but I have 32gb of RAM. @martinklepsch If I understand cljdoc infrastructure correctly, it only has 2gb of RAM?

lread 1 hour ago
I guess something interesting about the holy-lambda repo is that it has some larger files. I’ll restrict RAM on my test locally and see if I can trigger the same error. (edited)

lread 1 hour ago
Ya, I can trigger a similar error if I run cljdoc ingest with -J-Xmx128m.

lread 1 hour ago
I’ll have a look-see at what cljdoc/jgit are up to.

Karol Wójcik 1 hour ago
Hmm maybe it looks in modules folder?

lread 1 hour ago
Yeah, it clones the entire repo. And jgit can, depending on repo size, need RAM, it seems.

lread 1 hour ago
Might be easiest/reasonable just to bump up RAM of cljdoc server? I dunno who pays for that though. @martinklepsch?

lread 1 hour ago
Or… we could switch to spawning out to a git executable instead of using jgit, and see if that helps. Clojure tools deps has switched from jgit to git executable.

lread 1 hour ago
Or… we could think more about other options. :simple_smile:

[RickMoynihan]

Just a note to say that I also experienced this OutOfMemoryException here:

https://cljdoc.org/builds/52648

It was because my repo had an unmerged branch which contained two large files (one 25mb, another 75mb). The .git directory itself was however just 11Mb as blobs are compressed.

Fortunately the unmerged branch was just an experimental perf-test, that contained some large fixture data; so I was able to migrate the files in it to git lfs and rewrite the history. Force pushing the affected branches, then meant the .git directory of a fresh clone was just 300kb, and cljdoc could generate the docs without error.

[lread]

@FieryCod, the recent RAM bump on the cldjoc server means your build no longer suffers out of memory errors. I retried the specific versions you reported and had success:
https://cljdoc.org/d/io.github.FieryCod/holy-lambda/0.2.2/doc/readme
https://cljdoc.org/d/io.github.FieryCod/holy-lambda/0.2.3/doc/readme

I think we can close this specific issue. Please re-open if you disagree.