Compile Time & Resources (Formerly: Cannot compile on Raspberry Pi 3)

[Sigafoos]

Both go build and go install fail for streams and vocab on a Raspberry Pi 3 due to running out of memory. (I can get specifics later today if needed; the pi is so upset with me it's no longer responding to ssh)

I understand why: the packages are huge, with good reason. A simple fix would potentially be to offer releases for major/minor versions.

(users can also compile on a better machine with env GOOS=linux GOARCH=arm go build and then copy the binary)

[cjslep]

I am going to make this more general about the compilation time and resources, not just for a Raspberry Pi 3. I am not considering offering binaries at this point as a specific solution, since there's coding solutions available. If someone feels strongly about binary offerings, please open a new issue.

Quoting myself from #48:

Luckily, the compilation time & resource issue may be addressable in major version 0.

The bulk of the compilation time by far should be the fact that the library generates intermediate types to encapsulate that ActivityPub permits a property to be an IRI, value, Object, or array. And in the case of an array, it applies recursively. So under most conditions, a property is stored as this intermediate type.

The way I initially implemented intermediate types was incredibly naiive: every property for every type gets its own intermediate value. This means for N properties and M types, instead of N intermediate types, there's N*M. This should be fixable without any major interface changes.

[cjslep]

af5ed41 eliminates ~230,000 lines of code in the vocab package. It still contains >300,000 however. Please let me know if a Raspberry Pi 3 still cannot compile the vocab package, and I will continue to investigate ways to:

• Reduce code redundancy to permit compilation on memory-constrained environments.
• Improve first-compilation-time (speed).

As mentioned before, I'd like to keep this issue focused with all code-gen optimization discussions. I'll continue using the Raspberry Pi 3 as the metric whether to keep this issue open/closed.

[cjslep]

Note that I talked offline to @Sigafoos and learned that the above change is not sufficient; a Pi will will lock up during compilation. Leaving this issue open, and it may be completed at a later milestone.

[cjslep]

Changed milestones because realistically it won't be easy to solve before the API is reexamined.

[cjslep]

Moving to backlog (no milestone)

[ivucica]

Random thought: generate one package per type and nudge any users to switch to the new API?

This may be easy, as godoc/gowalker don't show many any users, and a very, very cursory look at arbitrarily chosen gen_article.go doesn't indicate that there's much cross-referencing between types anyway (e.g. even a check such as 'this property can only be an ActivityStreams Object' or 'this property can only be an ActivityStreams Link')

[cjslep]

If someone with a Raspberry Pi would like to pull the latest master tools/exp and test compiling the ActivityPub implementation being worked on for go-fed v1, that would be great. The steps are:

cd $GOPATH/src/github.com/go-fed/activity/tools/exp
go build
mkdir tmp
cd tmp
../exp -spec ../spec.json -individual
go build ./...

Once the results of the build are in, cleanup is just:

cd ..
rm -r tmp/
rm ./exp

I am hoping that compilation will succeed on such a constrained environment with the way the code generator separates implementations into smaller packages, but I don't have one handy to test myself.

[Sigafoos]

It does compile successfully! Huzzah.

[cjslep]

YES! Thank you so so much for being patient and waiting the better part of a year to see any movement on this.

I will mark this issue as closed and continue working towards v1 release which will hopefully not regress on this in the meantime. Feel free to reopen again in the future.