Use Athens as repository in private (offline) network

[RonAmihai]

I'm working in an offline environment which consists of private (offline) network.

I can bring packages (3rd party) which I've downloaded in an online machine (moving them to the offline network over USB media storage), but the machines in the offline network don't have direct access to the worldwide internet.

In Node/JavaScript project, I'm using [Verdaccio] (https://github.com/verdaccio/verdaccio) as private repository running in my private offline network.
In an online machine I'm downloading packages with it's dependencies using the npm-offline-packager tool, then moving the downloaded tarball files to my private offline network. Afterwards I'm using npm-offline-packager or npm-publish to publish the packages to my Verdaccio.

In Python projects, I don't have proxy/repository solution (like JFrog Artifactory), so what I'm doing is to download packages+dependencies online using the "pip download", than moving the files to our offline private network, than installing the files locally using "pip install --no-index --find-links <path_to_packages_folder>".

I'm looking for applying one of the above approaches with Go project.
In this case - Athens should act similarly to Verdaccio, and I need a tool to download Go packages with all of it's dependencies online, then moving files to the offline network and publish them to Athens in my offline private network.

Is this approach possible? Does there any tool that can download go packages+dependencies (even from a list of packages) online and than publish them to Athens in an offline environment?

[arschles]

@RonAmihai can you check out our documentation on pre-filling the disk cache and let me know if that will work for you?

[RonAmihai]

@RonAmihai can you check out our documentation on pre-filling the disk cache and let me know if that will work for you?

Thank you for this very helpful information.
I guess I've skipped it while reading the documentation.

I'll give it a try in the next few days and share the results here

[arschles]

@RonAmihai great. I'm going to leave this issue open for now 😄

[RonAmihai]

@RonAmihai can you check out our documentation on pre-filling the disk cache and let me know if that will work for you?

Are there convenient alternatives to pacmod for this method?

Pacmod doesn't seem to support packaging with all dependencies.
I want to PR this feature soon to pacmod, but till than I'm wondering if I can bring package will all of it's dependencies to Athens in other way.

(By the way, maybe Athens should have support for this case of uploading/deployment archive with many packages. I'll be happy to help in this section)

[avivdolev]

Hi,
My situation is much like @RonAmihai 's
I thought I found a better alternative to pacmod: Run an Athens instance on my online machine, with storage configuration set to "disk", then copy the whole directory (with the correct folder structures, .mod and .info files).
However, I ran into this problem (v0.7.0):
localhost:3000/modulename/@v/list just returns 404, because Athens tries to run go list -m, and github.com is unavailable...
This prevents me from using a regular go get without specifying an explicit version.
I did manage to get an explicit version, with go env -w GOSUMDB=off on the client.

So, my questions:

• Do I miss any offline-friendly configuration that can convince Athens to serve the list of versions it already has? Explicit versions on go get are very uncomfortable 😐
• Any chance that Athens will also download the checksum data, and provide the sumdb itself? (sorry if its a dumb question, not entirely sure how the checksum works 😳)

Thank you for a great project, we sure want to use it in our offline dev environment!

[praseodym]

• Do I miss any offline-friendly configuration that can convince Athens to serve the list of versions it already has? Explicit versions on go get are very uncomfortable neutral_face

There's a discussion going on about this in #1532 🙂

• Any chance that Athens will also download the checksum data, and provide the sumdb itself? (sorry if its a dumb question, not entirely sure how the checksum works flushed)

Currently GOSUMDB=off is the best way to go. The idea behind having a checksum server is that there's a third party that ensures the proxy is behaving as expected. Having Athens serve its own checksums defeats that purpose.

[arschles]

Adding to @praseodym, in an untrusted environment that deals with private code, it would be ideal (in my opinion) if Athens could serve checksums for that private code. I don't know if there is such a thing as an untrusted environment that deals with private code, though.

More importantly, it is impossible to have sum.golang.org to serve one part of the checksum tree and have Athens serve the other. The next best thing, then, is to set up your sum DB exclude list on Athens, and GONOSUMDB on your development machine.

I think the docs at https://docs.gomods.io/configuration/sumdb/ describe what I'm talking about pretty clearly. That could be my bias though! So please let me know if something remains unclear by filing issues.

[arschles]

@RonAmihai we threw a lot of information in this issue, so I'm wondering if your original question got drowned out. Can you let me know if things are still not working for you, and if you still have questions?