pkg/mod storage backend

[raulk]

I'd like to start Athens and link its storage backend with an existing $GOPATH/pkg/mod repository on the host. That way it would be able to reuse any modules that have been downloaded on the machine.

One scenario where this is useful is to speed up Golang Docker builds by starting up a proxy on the host to share modules downloaded there with the build container.

[arschles]

@raulk I think this is a really cool idea! There is some related work in #1414. Do you think it would it make sense to modify that PR to fit your needs before we merge it?

[raulk]

@arschles Yeah, this virtual storage backend would basically automate that process, feeding from pkg/mod, calling pacmod when a module is requested, and serving those assets via the proxy. Maintaining a cache would be handy.

[raulk]

For the record, we're currently using goproxyio/goproxy for our Docker builds because it supports this use case. If you're curious, here's a link to how we do it:

https://github.com/ipfs/testground/blob/ad5608acd71d01702b1462e0854ec9ad527dc6d5/pkg/build/golang/docker.go#L327-L352

Thanks!

[arschles]

I see what you mean. If we had a different backend, or a "v2" backend of the current filesystem backend that followed the same format as what's in pkg/mod, we probably wouldn't need to involve pacmod at all, right? It would be nice to have one less piece.

Either way, I am totally ok with having such a backend.

Regarding #1414, I am gonna merge that one and I'm happy to modify it later on depending on what direction this goes.

If you'd like to get started on it, that's totally cool with me, otherwise I can. No pressure either way, just let me know.

[arschles]

@raulk do you still need this feature? I'm happy to go and figure out how to implement it.

[arschles]

@raulk now that Athens supports external storage drivers, I am going to close this since a simple external server could be built to serve the local cache