Support for running athens behind a company proxy

[Dominik-Robert]

Hello, I'm trying to run the athens proxy in my company but it failed with a 404 error. I think he can't get the resource from the VCS from the internet. One more problem is that we have a proxy certificate, so the certificate is invalid.

I dont know if it is already supportet but it would be great if you can implement a support for own ssl-certificates and an option to run this behind a company proxy.

[arschles]

@Dominik-Robert Athens gets code from the VCS on-demand, so if it can't reach the internet, you'll need to pre-load code yourself. See this document for how to do that with disk storage.

Regarding custom TLS certificates, Athens runs unsecured by default, but you can specify your own with these configuration variables

athens/config.dev.toml

Lines 121 to 125 in 4530a58

	# Certificate and key to make athens serve using https instead of plain text http.
	# Set both to enable.
	# Env override: ATHENS_TLSCERT_FILE, ATHENS_TLSKEY_FILE
	#TLSCertFile = "server.cer"
	#TLSKeyFile = "server.key"

Can you let me know if that helps?

[Dominik-Robert]

Thank you for your answer. I get it running with preload code, but is it possible that athens fetch the code from VCS from the internet behind a company proxy?

[arschles]

@Dominik-Robert it depends on the proxy, I suppose. Athens doesn't have built-in support for any proxy protocols. Is that what you'd need?

[Dominik-Robert]

Yes I need that. Is it possible to support that?

[arschles]

@Dominik-Robert behind the scenes, Athens uses the go CLI to do its interaction with the internet at-large, so we wouldn't need to make big changes to the Athens codebase to do this. What protocol(s) would you need to support?

[marwan-at-work]

Just an FYI: if you can git clone a repo from within the VM that Athens is running on (whether it's exposed to the internet or within your company VPN), then Athens should be able to reach it as well. As long as you configure git correctly from within that VM and/or configure the .netrc file in Athens as well: https://github.com/gomods/athens/blob/master/config.dev.toml#L182

[Dominik-Robert]

Oh @marwan-at-work that is a good point I will look at.

@arschles I don´t know exactly I want to use Athen normally behind a proxy. So I would think http and https for the VCS-Systems would be enough or am I missing something?

[arschles]

@Dominik-Robert yes, http/https for VCS systems definitely would be enough. I thought that we were talking about a corporate auth proxy like NTLM or something similar. Sorry about that. Let us know if you can do the git clone that @marwan-at-work mentioned.

[Dominik-Robert]

It so far as I can download the first 5 packages. I don't know why it is failing for the other packages. It says it is not finding them.

My go.mod file

module github.com/dominik-robert/license-manager-api

go 1.13

require (
        github.com/99designs/gqlgen v0.10.2
        github.com/gorilla/mux v1.6.1
        github.com/lib/pq v1.2.0
        github.com/rs/cors v1.6.0
        github.com/spf13/viper v1.4.0
        github.com/vektah/gqlparser v1.2.0
)

Athens-Output

INFO[6:19AM]: Exporter not specified. Traces won't be exported	
2020-03-03 06:19:49.949987 I | Starting application at port :3000
DEBUG[6:21AM]: saving github.com/gorilla/mux@v1.6.1 to storage...	http-method=GET http-path=/github.com/gorilla/mux/@v/v1.6.1.mod 
DEBUG[6:21AM]: saving github.com/lib/pq@v1.2.0 to storage...	http-method=GET http-path=/github.com/lib/pq/@v/v1.2.0.mod 
DEBUG[6:21AM]: saving github.com/spf13/viper@v1.4.0 to storage...	http-method=GET http-path=/github.com/spf13/viper/@v/v1.4.0.mod 
DEBUG[6:21AM]: saving github.com/vektah/gqlparser@v1.2.0 to storage...	http-method=GET http-path=/github.com/vektah/gqlparser/@v/v1.2.0.mod 
DEBUG[6:21AM]: saving github.com/99designs/gqlgen@v0.10.2 to storage...	http-method=GET http-path=/github.com/99designs/gqlgen/@v/v0.10.2.mod 
DEBUG[6:21AM]: saving github.com/rs/cors@v1.6.0 to storage...	http-method=GET http-path=/github.com/rs/cors/@v/v1.6.0.mod 
INFO[6:21AM]: github.com/rs/cors@v1.6.0: invalid version: unknown revision v1.6.0	http-method=GET http-path=/github.com/rs/cors/@v/v1.6.0.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/rs/cors/@v/v1.6.0.mod http-status=404 
INFO[6:21AM]: github.com/gorilla/mux@v1.6.1: invalid version: unknown revision v1.6.1	http-method=GET http-path=/github.com/gorilla/mux/@v/v1.6.1.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/gorilla/mux/@v/v1.6.1.mod http-status=404 
INFO[6:21AM]: github.com/spf13/viper@v1.4.0: invalid version: unknown revision v1.4.0	http-method=GET http-path=/github.com/spf13/viper/@v/v1.4.0.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/spf13/viper/@v/v1.4.0.mod http-status=404 
INFO[6:21AM]: github.com/vektah/gqlparser@v1.2.0: invalid version: unknown revision v1.2.0	http-method=GET http-path=/github.com/vektah/gqlparser/@v/v1.2.0.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/vektah/gqlparser/@v/v1.2.0.mod http-status=404 
INFO[6:21AM]: github.com/lib/pq@v1.2.0: invalid version: unknown revision v1.2.0	http-method=GET http-path=/github.com/lib/pq/@v/v1.2.0.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/lib/pq/@v/v1.2.0.mod http-status=404 
INFO[6:21AM]: github.com/99designs/gqlgen@v0.10.2: invalid version: unknown revision v0.10.2	http-method=GET http-path=/github.com/99designs/gqlgen/@v/v0.10.2.mod kind=Not Found module= operation=download.VersionModuleHandler ops=[download.VersionModuleHandler pool.GoMod protocol.GoMod protocol.processDownload stash.Pool stasher.Stash stasher.fetchModule goGetFetcher.Fetch module.downloadModule] version= 
INFO[6:21AM]: incoming request	http-method=GET http-path=/github.com/99designs/gqlgen/@v/v0.10.2.mod http-status=404 

*EDIT:
I misread the logs, athen doesn't find the packages. But why? I started the proxy with environment-variables and git config for the ip of the company proxy in a docker container.

When I change the entrypoint to /bin/sh and go inside the container with a shell to set the proxies in git I get a 500 error.

When I unset GOPROXY and run go mod download all packages are downloaded

[marwan-at-work]

@Dominik-Robert I wasn't able to reproduce the issue:

~ docker run --rm -p 3000:3000 gomods/athens:v0.7.2
2020-03-03 18:29:18.859752 I | Starting application at port :3000
DEBUG[6:29PM]: saving github.com/rs/cors@v1.6.0 to storage...	http-method=GET http-path=/github.com/rs/cors/@v/v1.6.0.mod
INFO[6:29PM]: incoming request	http-method=GET http-path=/github.com/rs/cors/@v/v1.6.0.mod http-status=200

Logs are a result of ~ curl localhost:3000/github.com/rs/cors/@v/v1.6.0.mod

I have noticed however, that there has been intermittent failures with git so what I'd do is to just try again and see if that works. Kind of a bad answer, but if you can find a way to reproduce this please include the reproduction steps so we can dig into why it might be happening :)

[arschles]

@Dominik-Robert I'm going to close this issue. If you still have this problem, please let us know.