Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/pkgsite: local setup - tracking issue #40371

Open
1 task done
julieqiu opened this issue Jul 23, 2020 · 9 comments
Open
1 task done

x/pkgsite: local setup - tracking issue #40371

julieqiu opened this issue Jul 23, 2020 · 9 comments

Comments

@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jul 23, 2020

Today, users can run pkgsite locally without setting up a local database by running:

go run cmd/frontend/main.go -direct_proxy -proxy_url=<your proxy URL>

See flags in cmd/frontend/main.go, and doc/frontend.md for docs.

Doing so allows users to view the package documentation, overview, imports, and licenses tabs. Search and other tabs on the package page are not supported in direct proxy mode.

However, there are cases when a user might want to run pkgsite on their local machine, and be able to view a private repository that is not available via a proxy.

Issues related to this topic:

  • #40159: run locally to view documentation of a pending CL

This is an umbrella issue to discuss what features are needed for this to happen, so please comment on your use cases below. For discussions about hosting a private instance of pkgsite, including your own internal proxy, see #39827.

@gopherbot gopherbot added this to the Unreleased milestone Jul 23, 2020
@julieqiu julieqiu changed the title x/pkgsite: pkgsite local setup - tracking issue x/pkgsite: local setup - tracking issue Jul 23, 2020
@achille-roussel
Copy link

@achille-roussel achille-roussel commented Jul 29, 2020

Hello @julieqiu

If I can add color to the pain points of running locally, having to start a proxy isn't too much trouble, but the constraint of requiring https is a bigger barrier:

$ go run ./cmd/frontend -direct_proxy -proxy_url http://localhost:8888
config: {
    "AuthHeader": "",
    "ProxyURL": "https://proxy.golang.org",
    "IndexURL": "https://index.golang.org/index",
    "Port": "",
    "DebugPort": "",
    "ProjectID": "",
    "ServiceID": "",
    "VersionID": "",
    "ZoneID": "",
    "InstanceID": "",
    "LocationID": "us-central1",
    "QueueService": "",
    "GaeEnv": "",
    "GoogleTagManagerID": "",
    "AppMonitoredResource": {
        "type": "gae_app",
        "labels": {
            "module_id": "",
            "project_id": "",
            "version_id": "",
            "zone": ""
        }
    },
    "FallbackVersionLabel": "20200729t023455",
    "DBSecret": "",
    "DBUser": "postgres",
    "DBHost": "localhost",
    "DBPort": "5432",
    "DBName": "discovery-db",
    "DBSecondaryHost": "",
    "RedisCacheHost": "",
    "RedisCachePort": "6379",
    "RedisHAHost": "",
    "RedisHAPort": "6379",
    "UseProfiler": false,
    "Quota": {
        "QPS": 10,
        "Burst": 20,
        "MaxEntries": 1000,
        "RecordOnly": true,
        "AuthHeader": "",
        "AuthValues": null
    },
    "TeeproxyAuthValue": "",
    "TeeproxyForwardedHosts": null
}
2020/07/29 02:34:55 Error: proxy.New("http://localhost:8888"): scheme must be https (got http)
exit status 1

What do you think about relaxing this constraint?

@gopherbot
Copy link

@gopherbot gopherbot commented Jul 29, 2020

Change https://golang.org/cl/245639 mentions this issue: internal/proxy: remove URL validation in New

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jul 29, 2020
Rather than validating the URL in proxy.New, assume that the URL that is
passed in is valid. This allows users to connect to a proxy running
locally in direct proxy mode.

For golang/go#40371

Change-Id: Id51cb27148987e58d214cef1c805b26b5138a6de
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/245639
Run-TryBot: Julie Qiu <julie@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@julieqiu
Copy link
Contributor Author

@julieqiu julieqiu commented Jul 29, 2020

@achille-roussel - done! Thanks for the feedback.

@MicahParks
Copy link

@MicahParks MicahParks commented Aug 1, 2020

To run pkgsite in a docker container, it'd be good to customize what networking interface the program binds to.

Executables in docker containers can more easily publish their services on the host's networking interface if bound to 0.0.0.0. It would be possible to allow this configuration via a flag by changing these lines:

https://github.com/golang/pkgsite/blob/ef183200d192b20102bb1246a86ae11de2d0738e/cmd/frontend/main.go#L170
https://github.com/golang/pkgsite/blob/ef183200d192b20102bb1246a86ae11de2d0738e/cmd/frontend/main.go#L148

to use a variable that is assigned by a flag instead of the "localhost" string.

@achille-roussel
Copy link

@achille-roussel achille-roussel commented Aug 3, 2020

We had to solve for this as well in the fork we maintain at https://github.com/segmentio/pkgsite:

diff --git a/cmd/frontend/main.go b/cmd/frontend/main.go
index 5773d8e..e81022b 100644
--- a/cmd/frontend/main.go
+++ b/cmd/frontend/main.go
@@ -42,6 +42,7 @@ var (
                "for direct proxy mode and frontend fetches")
        directProxy = flag.Bool("direct_proxy", false, "if set to true, uses the module proxy referred to by this URL "+
                "as a direct backend, bypassing the database")
+       httpAddr = flag.String("http", "localhost:8080", "address to listen for incoming requests on")
 )

 func main() {
@@ -167,7 +168,7 @@ func main() {
                middleware.Timeout(54*time.Second),
                middleware.Experiment(experimenter),
        )
-       addr := cfg.HostAddr("localhost:8080")
+       addr := cfg.HostAddr(*httpAddr)
        log.Infof(ctx, "Listening on addr %s", addr)
        log.Fatal(ctx, http.ListenAndServe(addr, mw(router)))
 }

Taking a closer look at the way the configuration is setup, it appears that if the PORT environment variable is set the server will be listening on all network interfaces (as the address will be :$PORT), so maybe that change was unnecessary:

// HostAddr returns the network on which to serve the primary HTTP service.
func (c *Config) HostAddr(dflt string) string {
	if c.Port != "" {
		return fmt.Sprintf(":%s", c.Port)
	}
	return dflt
}

@julieqiu julieqiu removed this from the Unreleased milestone Aug 27, 2020
@julieqiu julieqiu added this to the pkgsite/unreleased milestone Aug 27, 2020
@julieqiu julieqiu removed the go.dev label Sep 19, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 8, 2020

Change https://golang.org/cl/260779 mentions this issue: cmd/frontend: add -local flag

gopherbot pushed a commit to golang/pkgsite that referenced this issue Oct 16, 2020
Create -local and -gopath_mode flags to enable using a local datasource
and loading local modules to memory.

Updates golang/go#40371
Fixes golang/go#40159

Change-Id: I36941adde9c6b186d95b5792051854ac3d1a2ac8
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/260779
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Trust: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
@julieqiu julieqiu removed the go.dev label Oct 16, 2020
gopherbot pushed a commit to golang/pkgsite that referenced this issue Feb 5, 2021
Functionality for running the pkgsite frontend locally is moved from
cmd/frontend to cmd/pkgsite, since cmd/frontend is currently overloaded
with flag options and running locally does not need all the dependencies
for running cmd/frontend.

Additional functionality will be added to cmd/pkgsite in future CLs.

For golang/go#40371

Change-Id: I4230aa9539c94e01a68eda33cc6492ae377debff
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/290134
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Trust: Julie Qiu <julie@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Feb 5, 2021

Change https://golang.org/cl/290134 mentions this issue: cmd/pkgsite: add command

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 5, 2021

Change https://golang.org/cl/290135 mentions this issue: cmd/pkgsite: add -http flag

gopherbot pushed a commit to golang/pkgsite that referenced this issue Feb 5, 2021
A -http flag is added which allows the user to specify which HTTP addr
to listen in on.

For golang/go#40371

Change-Id: Ibfe32281e9a821444df5e538fd7057f39318c546
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/290135
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Trust: Julie Qiu <julie@golang.org>
@akavel
Copy link
Contributor

@akavel akavel commented Sep 28, 2021

I was redirected here from an issue requesting a flag in godoc for showing internal packages (for use during internal/company development purposes, to make it possible to easily browse internal APIs) - #12092 (comment). Thus (per the request for use cases in the main comment above), that's a use case me and several other people expressed interest in. If godoc is seen as "in maintenance mode" (which was quite a surprise to me), it would be nice if the plan for the new replacement tool was outlined here (?) more clearly, or something. Currently, the situation & status may appear unclear to community members (like me).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants