Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: revamp the security model used by gomote #47521

Open
15 of 16 tasks
cagedmantis opened this issue Aug 3, 2021 · 42 comments
Open
15 of 16 tasks

x/build: revamp the security model used by gomote #47521

cagedmantis opened this issue Aug 3, 2021 · 42 comments
Assignees
Labels
Builders NeedsFix
Milestone

Comments

@cagedmantis
Copy link
Contributor

cagedmantis commented Aug 3, 2021

This is a tracking issue for the redesign of the authentication for the gomote application. The gomote application will have the authentication method changed as well as the addition of an authorization layer. This work will be broken down into many steps which will be added here as they are fully decided upon:

  • Collect metrics on gomote usage #48579
  • Update the gomote client to notify users of an upcoming change to authentication #48726
  • Determine if a new DNS address is needed for gomote testing #48727
  • Enable IAP #48728
  • Configure HTTPS Load Balancers #49191
  • Add Authentication #48729
  • Add Authorization #48730
  • Add proxy #48733
  • Add gomote manager #48735
  • Implement the gomote API #48742
  • Add new gomote client commands #48737
  • Add authentication to gomote client #48739
  • Add certificate authentication to SSH Server #52594
  • Create IAM groups #48741
  • List instances on farmer.golang.org
  • Ask users to request accounts #48725

@golang/release

@cagedmantis cagedmantis added Builders NeedsFix labels Aug 3, 2021
@cagedmantis cagedmantis added this to the Unplanned milestone Aug 3, 2021
@cagedmantis cagedmantis self-assigned this Aug 3, 2021
@heschi heschi added this to In Progress in Go Release Team Aug 3, 2021
@gopherbot
Copy link

gopherbot commented Sep 28, 2021

Change https://golang.org/cl/352809 mentions this issue: cmd/coordinator: add metrics for gomote usage

gopherbot pushed a commit to golang/build that referenced this issue Sep 29, 2021
This change introduces some metrics collection around gomote usage. It
records:
- gomote creates and the associated builder types.
- gomote ssh and the success of the call.
- gomote RDP.

Updates golang/go#47521
Fixes golang/go#48579

Change-Id: I5dfa04862254de0ceae747d0328918480d11db7c
Reviewed-on: https://go-review.googlesource.com/c/build/+/352809
Trust: Carlos Amedee <carlos@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

gopherbot commented Oct 18, 2021

Change https://golang.org/cl/356589 mentions this issue: internal/gomote/protos: add a skeleton for a gomote api

gopherbot pushed a commit to golang/build that referenced this issue Oct 21, 2021
This change adds a skeleton for a new GRPC gomote API. This work
is part of a reworking of the security model around gomotes.

Updates golang/go#47521
Updates golang/go#48742

Change-Id: I4b0ae84bf58fe6e999fb34c17e670a6f638055f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/356589
Trust: Carlos Amedee <carlos@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@gopherbot
Copy link

gopherbot commented Oct 26, 2021

Change https://golang.org/cl/358915 mentions this issue: internal/access: add access package

@gopherbot
Copy link

gopherbot commented Nov 3, 2021

Change https://golang.org/cl/361098 mentions this issue: internal/gomote, cmd/coordinator: add GRPC gomote server

gopherbot pushed a commit to golang/build that referenced this issue Nov 8, 2021
This change adds an access package which is intented to contain
functions which will handle Identity Aware Proxy authentication. It
may be extended to include authorization logic in the future.

Fixes golang/go#48729
Updates golang/go#47521

Change-Id: I68cd90c3e83066763e3194fcb58e324c3630f811
Reviewed-on: https://go-review.googlesource.com/c/build/+/358915
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
@gopherbot
Copy link

gopherbot commented Nov 19, 2021

Change https://golang.org/cl/365735 mentions this issue: deploy: add GRPC servers to build.golang.org

gopherbot pushed a commit to golang/build that referenced this issue Nov 23, 2021
This change:
- Adds a simple GRPC gomote server.
- Updates the documentation for the audiance required for IAP authentication.
- Adds a field for the backend service id in the build enviornment package.
- Creates middleware for the GRPC server use in the existing HTTP servers.

Updates golang/go#47521
Updates golang/go#48742

Change-Id: I2a56e39b96bf1b429f807f79c58aee3f72a45a33
Reviewed-on: https://go-review.googlesource.com/c/build/+/361098
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 23, 2021
This change mounts the gomote and coordinator servers in the proper
locations.

Updates golang/go#47521
Updates golang/go#49191

Change-Id: I7c0054028fa928ba025b3c511701512e183894fd
Reviewed-on: https://go-review.googlesource.com/c/build/+/365735
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

gopherbot commented Nov 29, 2021

Change https://golang.org/cl/367554 mentions this issue: cmd/coordinator: set buildenv when on GCE

gopherbot pushed a commit to golang/build that referenced this issue Nov 29, 2021
This change ensures the buildenv is set when the coordinator is
running in production.

Updates golang/go#47521

Change-Id: Ibd1a31609f5e85ac6445bad5daec5222a06b13e4
Reviewed-on: https://go-review.googlesource.com/c/build/+/367554
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

gopherbot commented May 12, 2022

Change https://go.dev/cl/406014 mentions this issue: internal/gomote: various updates

@gopherbot
Copy link

gopherbot commented May 12, 2022

Change https://go.dev/cl/406015 mentions this issue: cmd/gomote: implements GRPC put command

gopherbot pushed a commit to golang/build that referenced this issue May 12, 2022
These changes are being made in an attempt to move logic from the
client to the server.
- Changes the function which extracts the object name from a URL.
- Moves setting the environment variables to the server in ExecuteCommand endpoint.

For golang/go#47521
Updates golang/go#48742

Change-Id: I4fa370a1b3c949bd5913491d1650d131577ff30f
Reviewed-on: https://go-review.googlesource.com/c/build/+/406014
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 13, 2022
This change adds the implementation for the GRPC run command to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: I7e5fe3b66f552a10623d59e84adcea9856fe6683
Reviewed-on: https://go-review.googlesource.com/c/build/+/398496
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

gopherbot commented May 13, 2022

Change https://go.dev/cl/405256 mentions this issue: internal/coordinator/remote: add certificate authentication to server

gopherbot pushed a commit to golang/build that referenced this issue May 13, 2022
This change adds the implementation for the GRPC ping command to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: I6bbd7fd5b9f5a3d58063b4c433cab330bbb4259f
Reviewed-on: https://go-review.googlesource.com/c/build/+/398695
Reviewed-by: Alex Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
@gopherbot
Copy link

gopherbot commented May 13, 2022

Change https://go.dev/cl/406334 mentions this issue: deploy: correct IAP backend timeout

gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change updates the backend service timeout config and sets the
timeout to 2 hours. The default timeout is 30 seconds. This is
currently a problem when we issue a gomote create that takes more than
30 seconds to allocate a gomote instance. This timeout will be
encountered in other remote buidlet operations that follow the new path.

Updated golang/go#47521

Change-Id: Id88b0e1263a088f4841371cb37ff8c931580b109
Reviewed-on: https://go-review.googlesource.com/c/build/+/406334
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change refactors the SSH Server used by the coordinator. Portions
of the server have been moved into the internal/coordinator/remote
package and are being prepared for the addition of a different
authentication scheme.

Updates golang/go#52594
For golang/go#47521

Change-Id: Ib1e961ea6d27c861f787068d237a02a47b6b0a2c
Reviewed-on: https://go-review.googlesource.com/c/build/+/405255
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change adds OpenSSH certificate authentication to the SSH Server.
The change also:
- Moves the public key authentication logic into the remote package.
- Enables a chain of authentication methods such that both public
  key authentication and certificate authentication are used.
- Usees a dynamically created CA certificate for certificate
  authentication. Each time the server is started, a new certificate
  is created.

Fixes golang/go#52594
For golang/go#47521

Change-Id: I8198d436b53844357af01510dcf8f2fc67af83b4
Reviewed-on: https://go-review.googlesource.com/c/build/+/405256
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change adds the implementation for the GRPC ssh command to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: I1e1f09ad23b0f07d28e0c5d06ad00cb948bb41f8
Reviewed-on: https://go-review.googlesource.com/c/build/+/405514
Reviewed-by: Alex Rakoczy <alex@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change adds the implementation for GRPC rm command to the gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: Iaf1ed08d155cbad11c7aa2b74a9535f4b32233ca
Reviewed-on: https://go-review.googlesource.com/c/build/+/405515
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
This change moves the SSH handlers into the internal packages. It also
adds the handler which will use the session pool instead of the remote
buildlets.

Updates golang/go#52594
For golang/go#47521

Change-Id: I7e99fdbb16e0f80a871696cec79a9b638354e662
Reviewed-on: https://go-review.googlesource.com/c/build/+/405257
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 16, 2022
… status

This adds the gomote instances to the status page presented at
farmer.golang.org.

Updates golang/go#52594
For golang/go#47521

Change-Id: I29c73262031fc95cc85cdb43734da49149c958b3
Reviewed-on: https://go-review.googlesource.com/c/build/+/405258
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
@gopherbot
Copy link

gopherbot commented May 17, 2022

Change https://go.dev/cl/406857 mentions this issue: cmd/gomote: implements GRPC gettar command

gopherbot pushed a commit to golang/build that referenced this issue May 17, 2022
This change adds the implementation for the GRPC gettar comamnd to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: I8b8f12a3104977128d912ced41215faed69ea719
Reviewed-on: https://go-review.googlesource.com/c/build/+/406857
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue May 19, 2022
This change adds the implementation for GRPC put command to the gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: Ib2376444321ef9d0a754b60bcd3783f66a932f3d
Reviewed-on: https://go-review.googlesource.com/c/build/+/406015
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
@gopherbot
Copy link

gopherbot commented May 23, 2022

Change https://go.dev/cl/407878 mentions this issue: cmd/gomote: implements GRPC puttar command

@gopherbot
Copy link

gopherbot commented Jun 7, 2022

Change https://go.dev/cl/410818 mentions this issue: internal/gomote,cmd/gomote: implements GRPC add bootstrap

@gopherbot
Copy link

gopherbot commented Jun 7, 2022

Change https://go.dev/cl/410819 mentions this issue: cmd/gomote: implements GRPC push command

gopherbot pushed a commit to golang/build that referenced this issue Jun 7, 2022
This change adds the implementation for GRPC puttar command to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: I9b500b2f3ca70c78c3f288d0280eba02a1c59554
Reviewed-on: https://go-review.googlesource.com/c/build/+/407878
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Jun 7, 2022
This change adds the implementation for GRPC putbootstrap command to the
gomote client. It also adds the gomote server implementation of the
AddBootstrap endpoint. This endpoint adds the bootstrap Go version to
an existing client.

Updates golang/go#48737
Updates golang/go#48742
For golang/go#47521

Change-Id: Ib0807a13e85a0e350485c8300ac2e180456bd0fc
Reviewed-on: https://go-review.googlesource.com/c/build/+/410818
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Jun 7, 2022
This change adds the implementation for GRPC push command to the
gomote client.

Updates golang/go#48737
For golang/go#47521

Change-Id: Ibb40dff14b9be0c273fb26a625d5e64b1bca25f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/410819
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
@gopherbot
Copy link

gopherbot commented Jun 10, 2022

Change https://go.dev/cl/411065 mentions this issue: internal/gomote: fix ExecuteCommand

gopherbot pushed a commit to golang/build that referenced this issue Jun 12, 2022
This change fixes an incorrect variable in the ExecuteCommand endpoint.

For golang/go#47521
Updates golang/go#48742

Change-Id: Ic0f63e1ce83ba86a566981bdca16d57074dbb544
Reviewed-on: https://go-review.googlesource.com/c/build/+/411065
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
@gopherbot
Copy link

gopherbot commented Jun 15, 2022

Change https://go.dev/cl/412374 mentions this issue: cmd/gomote: adds missing field to GRPC push

gopherbot pushed a commit to golang/build that referenced this issue Jun 15, 2022
This change adds a missing directory setting to the GRPC push command.

For golang/go#48737
For golang/go#47521

Change-Id: I33daab7da55403df83033d0d4b6921bfeb10623c
Reviewed-on: https://go-review.googlesource.com/c/build/+/412374
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders NeedsFix
Projects
Go Release Team
In Progress
Development

No branches or pull requests

2 participants