Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade build tools from debian buster to bullseye #2224

Closed
markmandel opened this issue Aug 17, 2021 · 0 comments · Fixed by #2769
Closed

Upgrade build tools from debian buster to bullseye #2224

markmandel opened this issue Aug 17, 2021 · 0 comments · Fixed by #2769
Labels
area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. kind/bug These are bugs. kind/cleanup Refactoring code, fixing up documentation, etc
Milestone

Comments

@markmandel
Copy link
Member

Looks like we're getting some blocking on CI with Debian's latest release "bullseye" / 11:

You can see it here:
https://console.cloud.google.com/cloud-build/builds/3a2543f4-524b-4b1f-9720-19daae250155?project=258182270954

make[4]: Entering directory '/workspace/build'
docker build --tag=agones-build-sdk-csharp:b954dd54f8 /workspace/build/build-sdk-images/csharp --build-arg BASE_IMAGE=agones-build-sdk-base:48cd8562c3
Sending build context to Docker daemon  11.26kB
Step 1/8 : ARG BASE_IMAGE=agones-build-sdk-base:latest
Step 2/8 : FROM $BASE_IMAGE
 ---> 432b721aac95
Step 3/8 : RUN  apt-get update     && apt-get install -y wget
 ---> Running in 42fd8fbae443
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Reading package lists...
E: Repository 'http://security.debian.org/debian-security buster/updates InRelease' changed its 'Suite' value from 'stable' to 'oldstable'
E: Repository 'http://deb.debian.org/debian buster InRelease' changed its 'Suite' value from 'stable' to 'oldstable'
E: Repository 'http://deb.debian.org/debian buster-updates InRelease' changed its 'Suite' value from 'stable-updates' to 'oldstable-updates'
includes/sdk.mk:106: recipe for target 'build-build-sdk-image' failed
make[4]: Leaving directory '/workspace/build'
make[3]: *** [ensure-image] Error 2
includes/build-image.mk:54: recipe for target 'ensure-image' failed
includes/sdk.mk:123: recipe for target 'ensure-build-sdk-image' failed

Can see a workaround here:
https://www.reddit.com/r/debian/comments/ca3se6/for_people_who_gets_this_error_inrelease_changed/

Will look to see if we can do a quick workaround to unstick, but seems like a good opportunity to upgrade the base image everywhere.

@markmandel markmandel added kind/bug These are bugs. kind/cleanup Refactoring code, fixing up documentation, etc area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. labels Aug 17, 2021
markmandel added a commit to markmandel/agones that referenced this issue Aug 17, 2021
Switch out `apt-get update` commands that are run in CI to being
`apt-get --allow-releaseinfo-change update` to ignore the error on new
release change.

Workaround for googleforgames#2224 until we can upgrade everything to bullseye.
markmandel added a commit to markmandel/agones that referenced this issue Aug 17, 2021
Switch out `apt-get update` commands that are run in CI to being
`apt-get --allow-releaseinfo-change update` to ignore the error on new
release change.

Workaround for googleforgames#2224 until we can upgrade everything to bullseye.
markmandel added a commit that referenced this issue Aug 18, 2021
* Workaround for bullseye release CI blockage

Switch out `apt-get update` commands that are run in CI to being
`apt-get --allow-releaseinfo-change update` to ignore the error on new
release change.

Workaround for #2224 until we can upgrade everything to bullseye.

* Add data-proofer-ignore to vela.games

Getting a cloudflare 403 captcha with this url.
markmandel added a commit to markmandel/agones that referenced this issue Oct 20, 2022
The upgrade to gRPC solved one issue, and I also added a limit to number
of processes that could run for `make -j` otherwise the whole thing
would fall over (also would crash my dev machine!).

Closes googleforgames#2224
markmandel added a commit to markmandel/agones that referenced this issue Oct 20, 2022
The upgrade to gRPC solved one issue, and I also added a limit to number
of processes that could run for `make -j` otherwise the whole thing
would fall over (also would crash my dev machine!).

Closes googleforgames#2224
chiayi pushed a commit to chiayi/agones that referenced this issue Nov 17, 2022
nodepools and regional clusters

Updates to release checklist. (googleforgames#2772)

* Updates to release checklist.

Adding items that showed up in the recent release that were not written
down or required better clarification.

* Review updates, and some other small tweaks.

Co-authored-by: Robert Bailey <robertbailey@google.com>

Release 1.27.0 (googleforgames#2776)

* Release 1.27.0

* Update FAQ on ExternalDNS (googleforgames#2773)

The feature flag it points to have been moved to stable, so the link
is not useful any more.

Also removed notes on ipv6, since they aren't 100% accurate, as we were
discussing in googleforgames#2767.

* Updates to release checklist. (googleforgames#2772)

* Updates to release checklist.

Adding items that showed up in the recent release that were not written
down or required better clarification.

* Review updates, and some other small tweaks.

Co-authored-by: Robert Bailey <robertbailey@google.com>

* Release-changes

* Review comment

* Review changes

Co-authored-by: Mark Mandel <markmandel@google.com>
Co-authored-by: Robert Bailey <robertbailey@google.com>

Version updates (googleforgames#2778)

Players in-game metric for when PlayerTracking is enabled (googleforgames#2765)

* Check for DeletionTimestamp of fleet and gameserverset before scaling

* Add metric to track player count in gameservers

* check PlayerStatus is not nil

* Update metrics available in docs

* Wrong relref path

* typo

* Change name for players in game metric to player connected. Add player capacity metric. Hide docs until next agones release.

* Duplicate metrics table

* add gameserver player tracking metrics to fleetViews

Co-authored-by: Mark Mandel <markmandel@google.com>

Remove generation for swagger Go code and Add static swagger codes for test (googleforgames#2757)

Co-authored-by: Mark Mandel <markmandel@google.com>

Updated allocation yaml files under examples/ to use selectors

Show how to set graceful termination in a game server that is safe to (googleforgames#2780)

evict.

Avoid retry from allocateFromLocalCluster under context kill. (googleforgames#2783)

* Version updates

* issue-2736-changes

Co-authored-by: Mark Mandel <markmandel@google.com>

Bring SDK base image to debian:bullseye (googleforgames#2769)

* Bring SDK base image to debian:bullseye

The upgrade to gRPC solved one issue, and I also added a limit to number
of processes that could run for `make -j` otherwise the whole thing
would fall over (also would crash my dev machine!).

Closes googleforgames#2224

* Force refresh of cpp cache on Cloud Build.

* Fixes for CI:

* Revert CI cache increment (don't think we need it)
* Add shell to cpp image for debugging.
* Fix formatting issue that is breaking CI.

Co-authored-by: Robert Bailey <robertbailey@google.com>

Update health-checking.md (googleforgames#2785)

Fixed spell error: spec.health.failureTheshold to spec.health.failureThreshold

Updated allocation yaml files under examples/ to use selectors (googleforgames#2787)

Cleanup of load tests (googleforgames#2784)

* issue-2744 updated changes with new description
* 2744 review changes

Sync Pod host ports back to GameServer in GCP (googleforgames#2782)

This is the start of the implementation for googleforgames#2777:

* Most of this is mechanical and implements a thin cloud product
abstraction layer in pkg/cloud, instantiated with New(product). The
product abstraction provides a single function so far:
SyncPodPortsToGameServer.

* SyncPodPortsToGameServer is inserted as a hook while syncing
IP/ports, to let different cloud providers handle port allocation
slightly differently (in this case, GKE Autopilot)

* In GKE Autopilot, we look for a JSON string like
`{"min":7000,"max":8000,"portsAssigned":{"7001":7737,"7002":7738}}`
as an indication that the host ports were reassigned (per policy).
As a side note to anyone watching, this is currently an unreleased
feature. If we see this, we use the provided mapping to map the
host ports in the GameServer.Spec.

With this change, it's possible to launch a GameServer and get a
healthy GameServer Pod by adding the following annotation:

```
annotations:
  cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
  autopilot.gke.io/host-port-assignment: '{"min": 7000, "max": 8000}'
```

If this PR causes any issues, the cloud product auto detection can
be disabled by setting `agones.cloudProduct=generic`, or forced to
GKE Autopilot using `agones.cloudProduct=gke-autopilot`.

In a future PR, I will add the host-port-assignment annotation
automatically on Autopilot

Co-authored-by: Mark Mandel <markmandel@google.com>

Update gke terraform files to allow autoscaling

Fix (not really) problems reported by VSCode (googleforgames#2790)

VSCode reports `main redeclared` between allocationload.go and
runscenario.go due to the fact that they both look like `package main`
binaries in the same directory, similar e.g. [this poster on a
different
project](https://stackoverflow.com/questions/66970531/vs-code-go-main-redeclared-in-this-block)

To fix it, it's easy enough to just give these binaries their own
package path and fix up the calling scripts.

Along the way, fix a lint complaint in runscenario.go

Add location variable for cluster location argument

Minor fix

changed default of location var to empty string

GameServerRestartBeforeReadyCrash: Run serially (googleforgames#2791)

Narrow the race in googleforgames#2445 by running GameServerRestartBeforeReadyCrash serially. See googleforgames#2445 (comment) for a detailed analysis.

Does not fix the issue - this is stopgap until we understand how to fix it.

Enable fieldalignment linter, then mostly ignore it (googleforgames#2795)

Enable the fieldalignment linter by enabling all `govet` checks
except shadowing. Ignore large swaths of code (tests, cmd/, APIs),
and nolint'd existing complaints that seemed irrelevant.

Along the way:

* removed existing nolint:maligned, as `maligned` is no more.
* disabled `structcheck` and `deadcode` as they are deprecated (and I
think have been subsumed by other linters?)
* changed `gameServerCacheEntry` to `gameServerCache`. It is the
cache, not just an entry.
* fixed alignment of `gameServerSetCacheEntry`.

Add fswatch library to watch and batch filesystem events, use in allocator (googleforgames#2792)

This pull refactors the fsnotify code in allocator/main out to a
shared library, and in that shared library implements a batched
notification processor.

Closes googleforgames#1816: This takes a slightly different approach than specified
in the issue, instead choosing to just delay processing until after a
batch processing period. I chose 1s - it's far longer than necessary,
but still much shorter than it takes for the secret changes to
propagate to the container anyways.

I considered the approach in googleforgames#1816 of trying to parse the actual
events, but it's too fiddly to get exactly right: e.g. maybe you only
refresh on "write", but then "chmod" could make the file readable
whereas it wasn't before, "rename" could expose a file that wasn't
there before, etc.

Cloud product: Split port allocators, implement Autopilot port allocation/policies (googleforgames#2789)

In the Agones on GKE Autopilot implementation, we have no need for the
port allocator - the informer/etc. is an unnecessary moving piece.
This PR allows for cloud products to provide their own port allocation
implementation, and implements the GKE Autopilot "allocator". We do
this by:

* Splitting portallocator off to its own package. It was basically
self-sufficient anyways, except it was a little too friendly with
controller_test.go. I solved that by introducing a TestInterface for
controller_test.go to upcast to.

* Allow cloud product implementations to define their own port
allocator.

* Defining a new port allocator for GKE that does a simple per-port
HostPort allocation, and adds the host-port-assignment annotation to
the pod template.

* Extend cloudproduct again to add a GameServer validator

* And in Autopilot, reject if the PortPolicy is not `Dynamic`

Release: Note to switch away from `agones-images` (googleforgames#2809)

Since we have few guardrails on accidentally touching `agones-images`
project, adding a note in the release checklist to switch back to a
local development project after running a release.

Flake: TestControllerGameServerCount (googleforgames#2805)

Made it deterministic in the test, and got rod of the potential race
conditions.

Also fix it such that the util function for generating GameServer names
always produce a unique name.

Closes googleforgames#2804

Co-authored-by: Robert Bailey <robertbailey@google.com>

Remove Windows FAQ Entry (googleforgames#2811)

The contents are no longer accurate, and are covered in the installation
section now.

Makefile changes for adding location variable

added autoscale parameters to Makefile and README

Markdown fix in readme

Changed LOCATION to always be set with ZONE as default

use  only if the variable has a value

fixed extraneous characters

update gke terraform exmaple module

Update Node.js dependencies and package (googleforgames#2815)

* Update all dependencies and Node,js to LTS version

* Update other docker images that use Node.js

Added autoscale to example cluster and added to website docs

Added defaults and feature expiry

Remove zone from gke/variable.tf file.
@mangalpalli mangalpalli added this to the 1.28.0 milestone Nov 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. kind/bug These are bugs. kind/cleanup Refactoring code, fixing up documentation, etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants