…fter delete

More see b/339458480

* docs: add feature docs

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

* docs: also link from release notes

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

* Apply suggestions from code review

---------

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

The older version has a CVE (though it does not appear to be reachable)

Wiring up support for our direct controllers here.

A step in running samples more like fixtures.

This was auto-generated using the dev-to-proto tool, and then a few
fixes were made manually:

* Commented out some fields that we aren't supporting yet (with NOTYET)
* Changed some `*Duration` to `*string`
* Fixed up the definition of `Empty`
* Added `+required` annotations
* Fixed up the Spec and Status
* Added some registration boilerplate

Also, I switched to the latest controller-gen in
dev/tasks/generate-crds, so that we can use the `+required` annotation
(and to pick up the omitempty fix)

We now need to support int32

This allows us to see unsupported fields.

Some APIs may include objects in maps, we want to output those objects
also.

A first-pass at a tool to generate types & map functions.

Including around refs

It's not perfect (v3 vs v1 is tricky), but it's much closer.

This allows us to record test output even when we must create a project.

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

- Allows structured config for inbuilt expanders
- The ability to support refs for external configs
- remove proto annotations (cleanup)

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

This avoids depending on the current gcloud project (which is likely
not set in github actions, at least).

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

…tests

Working towards being able to run samples & fixtures the same way.

This avoids whitespace changes which make the diff harder to review.

We have a test that detected this field is being removed (it's an
alpha resource, so not a problem).

Remove the mapping to keep the test happy.

This should make it easier to understand why re-reconciliation is happening.

- Was added initially when we hardcoded the image in the job template
- ExpanderVersion removes the need for this parameter

…-spec

Using the field manager, we can detect and remove any fields that were
set by state-into-spec.

This way our various tests can share one normalization function

This enables us to test our powertools in our existing framework

Changing the location of a StorageBucket is a good example usage.

…dia-tesla-k80'

Trying to put this in a "magic directory", to trigger automated job
creation in future tooling.

* docs: add instr to gen proto

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

---------

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

- Implement grpc based getter expander
- Getter config is inline in Composition CRD
- Add release manifests for getter-expander
- Add e2e tests for getter-expander
- Add presubmit for getter-expander e2e

Next steps:

- Split out Getter config into its own CRD

- rename gjinjg2 -> jinja2 (making jinja2 grpc based)
- rename existing pod based jinja2 -> pjinja2
- Add `image` field to ExpanderVersion
  - allow specifying an image that doesnt follow the
    removePrefix(expanderversion.name , "composition-") pattern.
  - provides more flexibility
  - In our case the type is pjinja and the image is expander-jinja
- Add a custom ratelimiter for the facade reconciler
  - Due to a bug, the reconciler is not restarted across tests.
  - Old failures add to rate limiting causing random failures
  - make ratelimiter max backoff 120s
- Add a small sleep b/w tests
- Move all ExistTimeout to CompositionReconcileTimeout
- Check if a plan was updated and if so check if the applier sees the
  latest plan object. If not retry again.
- Add InputGeneration and Generation to plan status

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

Signed-off-by: Alex Pana <8968914+acpana@users.noreply.github.com>

my tests are based on the alloydb which is already in that regex

- Remove inline ValuesFrom from Compositions
  - Remove `ValuesFrom` from the composition spec
  - Remove code that waits for and fetches values from the expander
    reconciler (Expander_fetcher.go)
  - Change e2e test to refelect this change
- Add a new CRD `Getter` that acts as a Getter configuration
  - Add validation for this CRD
- Proto changes
  - GRPC `Evaluate` returns Values or manifests.
  - WAIT as a return status
- New changes in expander reconciler
  - Accumulate values so that they are presented to all subsequent
    stages
  - Get Expander config by reference and pass it to Evaluate
  - Get expander config GVK by looking up the expander version
  - When expander returns WAIT retry sooner in 5s
- ExpanderVersion changes
  - Add GVK to bind an Expander with its Config CRD
- Expander GRPC Service changes
  - Receive the whole Getter config object as Config
  - Introduce WAIT as a return value when object or fieldpath doesnt
    exist

TODO:
- Check Getter CR validation error in expander reconciler or Getter grpc
  service
- Move Samples to use the new getter
- Move from a projection based value extraction to whole object getter
- Exp backoff for WAIT state ?

…e comparison when empty to fix scenario test data for powertool

- regenerate proto bufs with latest lib to fix build failures

- KCC Test changes
  - In CloudSQL test, Create an AppTeam and in the new project create
- Samples changes
  - Remove alpha from expander names (grpc ones)
  - In hasql composition,
    - Use new grpc getter, GetterConfiguration
- Test infra changes
  - Add support for dependent samples for KCC Scenario (CloudSQL on AppTeam)
    CloudSQL (testing user , demo flow)
- Makefile changes
  - add run-test target to run a single testcase
  - make CloudSQL the default kcc-test
- Reconciler changes
  - dont json.Marshall() the expander template
  - conversely no need to unquote the result
  - json.Marshall() was escaping double quotes, and converting '>' to
    uni-code
- e2e Test changes
  - Add new test case to verify roundtripping template is not broken
    - TestSimpleExpanderJinjaWithQuotes
  - Add more logs to debug kind cluster create failures
  - Fix sample expander versions

We want to keep these separated (experiments has its own build/CI scripts)