[v1.16.x] Backport Extractor Regex Replace functionality (#9236)

* Control plane support for Extractor regex replace (#9124)

* update envoy transformation API w changes as of 55ddd1bfc4d9976bc953cab2dd0fcc874edd9405

* add new settings to user-facing API

* update transformation plugin + add unit tests

* bump envoy-gloo version to pre-release hash

* unit testing updates

* check regex validity in transformation plugin

* use table based tests

* add e2e tests

* update comment, add DO_NOT_SUBMIT tag

* testing updates

* create error type

* support dynamic_metadata API field

* support dynamic_metadata API field in user-facing API

* Revert "support dynamic_metadata API field in user-facing API"

This reverts commit 1602aba.

* Revert "support dynamic_metadata API field"

This reverts commit a001c66.

* update inline documentation of transformation filter to match envoy-gloo changes

* add changelog entry

* use pre-release ENVOY_GLOO_IMAGE

* remove DO_NOT_MERGE, add new test against default mode

* add documentation for new feature

* clean up plugin.go todos and commentary

* update external API documentation

* remove remaining TODO in plugin.test.go

* unfocus tests in plugin_test.go

* let control plane be dumb about regex semantics; already covered by envoy validation mode

* update proto documentation for new modes

* docs updates

* Adding changelog file to new location

* Deleting changelog file from old location

* bump ENVOY_GLOO_IMAGE to 1.28.1-patch2

* documentation updates

* update user guide, .proto inline documentation

* remove unintentially committed bananaToOrange test

* resolve typos in .proto files

* update tests with new defaults, move envoy transformation validation tests to kube2e suite

* Adding changelog file to new location

* Deleting changelog file from old location

* update envoy-gloo dep to point to new 1.27 release

* empty commit to kick build

* clean up modeName check when generating extractor errors

Co-authored-by: Jacob Bohanon <jacob.bohanon@solo.io>

* remove duplicated default extraction mode handler

* validate new default mode handling in new test

---------

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: changelog-bot <changelog-bot>
Co-authored-by: Jacob Bohanon <jacob.bohanon@solo.io>

* update changelog location

* respsect updated variable name in kube2e gateway_test.go

* update language in user guide

* correct typo in .proto documentation

---------

Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Jacob Bohanon <jacob.bohanon@solo.io>

Makefile

@@ -48,7 +48,7 @@ VERSION ?= 1.0.1-dev
 
 SOURCES := $(shell find . -name "*.go" | grep -v test.go)
 
-ENVOY_GLOO_IMAGE ?= quay.io/solo-io/envoy-gloo:1.27.3-patch1
+ENVOY_GLOO_IMAGE ?= quay.io/solo-io/envoy-gloo:1.27.3-patch2
 LDFLAGS := "-X github.com/solo-io/gloo/pkg/version.Version=$(VERSION)"
 GCFLAGS := all="-N -l"
 

changelog/v1.16.8/extractor-regex-replace.yaml

@@ -0,0 +1,11 @@
+changelog:
+  - type: DEPENDENCY_BUMP
+    resolvesIssue: false
+    issueLink: https://github.com/solo-io/gloo/issues/8706
+    dependencyOwner: envoy-gloo
+    dependencyRepo: solo-io
+    dependencyTag: 1.27.3-patch2
+    description: >-
+      Upgrade envoy-gloo version to pull in transformation filter updates.
+      With this bump, transformation filter extractors now support regex
+      replace/replace all operations on extracted values.

docs/content/guides/traffic_management/request_processing/transformations/_index.md

@@ -270,6 +270,76 @@ Extracted values can be used in two ways:
   - If `advancedTemplates` is set to `true`, format the string such as `{{ extraction(my-extractor) }}`.
 - You can use them in conjunction with the `mergeExtractorsToBody` body transformation type to merge them into the body.
 
+##### Extractor Modes
+
+Starting with Gloo Edge 1.15, extractors support multiple different modes of operation. These modes enable more powerful and flexible extraction capabilities. You can specify the mode of operation for an extractor using the `mode` field. The supported modes are:
+ - `Extract` (default): Extracts the value of the specified capturing group from the source. This is the standard behavior of extractors prior to this version.
+ - `Single Replace`: Replaces the value of the n-th capturing group, where n is the value selected by `subgroup`, with the text specified in `replacement_text`.
+ - `Replace All`: Replaces all occurrences of the pattern specified in the `regex` field within the source with the text specified in `replacement_text`.
+
+{{% notice warning %}}
+The semantics of the `regex` field vary by mode and must be carefully considered.
+{{% /notice %}}
+
+ - `Extract`: The `regex` must match the entire source. Only the specified `subgroup` is extracted.
+ - `Single Replace`: The `regex` must match the entire source for replacement to occur. The `subgroup` specifies which part of the match to replace.
+ - `Replace All`: The `regex` can match any part of the source. All matches are replaced, and configuration which specifies a `subgroup` will be rejected.
+
+##### Extractor Configuration Validation
+
+To ensure correct configuration, Gloo Edge performs validation based on the extractor's mode:
+ - For **Extract** mode, the `replacement_text` must not be set. If set, the configuration will be rejected.
+ - For **Single Replace** and **Replace All** modes, the `replacement_text` must be set. If not set, the configuration will be rejected.
+ - For **Replace All** mode, the `subgroup` must not be set, as it is not used. If set, the configuration will be rejected.  
+
+##### Extractor Example - Single Replace Mode
+
+Define an extractor that replaces the first occurrence of the item `banana` with `orange` in a list of fruits. The exact composition of the list is unknown, but the format is the same, such as a comma-separated list.
+Fruits: `[apple, pear, banana, pineapple, etc.]`
+```
+
+```yaml
+extractors:
+  BananaToOrange:
+    body: {}
+    mode: SINGLE_REPLACE
+    replacement_text: 'orange'
+    regex: '.*(banana).*'
+    subgroup: 1
+```
+In this configuration:
+ - `BananaToOrange` is the name of the extractor.
+ - `body: {}` indicates that the request/response body will be used as input to the extractor.
+ - `regex: '.*(banana).*'` is designed to match the entire source, and capture the word `banana` in the first capturing group.
+ - `mode: SINGLE_REPLACE` specifies the operation mode, indicating a single replace operation.
+ - `replacement_text: 'orange'` replaces the content in the first capturing group with the text `orange`.
+ - `subgroup: 1` specifies that the replacement should only apply to the first captured group, which is banana in this case.
+
+In this case, if the extractor processes a body containing the text `Fruits: [apple, pear, banana, pineapple]`
+
+The result will be `Fruits: [apple, pear, orange, pineapple]`
+
+#### Extractor Example - Replace All Mode  
+
+Define an extractor that replaces all occurrences of the pattern `foo` with the text `bar` in the source.
+```yaml
+extractors:
+  FooToBar:
+    body: {}
+    regex: 'foo'
+    mode: REPLACE_ALL
+    replacement_text: 'bar'
+```
+
+In this configuration:
+ - `FooToBar` is the name of the extractor.
+ - `body: {}` indicates that the request/response body will be used as input to the extractor.
+ - `regex: 'foo'` specifies the pattern to match.
+ - `mode: REPLACE_ALL` specifies the mode of operation.
+ - `replacement_text: 'bar'` specifies the text to replace the matched pattern with.
+
+In this case, if the extractor processes a body containing the text `foo foo foo`, the result will be `bar bar bar`.
+
 ##### headers
 Use this attribute to apply templates to request/response headers. It consists of a map where each key determines the name of the resulting header, while the corresponding value is a {{< protobuf display="template" name="envoy.api.v2.filter.http.InjaTemplate" >}} which will determine the value.