Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update generate-manifest-flow-aggregator.sh with new config #3526

Merged
merged 1 commit into from
Mar 28, 2022
Merged

Update generate-manifest-flow-aggregator.sh with new config #3526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions ci/jenkins/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,11 @@ DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
daily validation of elk flow collector manifest. If build fails, Jenkins will send an email to
projectantrea-dev@googlegroups.com for notification.

* [daily-flow-visibility-validate](https://jenkins.antrea-ci.rocks/job/antrea-daily-flow-visibility-validate-for-period/):
[![Build Status](http://jenkins.antrea-ci.rocks/buildStatus/icon?job=antrea-daily-flow-visibility-validate-for-period)](http://jenkins.antrea-ci.rocks/view/cloud/job/antrea-daily-flow-visibility-validate-for-period/)
daily validation of Flow Visibility manifest. If build fails, Jenkins will send an email to
projectantrea-dev@googlegroups.com for notification.

* [matrix-test [weekly]](https://jenkins.antrea-ci.rocks/job/antrea-weekly-matrix-compatibility-test/):
runs Antrea e2e, K8s Conformance and NetworkPolicy tests, using different combinations of various operating systems and K8s releases.
| K8s Version | Node OS | Status |
Expand Down
2 changes: 1 addition & 1 deletion ci/test-flow-visibility.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ config_antrea() {

setup_flow_aggregator() {
echo "=== Config Antrea Flow Aggregator ==="
perl -i -p0e 's/ # Enable is the switch to enable exporting flow records to ClickHouse.\n #enable: false/ # Enable is the switch to enable exporting flow records to ClickHouse.\n enable: true/' ./build/yamls/flow-aggregator.yml
$GIT_CHECKOUT_DIR/hack/generate-manifest-flow-aggregator.sh -ch > ${GIT_CHECKOUT_DIR}/build/yamls/flow-aggregator.yml
echo "=== Start Antrea Flow Aggregator ==="
kubectl apply -f ${GIT_CHECKOUT_DIR}/build/yamls/flow-aggregator.yml
echo "=== Waiting for Antrea Flow Aggregator to be ready ==="
Expand Down
31 changes: 21 additions & 10 deletions hack/generate-manifest-flow-aggregator.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,17 @@ function echoerr {
>&2 echo "$@"
}

_usage="Usage: $0 [--mode (dev|release)] [-fc|--flow-collector] [--keep] [--help|-h]
_usage="Usage: $0 [--mode (dev|release)] [-fc|--flow-collector <addr>] [-ch|--clickhouse] [--keep] [--help|-h]
Generate a YAML manifest for the Flow Aggregator, using Kustomize, and print it to stdout.
--mode (dev|release) Choose the configuration variant that you need (default is 'dev')
--flow-collector Flow collector is the externalFlowCollectorAddr configMap parameter
It should be given in format IP:port:proto. Example: 192.168.1.100:4739:udp
--keep Debug flag which will preserve the generated kustomization.yml
--coverage Generate a manifest which supports measuring code coverage of the Flow Aggregator binaries.
--verbose-log Generate a manifest with increased log-level (level 4) for the Flow Aggregator.
This option will work only with 'dev' mode.
--help, -h Print this message and exit
--mode (dev|release) Choose the configuration variant that you need (default is 'dev').
--flow-collector, -fc <addr> Specify the flowCollector address.
It should be given in format IP:port:proto. Example: 192.168.1.100:4739:udp.
--clickhouse, -ch Enable exporting flow records to default ClickHouse service address.
--keep Debug flag which will preserve the generated kustomization.yml.
--coverage Generate a manifest which supports measuring code coverage of the Flow Aggregator binaries.
--verbose-log Generate a manifest with increased log-level (level 4) for the Flow Aggregator.
This option will work only with 'dev' mode.
--help, -h Print this message and exit.

In 'release' mode, environment variables IMG_NAME and IMG_TAG must be set.

Expand All @@ -49,6 +50,7 @@ function print_help {
MODE="dev"
KEEP=false
FLOW_COLLECTOR=""
CLICKHOUSE=false
COVERAGE=false
VERBOSE_LOG=false

Expand All @@ -65,6 +67,10 @@ case $key in
FLOW_COLLECTOR="$2"
shift 2
;;
-ch|--clickhouse)
CLICKHOUSE=true
shift
;;
--keep)
KEEP=true
shift
Expand Down Expand Up @@ -138,7 +144,12 @@ mkdir configMap && cd configMap
# but instead to the generated YAML manifest, so our regexs need not be too robust.
cp $KUSTOMIZATION_DIR/base/conf/flow-aggregator.conf flow-aggregator.conf
if [[ $FLOW_COLLECTOR != "" ]]; then
sed -i.bak -E "s/^[[:space:]]*#[[:space:]]*externalFlowCollectorAddr[[:space:]]*:[[:space:]]\"\"+[[:space:]]*$/externalFlowCollectorAddr: \"$FLOW_COLLECTOR\"/" flow-aggregator.conf
perl -i -p0e 's/ # Enable is the switch to enable exporting flow records to external flow collector.\n #enable: false/ # Enable is the switch to enable exporting flow records to external flow collector.\n enable: true/' flow-aggregator.conf
sed -i.bak -E "s/^[[:space:]]*#[[:space:]]*address[[:space:]]*:[[:space:]]\"\"+[[:space:]]*$/ address: \"$FLOW_COLLECTOR\"/" flow-aggregator.conf
fi

if $CLICKHOUSE; then
perl -i -p0e 's/ # Enable is the switch to enable exporting flow records to ClickHouse.\n #enable: false/ # Enable is the switch to enable exporting flow records to ClickHouse.\n enable: true/' flow-aggregator.conf
Comment on lines +147 to +152
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope the perl dependency will not be an issue. But I understand that multiline sed is not practical. We can improve later if needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dreamtalen Can you open a github issue for getting rid of the perl dependency? Just to make sure we will remember to do that before it becomes a problem!

fi

# unfortunately 'kustomize edit add configmap' does not support specifying 'merge' as the behavior,
Expand Down