1761: PoC of cyclonus netpol test case generator

[mattfenwick]

This is a Proof-of-Concept of #1761 -- adding a generator for network policy testing.

Example invocation

Assuming you have a working kube cluster with antrea set up, you can grab a release and run:

$ cyclonus generate --noisy=true --mode simple-fragments --cleanup-namespaces=true
...
testing 152 policies

TODOs:

• make sure PR is up-to-date with newest cyclonus CLI args
• include a couple lines there about what cyclonus is trying to do and the long term objective
• define a new scheduled Github workflow there (https://github.com/vmware-tanzu/antrea/tree/main/.github/workflows) to run cyclonus every day
• refer to the current job for netpol: https://github.com/vmware-tanzu/antrea/blob/c2c24d40c70ad09fda6b565326421131c43d5992/.github/workflows/kind.yml#L400
• analyze test output for failures
• switch cyclonus job over to once-a-day schedule

[vmwclabot]

@mattfenwick, you must sign our contributor license agreement before your changes are merged. Click here to sign the agreement. If you are a VMware employee, read this for further instruction.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (main@dc0b4d5). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1765   +/-   ##
=======================================
  Coverage        ?   42.78%           
=======================================
  Files           ?      194           
  Lines           ?    16545           
  Branches        ?        0           
=======================================
  Hits            ?     7078           
  Misses          ?     8469           
  Partials        ?      998           

Flag	Coverage Δ
kind-e2e-tests	42.78% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

[vmwclabot]

@mattfenwick, VMware has approved your signed contributor license agreement.

[antoninbas]

Thanks for your PR! I am curious about the rationale for checking-in the main.go here? Shouldn't you just publish binaries as part of github.com/mattfenwick/cyclonus releases? We could then download the binary and use it directly.

Either way, we need a README.md file under hack/netpol-fuzzer with instructions on how to run the fuzzer.

[mattfenwick]

Hi @antoninbas thanks for the feedback!

Yeah I wasn't sure the best way to organize things so figured I'd just put something up to get the ball rolling :)

I'll make those updates to this PR 👍 👍 happy to sync up and talk more about this!

[antoninbas]

Hi @antoninbas thanks for the feedback!

Yeah I wasn't sure the best way to organize things so figured I'd just put something up to get the ball rolling :)

I'll make those updates to this PR 👍 👍 happy to sync up and talk more about this!

Ping me on the K8s Slack if want to talk about the PR. Looking forward to testing this. Does the fuzzer keep running tests until you terminate the process? Also, I would love to find some time to extend it to Antrea-native policies if this proves useful for K8s NetworkPolicies.

[mattfenwick]

Thanks for the review @antoninbas , working through the comments!! 👍 👍

[antoninbas]

LGTM, I think we should get a review from @abhiraut as well.

Did you run the job once in Github to confirm it works fine? Otherwise, if there is a typo, we won't know until the job runs for the first time, after the PR is merged. One way to do that is to temporarily change the schedule parameter to add a pull_request trigger, make sure it runs once, then roll back the temporary change,

[mattfenwick]

Temporarily added a pull_request action and rolled it back, here's the successful run:

https://github.com/vmware-tanzu/antrea/pull/1765/checks?check_run_id=1826892005

[abhiraut]

thanks.. the PoC looks good. Do we have a list of types of policies that it generates? also could you explain what the mode param is for? i see you set it to upstream in test and commit msg talks about simple-fragments.

[mattfenwick]

@abhiraut thanks for taking a look!

My plan is to get this proof-of-concept up and running for say a week, look at how well it's working (flaky/stable, how long it takes to run, etc.) and then make a tiny patch to the job to run additional tests.

Maybe it's worth discussing which test cases are highest impact/value to run for Antrea? The upstream mode is a subset of the kube e2e netpol tests which exercises a lot of features but doesn't take too long to run if you're just kicking the tires, the simple-fragments mode is closer to the "generator" idea but still at only 150 test cases, doesn't take forever 😄 to run!

[antoninbas]

/skip-all