[Workload] Microservice Test: Are processes reaped (zombies) properly?

[wvwatson]

Title: [Workload] Microservices test: process_reaped

Is your workload test idea related to a problem? Please describe.
How classic init systems handle orphaned processes
Classic init systems such as systemd are also used to remove (reap) orphaned, zombie processes. Orphaned processes—processes whose parents have died—are reattached to the process that has PID 1, which should reap them when they die. A normal init system does that. But in a container, this responsibility falls on whatever process has PID 1. If that process doesn't properly handle the reaping, you risk running out of memory or some other resources.
https://cloud.google.com/architecture/best-practices-for-building-containers

These problems have several common solutions, outlined in the following sections.

Describe the solution you'd like

• The test should determine if there are child processes in the CNF, and if there are, kill one of them. Then it should check to see if a zombie persists.
• https://www.alibabacloud.com/blog/zombie-processes-how-to-hunt-kill-and-remove-a-zombie-process-on-linux_597383

Test Category Name

• Microservice

Type of test (static or runtime)

• runtime

---

Documentation tasks:

• Update installation instructions if needed
• Update Test Categories md if needed
• Update USAGE md if needed
  • How to run
  • Description and details
    • What the best practice is
    • Why are we testing this
  • Remediation steps if test does not pass

QA tasks

Dev Review:

• walk through A/C
• do you get the expected result?
• if yes,
  • move to Needs Peer Review column
  • create Pull Request and follow check list
  • Assign 1 or more people for peer review
• if no, document what additional tasks will be needed

Peer review:

• walk through A/C
• do you get the expected result?
• if yes,
  • move to Reviewer Approved column
  • Approve pull request
• if no,
  • document what did not go as expected, including error messages and screenshots (if possible)
  • Add comment to pull request
  • request changes to pull request

[denverwilliams]

Skeleton code & research to test for unclean term signals being sent to sub-processes.
https://hackmd.io/X24dvbkTTMmFUuEU0f-Wtg

[agentpoyo]

Acceptance Criteria

• Setup or find a k8s cluster
• Pull down latest branch with code updates or main if merged already; build cnf-testsuite from source.
• Run setup
• Now run cnf_setup and use sample-cnfs/sample_good_zombie_handling/cnf-testsuite.yml as your CNF to setup
• If successfully installed onto your cluster, now run cnf-testsuite zombie_handled and you should see this PASS
• Run cnf_cleanup on the config file to remove the sample CNF
• Now run cnf_setup and use sample-cnfs/sample-bad-zombie/cnf-testsuite.yml as your CNF to setup
• If successfully installed onto your cluster, now run cnf-testsuite zombie_handled and you should see this FAIL
• I can see a screenshot of the results here.

[agentpoyo]

[agentpoyo]

[agentpoyo]

I had to modify the helm namspace creation, my cluster was giving me errors. It's likely an environment issue since using default namespace for the individual test resulted in desired output: