[runtime-security] Execute runtime security functional tests using Kitchen #6328

lebauce · 2020-09-07T11:37:01Z

What does this PR do?

Execute the runtime security agent functional tests using Kitchen

Motivation

Test the runtime security agent on multiple OS / kernel versions

KSerrania

You may want to make the agent-security team owner of /test/kitchen/site-cookbooks/dd-agent-runtime-security-check/ and /test/kitchen/kitchen-azure-runtime-security-test.yml in the CODEOWNERS file (you can add agent-platform as joint owners if you want to).

.gitlab-ci.yml

albertvaka

LGTM assuming ^TestMkdir$ does what it seems it does and all paths are correct.

Note that until now on kitchen tests we only cared about the OS they ran on, and not the kernel version. If you want to test the EBPF code against several kernels, you might want to use something more specific than .kitchen_os_centos, .kitchen_os_ubuntu, etc.

But that can be a second iteration, this is much better already than where we were before :D

Edit: +1 to Kylian's comments above ^

...test/integration/dd-agent-runtime-security-test/rspec/dd-agent-runtime-security-test_spec.rb

Kylian found something

guedou

Nice PR. I made some minor comments.

guedou · 2020-10-09T11:33:27Z

pkg/security/ebpf/kprobe_bpf.go

-	for i := 0; i != maxEnableRetry; i++ {
-		if err = m.EnableKprobe(secName, 512); err == nil {
-			break
+func (m *Module) tryEnableKprobe(secName string) error {


Shall we have a metric associated to retries to ease debugging?

We can discuss it with the team but I don't think this should be part of the PR

guedou · 2020-10-09T13:08:21Z

test/kitchen/site-cookbooks/dd-security-agent-check/recipes/default.rb

+        package_name 'glibc.i686'
+      when 'ubuntu', 'debian'
+        package_name 'libc6-i386'
+      # when 'suse'


Is there a reason why this is commented?

Yes, to not forget to do it when we'll be able to install packages on SUSE

guedou · 2020-10-09T13:09:22Z

test/kitchen/test/integration/dd-security-agent-test/rspec/dd-security-agent-test_spec.rb

+  it 'displays PASS and returns 0' do
+    output = `sudo /tmp/security-agent/testsuite -test.v 1>&2`
+    retval = $?
+    expect(retval).to eq(0)


Do you want to also add expect(output).not_to include("FAIL") like in the following describe ?

I don't think this is useful. If one test fails, it will return something different from 0

albertvaka · 2020-10-09T13:54:52Z

.gitlab-ci.yml

@@ -143,6 +145,9 @@ variables:
 .if_not_test_kitchen_deploy: &if_not_test_kitchen_deploy
  if: $CI_COMMIT_BRANCH != "master" && $CI_COMMIT_TAG == null && $DEPLOY_AGENT != "true" && $CI_PIPELINE_SOURCE != "web" && $CI_PIPELINE_SOURCE != "api"

+.if_test_kitchen_security_agent: &if_test_kitchen_security_agent


Maybe if_agent_security_team_label would be more descriptive.

It is not used any more (I wanted to trigger the kitchen tests based on labels on the PR but they are not available). Removed

lebauce added do-not-merge/WIP changelog/no-changelog team/agent-security labels Sep 7, 2020

lebauce added this to the 7.23.0 milestone Sep 7, 2020

lebauce requested a review from a team as a code owner September 7, 2020 11:37

KSerrania reviewed Sep 7, 2020

View reviewed changes

.gitlab-ci.yml Outdated Show resolved Hide resolved

albertvaka previously approved these changes Sep 7, 2020

View reviewed changes

...test/integration/dd-agent-runtime-security-test/rspec/dd-agent-runtime-security-test_spec.rb Outdated Show resolved Hide resolved

lebauce force-pushed the lebauce/kitchen-functional-tests branch from c8cceb8 to 568a030 Compare September 8, 2020 15:03

lebauce requested a review from a team as a code owner September 8, 2020 15:03

lebauce force-pushed the lebauce/kitchen-functional-tests branch 9 times, most recently from 9173989 to 2a180b6 Compare September 11, 2020 12:27

lebauce removed the do-not-merge/WIP label Sep 11, 2020

lebauce requested review from KSerrania and albertvaka September 11, 2020 14:16

albertvaka approved these changes Sep 14, 2020

View reviewed changes

lebauce force-pushed the lebauce/kitchen-functional-tests branch 7 times, most recently from 034e840 to 222f3ef Compare September 18, 2020 09:52

lebauce force-pushed the lebauce/kitchen-functional-tests branch 3 times, most recently from 3ab077e to 7c37116 Compare October 9, 2020 12:59

guedou approved these changes Oct 9, 2020

View reviewed changes

lebauce force-pushed the lebauce/kitchen-functional-tests branch from 7c37116 to ecaab51 Compare October 9, 2020 13:26

lebauce added 16 commits October 9, 2020 15:51

Add kitchen_functional_tests target to run tests locally with kitchen

96f9b16

Better cleanup in functional tests

c38847c

Updated Mitre tests according to new default policy

a8df470

Do not debug rules evaluation when running tests in verbose mode

c0ba16a

Set the logger once in tests

7f73d5f

Remove UNIX socket when stopping security module

89389c1

Use security_inode_getattr instead of vfs_getattr

3552c2b

Do not put both compat and ia32 kprobes

25eee80

Exclude /proc/*/exe parsing from test logs

8f79d57

Filter on process name for mkdir test

3f460be

Close loop backing file before calling losetup

a1803c8

Do not retry on missing kprobe

ba402db

Use more explicit filenames in tests and improve cleanup

c08a1e4

Do not unlock thread to kill thread that called setreuid

f57f277

Set compatibility mode for utime kprobes

74176fe

Retry when loading BPF modules return EAGAIN

e5d33ac

albertvaka reviewed Oct 9, 2020

View reviewed changes

lebauce and others added 2 commits October 9, 2020 16:21

Reorganize kitchen functional tests to properly cleanup

46bb95f

Use right mnt_id offset on Suse

4ef191d

lebauce force-pushed the lebauce/kitchen-functional-tests branch from ecaab51 to 4ef191d Compare October 9, 2020 14:21

lebauce merged commit d3ee653 into master Oct 9, 2020

lebauce deleted the lebauce/kitchen-functional-tests branch October 9, 2020 18:59

lebauce restored the lebauce/kitchen-functional-tests branch October 15, 2020 13:24

lebauce deleted the lebauce/kitchen-functional-tests branch October 15, 2020 13:24

p-lambert mentioned this pull request Nov 20, 2020

[networks] Add kitchen tests for system-probe #6814

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[runtime-security] Execute runtime security functional tests using Kitchen #6328

[runtime-security] Execute runtime security functional tests using Kitchen #6328

lebauce commented Sep 7, 2020

KSerrania left a comment

albertvaka left a comment •

edited

guedou left a comment

guedou Oct 9, 2020

lebauce Oct 9, 2020

guedou Oct 9, 2020

lebauce Oct 9, 2020

guedou Oct 9, 2020

lebauce Oct 9, 2020

albertvaka Oct 9, 2020 •

edited

lebauce Oct 9, 2020

[runtime-security] Execute runtime security functional tests using Kitchen #6328

[runtime-security] Execute runtime security functional tests using Kitchen #6328

Conversation

lebauce commented Sep 7, 2020

What does this PR do?

Motivation

KSerrania left a comment

Choose a reason for hiding this comment

albertvaka left a comment • edited

Choose a reason for hiding this comment

guedou left a comment

Choose a reason for hiding this comment

guedou Oct 9, 2020

Choose a reason for hiding this comment

lebauce Oct 9, 2020

Choose a reason for hiding this comment

guedou Oct 9, 2020

Choose a reason for hiding this comment

lebauce Oct 9, 2020

Choose a reason for hiding this comment

guedou Oct 9, 2020

Choose a reason for hiding this comment

lebauce Oct 9, 2020

Choose a reason for hiding this comment

albertvaka Oct 9, 2020 • edited

Choose a reason for hiding this comment

lebauce Oct 9, 2020

Choose a reason for hiding this comment

albertvaka left a comment •

edited

albertvaka Oct 9, 2020 •

edited