Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor flags in tracee-rules #500

Merged
merged 13 commits into from Feb 8, 2021
Merged

Refactor flags in tracee-rules #500

merged 13 commits into from Feb 8, 2021

Conversation

grantseltzer
Copy link
Contributor

@grantseltzer grantseltzer commented Feb 2, 2021

This addresses some of the points in #499. In particular:

  1. Removes the --trace-file, and --stdin-as flags.
  2. Adds a new flag called --input-trace that allows the user to specify the input file and input format for events to tracee-rules.

Still need to address:

  1. Need to add unit tests for the above parsing logic
  2. What should the default options be?
  3. Update tracee-rules UX to align with tracee-ebpf #499 (comment)

Signed-off-by: grantseltzer grantseltzer@gmail.com

@grantseltzer grantseltzer added this to the Tracee-Rules MVP milestone Feb 2, 2021
@grantseltzer grantseltzer marked this pull request as ready for review February 3, 2021 02:00
@grantseltzer
Copy link
Contributor Author

grantseltzer commented Feb 3, 2021

Thoughts on default options being reading gob from stdin?

Also for the eof option, should this mean cut the stream/execution once an EOF is reached in the input file?

Copy link
Collaborator

@itaysk itaysk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added some small comments, but the big one is to properly handle EOT flag. I've commented on the issue, let's first make sure you understand the requirement first (this is crucial to how tracee-ebpf->rules work)

tracee-rules/input.go Outdated Show resolved Hide resolved
tracee-rules/main.go Outdated Show resolved Hide resolved
tracee-rules/main.go Outdated Show resolved Hide resolved
tracee-rules/output.go Outdated Show resolved Hide resolved
tracee-rules/input.go Outdated Show resolved Hide resolved
@grantseltzer
Copy link
Contributor Author

@itaysk I addressed your feedback, can you also verify my usage of tracee.Event vs. types.Event. I had to rebase and solve the conflict so just want to make sure I got the idea right of where to use each.

tracee-rules/input.go Outdated Show resolved Hide resolved
tracee-rules/input.go Outdated Show resolved Hide resolved
tracee-rules/input.go Show resolved Hide resolved
tracee-rules/input.go Show resolved Hide resolved
@grantseltzer
Copy link
Contributor Author

I believe that based on the discussion in #508 this is ready for further review and the final question I have is what the default behavior should be for tracee-rules. I think it should be reading gob format from standard input.

@itaysk
Copy link
Collaborator

itaysk commented Feb 4, 2021

@grantseltzer the code in this PR doesn't implement the revised logic suggested in #508 , did you mean to say that it is?

regarding the defaults, if you want to include #508 in this PR, then:

--input-tracee file:{/path/to/file,stdin} (default: stdin)
--input-tracee format:{gob/json}

@grantseltzer
Copy link
Contributor Author

@itaysk I figured I would make that a separate PR but certainly can into this one.

@grantseltzer
Copy link
Contributor Author

I'm actually going to open it as a second PR, the two things are related but it's a different part of the repo.

@itaysk
Copy link
Collaborator

itaysk commented Feb 4, 2021

I figured I would make that a separate PR but certainly can into this one.

no that's fine, I just understood it this way because you mentioned the discussion.

so now, for gob you read until EOT but for JSON you don't. I'm not sure this is what we wanted, but we can let it slide because that's changing soon anyway

tracee-rules/input.go Outdated Show resolved Hide resolved
tracee-rules/input.go Show resolved Hide resolved
tracee-rules/input.go Outdated Show resolved Hide resolved
tracee-rules/main.go Outdated Show resolved Hide resolved
tracee-rules/input.go Show resolved Hide resolved
tracee-rules/input.go Outdated Show resolved Hide resolved
Comment on lines +35 to +40
if opts.inputFormat == jsonInputFormat {
return setupTraceeJSONInputSource(opts)
}
f, err := os.Open(traceeFilePath)
if err != nil {
return nil, fmt.Errorf("invalid file: %s", traceeFilePath)

if opts.inputFormat == gobInputFormat {
return setupTraceeGobInputSource(opts)
Copy link
Member

@simar7 simar7 Feb 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Consider replacing with a switch statement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's all the same to you, I think this looks cleaner than this:

	switch opts.inputFormat {
	case jsonInputFormat:
		return setupTraceeJSONInputSource(opts)
	case gobInputFormat:
		return setupTraceeGobInputSource(opts)
	default:
		return nil, errors.New("could not set up input source")

	}

// TODO: investigate impact of this and research alternatives
time.Sleep(time.Millisecond * 150)
continue
break
} else {
log.Printf("Error while decoding event: %v", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for the input file to be closed when we get here? If that happens we will be stuck in a loop forever.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I realize it's not a change of logic here but it's just me looking at this with a fresh set of eyes)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, that is a good point but i'm not sure how it would?

return nil, errors.New("could not set up input source")
}

func setupTraceeGobInputSource(opts *traceeInputOptions) (chan types.Event, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function tested somewhere else? Maybe we could write a simple test like:

func Test_setupTraceeGobInputSource(t *testing.T) {
	f, err := ioutil.TempFile("", "Test_setupTraceeGobInputSource-*")
	require.NoError(t, err)
	defer func() {
		_ = f.Close()
		_ = os.RemoveAll(f.Name())
	}()

	// write to file
	e := gob.NewEncoder(f)
	event := tracee.Event{
		EventName: "foo",
	}
	require.NoError(t, e.Encode(event))
	f.Seek(0, io.SeekStart)

	opts := traceeInputOptions{
		inputFile:   f,
		inputFormat: gobInputFormat,
	}

	res, err := setupTraceeGobInputSource(&opts)

	require.NoError(t, err)
	assert.Equal(t, "foo", (<-res).(tracee.Event).EventName)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the interest of unblocking Itay I'm going to include this in a follow up PR, but yes you do make a good point, testing these like how you suggested would be a big win.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue created: #517

Copy link
Collaborator

@itaysk itaysk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also update the readme accordingly and squash before you merge

Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
grantseltzer and others added 6 commits February 8, 2021 15:42
…ob from stdin)

Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
Signed-off-by: Simarpreet Singh <simar@linux.com>
Co-authored-by: Simarpreet Singh <simar@linux.com>
Signed-off-by: grantseltzer <grantseltzer@gmail.com>
@grantseltzer grantseltzer merged commit b54cfda into aquasecurity:main Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants