Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add cmdline opts library/initial falco application object #1886

Merged
merged 5 commits into from
Feb 24, 2022
Merged

Add cmdline opts library/initial falco application object #1886

merged 5 commits into from
Feb 24, 2022

Conversation

mstemm
Copy link
Contributor

@mstemm mstemm commented Feb 3, 2022 •
edited by jasondellaluce
Loading

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

If contributing rules or changes to rules, please make sure to also uncomment one of the following line:

/kind rule-update

/kind rule-create

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area rules

/area tests

/area proposals

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

refactor(userspace/falco): replace direct getopt_long() cmdline option parsing with third-party cxxopts library.

@mstemm mstemm force-pushed the add-cmdline-opts-library branch 2 times, most recently from 38d7bd3 to aaf2447 Compare February 3, 2022 00:30
leogr
leogr reviewed Feb 3, 2022
View reviewed changes
Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a quick look. I like the idea, but I don't know cxxopts, so I need some time to play with it a bit :)

Meanwhile, I have left some (not really important) suggestions.

cmake/modules/cxxopts.cmake Outdated Show resolved Hide resolved
userspace/engine/falco_engine.cpp Outdated Show resolved Hide resolved
userspace/falco/application.cpp Outdated Show resolved Hide resolved
userspace/falco/application.h Outdated Show resolved Hide resolved
jasondellaluce
jasondellaluce reviewed Feb 3, 2022
View reviewed changes
Copy link
Contributor

@jasondellaluce jasondellaluce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left you some comments. I like the direction this is going to!

userspace/falco/app_cmdline_options.cpp Outdated Show resolved Hide resolved
userspace/falco/app_cmdline_options.h Show resolved Hide resolved
userspace/falco/application.h Outdated Show resolved Hide resolved
userspace/falco/application.h Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
userspace/falco/application.h Show resolved Hide resolved
userspace/falco/application.h Show resolved Hide resolved
userspace/falco/falco.cpp Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
@mstemm mstemm force-pushed the add-cmdline-opts-library branch 2 times, most recently from bc552a8 to 84a1b27 Compare February 3, 2022 23:05
@mstemm
Copy link
Contributor Author

mstemm commented Feb 4, 2022 •
edited
Loading

One public-visible change is that --list, which used to support an optional argument, has been broken into two options. --list now always displays all fields, and a new option --list-source displays fields for a specific source. This is because the cxxopts library has a bug (jarro2783/cxxopts#104, jarro2783/cxxopts#211) that does not parse implicit options correctly when specified as "--list ".

Edit: No longer true, we'll keep the single option and users will have to use --list=<source> to specify a source, as current falco does.

jasondellaluce
jasondellaluce reviewed Feb 9, 2022
View reviewed changes
test/falco_tests.yaml Outdated Show resolved Hide resolved
userspace/falco/app_cmdline_options.cpp Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
userspace/falco/falco.cpp Show resolved Hide resolved
@mstemm
Add cxxopts command line parsing library
928299a 
We'll use this to better manage the fairly large set of command line
options in self-contained objects instead of a scattering of
individual stack variables.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
@mstemm
Add notion of falco application object/cmdline opts skeleton
08ab8de 
Add a notion of a falco application object. Eventually this will
replace the bulk of falco_init and contain methods to:

- Parse/validate command line options
- Parse/validate falco config
- Initialize prerequsites (inspector, falco engine, webserver, etc)
- Load plugins
- Load/validate rules
- Command/subcommand execution (e.g. --list/--list-fields, or
  nothing specified to run "main" loop)

For now, it is only responsible for command line options handling,
which is stubbed out.

Currently, the only public methods are init() to initialize everything
and copts() to access command line options.

Command line options are held in a different class
falco::app::cmdline_opts. application::copts() returns a reference to
that object, which allows access to parsed command line options bound
to various public instance variables.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
@mstemm
Copy link
Contributor Author

mstemm commented Feb 23, 2022

Furthermore, regarding your comment 👇

One public-visible change is that --list, which used to support an optional argument, has been broken into two options. --list now always displays all fields, and a new option --list-source displays fields for a specific source. This is because the cxxopts library has a bug (jarro2783/cxxopts#104, jarro2783/cxxopts#211) that does not parse implicit options correctly when specified as "--list ".
....
N.B.: I have tried it a bit, it seems that it perfectly emulates the behavior of the latest released version of Falco. Let me know if it works for you too!!!

Thanks for doing the extra debugging and the patch! I went back to a single option, where --list=<source> only works with equals.

@mstemm
Initial falco::app::cmdline_options class
4db8069 
Fill in an initial falco::app::cmdline_options class using cxxopts
library to hold options:

- falco::app::cmdline_options contains a cxxopts::Options object to
  parse options and a cxxopts::ParseResult to hold the result.
- The only meaningful public method is parse() which parses argc/argv
  and returns true/false + error.
- The parsed options are all public instance variables of the object
  and generally use the same names of the corresponding variables in
  the old falco_init(). These variables are all bound to the
  corresponding command line option and are updated in parse().
- In a few cases, the command line option does not directly map to a
  bound variable (e.g. -b to set buffer format, -p/-pk/-pc to set
  extra formatting options, etc.) In these cases the option values are
  read after parsing and update the public instance variable.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
@mstemm
Small changes to falco engine/config wrt new cmdline option parsing
7e1eaa3 
For the most part, replacing getopt() with cxxopts + falco application
had no effect on falco engine/config interfaces. However, there were a
few places where it was wasier to change the interface than add
middleware code that transformed from, for example, vectors to lists.

This commit has those changes.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
@mstemm mstemm force-pushed the add-cmdline-opts-library branch 2 times, most recently from cabfb62 to 3aeba3e Compare February 23, 2022 02:00
jasondellaluce
jasondellaluce reviewed Feb 23, 2022
View reviewed changes
Copy link
Contributor

@jasondellaluce jasondellaluce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all the comments, this looks much more solid to me! Left you some comments/concerns. I'm not super-strong about those, but I wanted to have some feedback from you on those points.

userspace/falco/app_cmdline_options.cpp Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
userspace/falco/falco.cpp Outdated Show resolved Hide resolved
@mstemm
Update falco main to use falco application + cmdline_opts
5a0e7b9 
Update falco's main falco_init() to use a falco::app::application and
falco::app::cmdline_opts object instead of storing all its command
line state in stack variables.

The bulk of the removed code is in usage() (not needed as cxxopt's
help() is self-documenting.) and getopt_long() which is replaced by
app.init(argc, argv).

For the most part, this is simply replacing references to local
variables (e.g. "all_events") to the bound variable inside the
cmdline_opts object (e.g. app.copts().all_events).

There are a few cases where more complex logic was used (output
formats, initializing k8s/mesos with string pointers), and those
changes are still in falco_init().

For the most part, the monolithic parts of falco_init that involve
reading config files, creating the inspector, loading rules, etc are
still present. Those will be addressed in later changes.

Signed-off-by: Mark Stemm <mark.stemm@gmail.com>
@leogr
Copy link
Member

leogr commented Feb 24, 2022

/milestone 0.32.0

@poiana poiana added this to the 0.32.0 milestone Feb 24, 2022
leogr
leogr approved these changes Feb 24, 2022
View reviewed changes
Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now.

Thank you, Mark!

/approve

@poiana poiana added the lgtm label Feb 24, 2022
@poiana
Copy link

poiana commented Feb 24, 2022

LGTM label has been added.

Git tree hash: 4557d2fd39e24792b9e5037da6178965c7a1e4f2

jasondellaluce
jasondellaluce approved these changes Feb 24, 2022
View reviewed changes
Copy link
Contributor

@jasondellaluce jasondellaluce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

🥳

@poiana
Copy link

poiana commented Feb 24, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jasondellaluce, leogr, mstemm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [jasondellaluce,leogr,mstemm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana merged commit 8335398 into master Feb 24, 2022
@poiana poiana deleted the add-cmdline-opts-library branch February 24, 2022 09:40
@jasondellaluce jasondellaluce modified the milestones: 0.32.0, 0.31.1 Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LucaGuerra LucaGuerra LucaGuerra left review comments

@leogr leogr leogr approved these changes

@jasondellaluce jasondellaluce jasondellaluce approved these changes

@mfdii mfdii Awaiting requested review from mfdii

Assignees

@leogr leogr

@jasondellaluce jasondellaluce

Labels
approved area/engine dco-signoff: yes kind/cleanup lgtm release-note size/XXL
Projects
None yet
Milestone
0.31.1
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@mstemm @leogr @poiana @jasondellaluce @LucaGuerra