Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stdin driver with file opener refactors #1605

Merged
merged 54 commits into from
Aug 11, 2017
Merged

Stdin driver with file opener refactors #1605

merged 54 commits into from
Aug 11, 2017

Commits on Aug 8, 2017

  1. logproto: document the layering violation of log_proto_server_get_fd() 

    Earlier only a note was saying that get_fd() is a layering violation. The
reason is that the presence of this function exposes that all LogProto
instances are based on a UNIX fd. This is always true as of now, but
still not nice. It would be a lot better to replace log_proto_server_get_fd()
with APIs that abstract away fds.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    36213cf View commit details
    Browse the repository at this point in the history

  2. logproto: increase LOG_PROTO_SERVER_OPTIONS_SIZE to 128 

    This constant contains the number of bytes allocated
in LogProtoServerOptionsStorage, so that derived classes may add new fields
in their own Options structure.

Actually, "32" the original value is smaller than LogProtoServerOptions,
e.g. no extra bytes were allocated.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    96cdbbb View commit details
    Browse the repository at this point in the history

  3. logproto: publish log_proto_text_server_validate_options_method() 

    In order to make this available for derived classes. For now it's just
an alias for the one in buffered server.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    cd4806d View commit details
    Browse the repository at this point in the history

  4. logproto: add options validation in LogProtoRegexpMultiLineServer 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    15bdb95 View commit details
    Browse the repository at this point in the history

  5. logreader: make handling of LPS_EOF in fetch_log a bit easier to read 

    I find it is a bit easier to read this way.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    d3cfa2e View commit details
    Browse the repository at this point in the history

  6. affile: fix "none" stats counter registration 

    log_reader_reopen() was once also doing log_reader_set_options(), which at
that point may have meant changing the stats_source/stats_level values.

Then, reopen and set_options were divorced, however the
log_source_init()/deinit() calls remained, which are not really good at
multiple fronts (at first it reinialized part of the LogReader, but not the
whole).

Anyway, because of this, a stats counter might be registered "none" as the
first log_reader_reopen() is called before set_options, thus stats_source is
not yet set.

To fix this issue, get rid off the superfluous init/deinit calls, which are
not needed.

I also tried to decouple reopen/set_options from the ordering issues, but I
miserably failed, and I didn't want to go there with 50+ patches already in
my series.

Would be great if we could remove this ordering dependency at one point in
the future.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    b53bc72 View commit details
    Browse the repository at this point in the history

  7. cfg: add a function call to request shutdown of a GlobalConfig instance 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    89daead View commit details
    Browse the repository at this point in the history

  8. transport: move /proc/kmsg related LogTransport to affile 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    b2e25f5 View commit details
    Browse the repository at this point in the history

  9. affile: change license to transport-prockmsg to GPL 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    5bb25e4 View commit details
    Browse the repository at this point in the history

  10. transport: publish read/write method implementations 

    Instead of keeping these static, make them available for derived classes.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    623e01c View commit details
    Browse the repository at this point in the history

  11. affile: extract log_transport_devkmsg_new() 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    e12e080 View commit details
    Browse the repository at this point in the history

  12. affile: extract log_transport_followed_file()

    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    e93f28b View commit details
    Browse the repository at this point in the history

  13. affile: extract log_transport_source_named_pipe_new() 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    bb296ba View commit details
    Browse the repository at this point in the history

  14. affile: rename affile-common to file-opener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    8c9461e View commit details
    Browse the repository at this point in the history

  15. affile: embed FilePermOptions into FileOpenOptions 

    As the next step creating the FileOpener interface, encapsulate
FilePermOptions into FileOpenerOptions.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    85eb3de View commit details
    Browse the repository at this point in the history

  16. affile: FileReaderOptions init/deinit should encapsulate all embedded… 

    … Options

FileReaderOptions incrporates a number of Options structure, yet it
was missing the init/deinit/defaults interface, so all callers
were explicitly poking into the struct, intializing members. This
is very error prone and this patch introduces the
file_reader_options_init/deinit functions, so that everything
underneath is hidden.

It removes code duplication between affile-source and wildcard-source
and makes the code a lot easier to follow.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    fc8c5ec View commit details
    Browse the repository at this point in the history

  17. affile: move FileReaderOptions instance to a constructor parameter 

    Instead of poking this value right into the FileReader instance,
pass it as a parameter.

Also, this patch renames the longish file_reader_options in FileReader
to merely "options" making the code easier to read.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    78b09be View commit details
    Browse the repository at this point in the history

  18. affile: added some whitespace to wildcard-source 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    ad83fc3 View commit details
    Browse the repository at this point in the history

  19. affile: rename FileOpenOptions to FileOpenerOptions to consistency 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    e19da44 View commit details
    Browse the repository at this point in the history

  20. affile: create FileOpener interface, decouple FileReader from FileOpener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    006716e View commit details
    Browse the repository at this point in the history

  21. affile: remove an unneeded indirection from opening files in file-reader 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    36d382d View commit details
    Browse the repository at this point in the history

  22. affile: pass stats_source as a member in FileReaderOptions 

    Instead of passing using the "is_pipe" value, expect the caller
to specify the stats source explicitly.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    68a2f52 View commit details
    Browse the repository at this point in the history

  23. affile: keep opener instances persistent, so construction can happen … 

    …earlier

This patch moves the creation of FileOpener instances into the respective
constructors, so that higher level constructors can pass in
different FileOpener instances.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    313c728 View commit details
    Browse the repository at this point in the history

  24. affile: migrate affile-dest to the FileOpener interface 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    8662fa5 View commit details
    Browse the repository at this point in the history

  25. affile: delegate transport construction to FileOpener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    26ad287 View commit details
    Browse the repository at this point in the history

  26. affile: add "self" to file-opener methods 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    df503cf View commit details
    Browse the repository at this point in the history

  27. affile: add "prepare_open" virtual method 

    This method can be used to prepare for an upcoming open call, for things
such as the creation of a named pipe, etc.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    b9a9531 View commit details
    Browse the repository at this point in the history

  28. affile: move mkfifo() call to named-pipe.c 

    With the new prepare-open call, we can remove a couple of pipe
related conditionals, such as the creation of the named pipe.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    54bc22f View commit details
    Browse the repository at this point in the history

  29. affile: move pipe/file mismatch to named-pipe 

    This is another pipe related specialization that can be delegated to
named-pipe.c

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    ede1469 View commit details
    Browse the repository at this point in the history

  30. affile: rename followed-files to regular-files 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    1da77c4 View commit details
    Browse the repository at this point in the history

  31. affile: move pipe/file mismatch check to regular-files 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    b7044af View commit details
    Browse the repository at this point in the history

  32. affile: remove _validate_file_type() 

    The content of this function has moved to various prepare_open() methods,
thus became empty. Remove the dead code.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    93ef769 View commit details
    Browse the repository at this point in the history

  33. affile: delegate src proto construction to FileOpener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    f7f39f4 View commit details
    Browse the repository at this point in the history

  34. affile: use construct_src_proto to create kmsg related protocols 

    This makes the file reader a lot simpler as it does not need /proc/kmsg
and /dev/kmsg specific code.

This patch also gets rid off the separate logproto-linux-proc-kmsg-reader
module.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    b2a8722 View commit details
    Browse the repository at this point in the history

  35. affile: remove "is_pipe" dependency from affile-source 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    5e07e50 View commit details
    Browse the repository at this point in the history

  36. affile: add "init_instance" method to FileOpener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    f3901d1 View commit details
    Browse the repository at this point in the history

  37. affile: delegate construct_dst_proto to FileOpener & get rid off is_pipe 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    4dfbbc9 View commit details
    Browse the repository at this point in the history

  38. logwriter: move stats_level/source to LogWriterOptions 

    Instead of getting this via the log_writer_set_options() call,
get them via LogWriterOptions.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    c4f276f View commit details
    Browse the repository at this point in the history

  39. logreader: move stats_level/source to LogReaderOptions 

    Instead of getting this via the log_reader_set_options() call,
get them via LogReaderOptions.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    75f6037 View commit details
    Browse the repository at this point in the history

  40. logreader: clarify ordering requirements between log_reader_reopen/se… 

    …t_options

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    e0ebdac View commit details
    Browse the repository at this point in the history

  41. affile: fix the test program

    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    dd99d97 View commit details
    Browse the repository at this point in the history

  42. affile: add direction argument to file_opener_open_fd() 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    fd4e409 View commit details
    Browse the repository at this point in the history

  43. affile: delegate open_flags to the FileOpener implementation 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    5251d88 View commit details
    Browse the repository at this point in the history

  44. affile: rename test_affile_open_file to test_file_opener 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    e91ca0c View commit details
    Browse the repository at this point in the history

  45. logproto: add LogProtoMultiLineServer 

    This new class consolidates multi-line options into a class, that makes
it easy to embed multi-line support in any LogReader based sources.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    c8b0e1f View commit details
    Browse the repository at this point in the history

  46. affile: add LogProtoFileReader 

    In a similar spirit to LogProtoFileWriter, we introduce a LogProtoFileWriter,
which is basically a set of options and a constructor that
creates the appropriate LogProtoServer instance based on these options.

Somewhat unfortunately FileReader and LogProtoFileReader are pretty similar
names, which could be fixed by changing the FileReader name to something
like FileReaderPipe, or AFFileSourceReader (akin to AFFileDestWriter).

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    1ebb85c View commit details
    Browse the repository at this point in the history

  47. affile: parse multi-line & padding options into LogProtoServerOptions 

    Instead of putting these into variables here and there, but them where
they belong, the Options structure of our LogProtoServer instance.

Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    ed9db00 View commit details
    Browse the repository at this point in the history

  48. affile: make the open() method virtual 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    c94519d View commit details
    Browse the repository at this point in the history

  49. affile: add stdin() driver 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    a9e6ef1 View commit details
    Browse the repository at this point in the history

  50. affile: move afpipe_sd_new() to named-pipe 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    06c4e1b View commit details
    Browse the repository at this point in the history

  51. affile: move afpipe_dd_new() to named-pipe 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    679ba57 View commit details
    Browse the repository at this point in the history

  52. affile: improve _setup_logreader() whitespace 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    c74c53c View commit details
    Browse the repository at this point in the history

  53. afprog: minor readability fixes 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    2f7b0b4 View commit details
    Browse the repository at this point in the history

  54. affile: fix up CMakeLists 

    Signed-off-by: Balazs Scheidler <balazs.scheidler@balabit.com>
    @bazsi
    bazsi committed Aug 8, 2017
    Configuration menu
    Copy the full SHA
    0ef0aa2 View commit details
    Browse the repository at this point in the history