Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/packetbeat: Add packetbeat module #97152

Open
wants to merge 2 commits into
base: master
from

Conversation

@lejonet
Copy link
Contributor

@lejonet lejonet commented Sep 4, 2020

Motivation for this change

Adds a packetbeat module so that it can be configured with
freeform settings. Made with big help from @aanderse and
@Infinisil.

We have the packetbeat package in nixpkgs for some time, but no module, so I decided to make one. It uses the freeform module concept.

Remade PR after screwing up the prior PR with a botched rebase.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
Adds a packetbeat module so that it can be configured with
freeform settings. Made with big help from @aanderse and
@Infinisil.
@lejonet
Copy link
Contributor Author

@lejonet lejonet commented Sep 4, 2020

@Infinisil here's the config again:

packetbeat = {
    enable = true;
    package = pkgs.packetbeat7;
    settings = {
      name = config.networking.hostName;
      packetbeat = {
        interfaces = {
          device = "ens3";
          type = "af_packet";
          ignore_outgoing = true;
        };
        protocols = {
          tls = {
            ports = [ 443 993 995 5223 8443 8883 9243 ];
            send_certificates = true;
            include_raw_certificates = false;
            include_detailed_fields = true;
            fingerprints = [ "md5" "sha1" "sha256" ];
          };
        };
      };
      output = {
        elasticsearch = lib.mkForce {};
        file = {
          path = "/var/log/packetbeat";
          filename = "packetbeat.log";
          rotate_every_kb = "102400";
          number_of_files = "20";
        };
      };
      fields = {
        env = "utility";
      };
      processors = [ "add_host_metadata: ~" ];
    };
  };
protocols = mapAttrs (name: mkDefault) {
icmp = {
enabled = true;
};
amqp = {
ports = [ 5672 ];
};
cassandra = {
ports = [ 9042 ];
};
dhcpv4 = {
ports = [ 67 68 ];
};
dns = {
ports = [ 53 ];
};
http = {
ports = [ 80 8080 8000 5000 8002 ];
};
memcache = {
ports = [ 11211 ];
};
mysql = {
ports = [ 3306 3307 ];
};
pgsql = {
ports = [ 5432 ];
};
redis = {
ports = [ 6379 ];
};
thrift = {
ports = [ 9090 ];
};
mongodb = {
ports = [ 27017 ];
};
nfs = {
ports = [ 2049 ];
};
tls = {
ports = [ 443 993 995 5223 8443 8883 9243 ];
};
};
Comment on lines +160 to +203

This comment has been minimized.

@lejonet

lejonet Sep 5, 2020
Author Contributor

@Infinisil As we discussed on IRC yesterday, apparently this "dictionary-style" (aka attrset of attrsets in Nix) is deprecated by packetbeat and packetbeat wants it to be a list of attrsets. So technically it should be easy to change this attrset of attrsets into a list of attrsets, but as you noted, that will make it immutable once set, so this mean that we somehow have to take the attrset of attrsets and convert it into a list of attrsets, to keep the possibility of the user to actually change this, while still conforming to the wishes of packetbeat.

This comment has been minimized.

@Infinisil

Infinisil Sep 15, 2020
Member

Do you have a link to where this deprecation is explained?

This comment has been minimized.

@lejonet

lejonet Sep 16, 2020
Author Contributor

I will see what I can find, so far, the only hint I've gotten was a warning in the log when running packetbeat with the generated config.

This comment has been minimized.

@lejonet

lejonet Sep 16, 2020
Author Contributor

The only reference I can find is this github issue: elastic/beats#3518, the dictionary style should've been removed in 7.0.0 but for some reason it still works, but throws that warning message. It was apparently deprecated already by 6.0.0. Apparently it was done so that you can define the same protocol analyzer several times, to get different settings, i.e. if you want one for regular HTTP and one for REST endpoints, like elasticsearch endpoint and so, which the dictionary style didn't allow because of merging.

This comment has been minimized.

@lejonet

lejonet Sep 16, 2020
Author Contributor

This comment has been minimized.

@lejonet

lejonet Sep 16, 2020
Author Contributor

Might there be some way that we can ask the yaml generator to generate a list of objects instead of a dictionary from our attrset of attrsets? or some type of trickery with map?

@aanderse
Copy link
Contributor

@aanderse aanderse commented Sep 15, 2020

@lejonet I think this looks good 👍 I'll defer to @Infinisil for final approval and merge, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.