Remove PF_Ring Submodule #57
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
PF_Ring provides packaged installation options from 7.5 onward. Current default installed by apt is 7.8 Tested successful station compilation:
|
…n a null pointer for registration source
…PI (#59) * updated wrapping struct for clientToStations and added a flag to registrations for presccanned * rust compilation issue fixed
…were butchering before (#60)
…o prevent compile time warnings
* Update detector to send source/decoy addresses to the application over the C2S struct * log addresses during new registration in application * implementatoin, but not integrated yet. * updated addrs, added small tests, implemented check in registration pipeline. * added checks for nil pointers to prevent crashes and switched to checking Covert which is what this should have been from the start. * rust compile warning fixes. * added logging for registrations dropped by blocklist. parse hostport covert format which should always be received from client
* Implemented and tested in golang tests, not yet tested in staging * added validity tag and tracking function to registration manager to track (and check tracking of) new registrations immediately * fixed mutex deadlock created by calling registrationExists from register, minor fixes. tested in staging - working
* add option to moderate client ip logging, default off * added option parse and client ip logging in registration API * fixed mistake and removed last client IPs and covert logging * race condition in registration tracking resulted in small data structure refactor. All regs tracking and retiring properly
…stration tracking
Although we don't technically care about closing the sockets (as the program will run until either all sockets fail or the process dies), the Go GC will collect the sockets if there isn't some reference to them; these defers retain that reference so they aren't collected.
* update to prevent zmq_proxy crash and to limit concurrent TCP socket use by liveness tests. also allows disable of v4 or v6 to limit unused registrations * fixes deadlock in most conditions. seems to still lock when all TCP sockets get used up. * dial liveness with timeout so that connections close and goroutines return in an expected amount of time.
* change removeOldRegistrations to only block intermittently instead of whollistically * tested working on single station staging * typo
* limit connection logging, and limit redis reconnections * update covert Blocklist to include domains and valid addresses for golang TCP dial * exercising redis usage through multithreaded testing * updated pubsub send on golang application side and pubsub receive on detector side with tests * checkpoint * validated reg API client address handling behavior with testing * updated tests and client address handling in application * If client registers with v6, only create and track registrations for IPv6 * prevent client Address logging and only add v6 registration when client registers using v6 * ensure that a client who registers with v4 will propogate registrations for both v4 and v6, but a client who registers with v6 only propogates v6 * client ip logging based on env var in session logging * invalidate registrations sent over v6 if they pick a v4 phantom address * small enhancements to logging * added loging if zmq_proxy escapes work loops
* Fix ZMQ race condition from proxy * Add test for concurrency on ZMQ proxy
* Make halfPipe log both up and down stats * Add stats tracking and periodic reporting
* Make halfPipe log both up and down stats * Add stats tracking and periodic reporting * break up read/writes so we can track bytes as they are transferred, not just at the end of the connection * track new instead of absolute local/api regs * Correct detection for local registrations Co-authored-by: Jack Wampler <jmwample@users.noreply.github.com>
* add external file to manage subnet generations so we don't have to rebuild the station to update subnets in use. * parsing error in V6Only subnet functor filter * parsing error in V6Only subnet functor filter * more test cleaning * more small testing fixes
… registration in ipv4 (#80)
* Added default value for PHANTOM_SUBNET_LOCATION environment variable * Copy test phantom_subnets.toml file to $PHANTOM_SUBNET_LOCATION path * Added mention of PHANTOM_SUBNET_LOCATION into README.md * Added a bind mount for a local copy of the phantom_subnets.toml * Included a copy of the test phantom_subnets.toml file into docker directory
* parsing error in V6Only subnet functor filter * quick fix for types missed in #76
|
Removing test release as a requirement to get this merged as that seems like it will be a bigger effort and is relatively independent from this PR. |
Open
…with rust 1.52.1 - temporary fix until protoc has a more stable update (#89)
* Modified .dockerignore to improve effectiveness of docker caching * Changed Dockerfile to use ntop pfring package * Added a script to add ntop repository and install pfring and ZC on the host system (Debian and Ubuntu) * Update docker default phantom_subnets.toml file * Added some sanity checks into zbalance container * Check if hugepages number is 512 * Check if ZC driver is loaded * If check not passed show some hints what can be wrong
…ars to conjure.conf, and updated det service to correct bin name
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Using PF_Ring as a submodule and compiling the detector using static libraries is limiting.
Solution
Install PF_Ring as a dependency on the station and then link against libpfring/libpcap shared object files. This way we can more easily package and distribute station code without managing pf_ring kernel module.
TODO
.github/workflows/build.ymlto use new build process.dark-decoytoconjureCreate a test release