Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
c87d765
merge 3.2.0 release, go 3.3.0-develop
weyrick Apr 16, 2021
fb1676e
add -L to curl AppImage cmd
weyrick Apr 16, 2021
862ba62
rfcs
weyrick Apr 16, 2021
f6709aa
rfcs
weyrick Apr 16, 2021
75e803c
rfcs
weyrick Apr 16, 2021
47cfe3b
readme updates
weyrick Apr 19, 2021
e0f44f2
Merge branch 'develop' into feature/rfcs
weyrick Apr 19, 2021
ac27556
Merge pull request #82 from ns1labs/feature/rfcs
weyrick Apr 19, 2021
c2a513f
set up yaml configuration support
weyrick Apr 19, 2021
f773920
yaml configuration
weyrick Apr 20, 2021
eadd424
break out configurable. add taps. start checking names/labels with regex
weyrick Apr 20, 2021
8005aae
some more abstraction for taps. take module life cycle (start/stop) o…
weyrick Apr 20, 2021
f72b86d
naming of modules and plugins, tap config work
weyrick Apr 20, 2021
dbcc185
get to handle on corrade plug in names and aliases
weyrick Apr 20, 2021
9cd6687
explicitly include fmt. add strict taps
weyrick Apr 20, 2021
118257d
Configurable from yaml
weyrick Apr 20, 2021
8e2dcd6
switch type to input_type. add taps endpoint to admin api
weyrick Apr 21, 2021
e749156
refactor CoreManagers out of CoreServer. add unit tests for taps
weyrick Apr 21, 2021
5206763
Merge pull request #86 from ns1labs/feature/taps
weyrick Apr 21, 2021
1cd2937
initial policy manager
weyrick Apr 21, 2021
d8445b3
refactor plugin and module interface
weyrick Apr 21, 2021
3243525
rename to registry
weyrick Apr 21, 2021
50fdc79
store corrade plugins by name
weyrick May 14, 2021
5037e13
update slack invite link
weyrick May 17, 2021
e27448b
go 3.2.1-develop
weyrick May 25, 2021
68e6236
fix #98 remove 300 message. in addition, fix erroneous pcpp::DnsLayer…
weyrick May 25, 2021
9fbf73f
issue #100 add tcp reassembly error tracking
weyrick May 26, 2021
fe107bd
pcap stats
weyrick May 26, 2021
f3a9255
pcap stats handler implementation
weyrick May 26, 2021
f4e976b
revert changes to nethandler
weyrick May 26, 2021
dcdfc26
add pcap stream handler to default pktvisord capture
weyrick May 26, 2021
98d2168
add pcap handler metrics to cli, make time out 30s
weyrick May 26, 2021
4c2ccf2
bump pcapplusplus to 21.05 release
weyrick May 26, 2021
e8e5967
issue #94 add TLS support to web server
weyrick May 26, 2021
cf5a487
fix overzealous refactor
weyrick May 26, 2021
21a01bc
address issue #84, improve flushing and error messages. change to DAE…
weyrick Jun 1, 2021
718c8be
fix typo
weyrick Jun 1, 2021
056a664
address #83, improved daemonize and logging.
weyrick Jun 1, 2021
2babf68
address #83, more improved daemonize and logging.
weyrick Jun 1, 2021
638a36b
better error handling and documentation, esp. TLS config
weyrick Jun 2, 2021
4e313f1
update download link in read me
weyrick Jun 2, 2021
10d1d01
Merge branch 'develop' into feature/policies
weyrick Jun 2, 2021
c05d25a
Merge branch 'release' into feature/3.2.1-merge
weyrick Jun 2, 2021
9ba36d4
3.2.1 merge fixes
weyrick Jun 2, 2021
4b37fe1
Merge pull request #102 from ns1labs/feature/3.2.1-merge
weyrick Jun 3, 2021
91ccac8
Merge branch 'develop' into feature/policies
weyrick Jun 3, 2021
3df9984
3.2.1 merge fixes
weyrick Jun 3, 2021
bbe3530
pcap comment, policy parse
weyrick Jun 7, 2021
fbf3bf1
add TLS support to CLI
weyrick Jun 7, 2021
17455c0
policy work
weyrick Jun 7, 2021
76b481e
policy and tap work. add stand alone Config class
weyrick Jun 8, 2021
ce3f5eb
policy and tap work
weyrick Jun 8, 2021
1bd789c
add mock input
weyrick Jun 8, 2021
3034d0f
policy work
weyrick Jun 9, 2021
68d6ca5
policy work, stream handler refactor
weyrick Jun 9, 2021
802cb48
functioning policies
weyrick Jun 9, 2021
3836eee
policy testing
weyrick Jun 9, 2021
552140b
policy testing and refactor fixes
weyrick Jun 9, 2021
08fa850
remove module specific create and delete http end points
weyrick Jun 9, 2021
dbf2d32
docs and parse policy on cmd line
weyrick Jun 9, 2021
f3e9489
begin using policy for default command line
weyrick Jun 10, 2021
86f9937
handle pcap stats monotonic counter at each new bucket.
weyrick Jun 10, 2021
f90f2b3
default policy from commandline. policy and module lifecycle.
weyrick Jun 10, 2021
368b713
change 3.0.x API metric iface to use "default" policy
weyrick Jun 10, 2021
7c0f65e
policies end point
weyrick Jun 11, 2021
12c18d6
individual policy end point
weyrick Jun 11, 2021
e053a54
refactor, prep for prometheus per policy
weyrick Jun 11, 2021
2655717
propagate add_labels, add policy label to prometheus
weyrick Jun 11, 2021
f45c9bf
policy json metrics
weyrick Jun 11, 2021
60f4b80
create policies through admin api.
weyrick Jun 11, 2021
081893f
remove plug in end points for now
weyrick Jun 11, 2021
9dd19b6
hoist period key out of metric json and into 3.0.x compat output only…
weyrick Jun 11, 2021
9069f57
add handler "module" label to policy prometheus output
weyrick Jun 11, 2021
9cd95db
policy iteration
weyrick Jun 11, 2021
75878db
fix policy units wrt exception string and lifecycle
weyrick Jun 11, 2021
e4461e2
Merge pull request #91 from ns1labs/feature/policies
weyrick Jun 11, 2021
52f2dd1
Merge branch 'release' into develop
weyrick Jun 13, 2021
12becf7
release 3.2.1 to develop (#103)
weyrick Jun 13, 2021
42808e5
Merge remote-tracking branch 'origin/develop' into develop
weyrick Jun 13, 2021
7adc843
update slack link (perm)
weyrick Jun 15, 2021
5697fbe
implement #78 dns filters (#105)
weyrick Jun 15, 2021
c8977e4
fix error message. default empty bpf config
weyrick Jul 13, 2021
46bc07d
switch random number generators used by sampling (#110)
weyrick Jul 14, 2021
4ec5025
run-pktvisor.sh IFS fix for args with spaces (#111)
rlahnemann Jul 15, 2021
57f6f88
Create CODE_OF_CONDUCT.md
iatdaitan Aug 25, 2021
d85137e
Merge pull request #113 from ns1labs/add-code-of-conduct
iatdaitan Aug 25, 2021
94fec6b
Feature/rfc77 update (#115)
weyrick Aug 30, 2021
29277df
s/name/title in RFC77
weyrick Sep 1, 2021
a9e4f7c
change pcapplusplus dependency back to ns1 fork
weyrick Sep 1, 2021
3f2d4fb
address corrade ci failure (#116)
weyrick Sep 2, 2021
e7e7c30
require input_type to be defined on both tap and policy input tap, an…
weyrick Sep 13, 2021
3bccb19
require a new key 'kind' to specify policy type, in preparation for f…
weyrick Sep 14, 2021
cd0fed1
feature/cli refactor (#120)
weyrick Sep 17, 2021
9ba68be
#121 remove policy (#122)
weyrick Oct 4, 2021
b42f76c
begin requiring NS1 fork of PcapPlusPlus to use PcapLiveDevice constr…
weyrick Oct 4, 2021
a1025d4
support string lists in yaml config. also add filtered count to ui. (…
weyrick Oct 5, 2021
a945187
missing yaml string list test
weyrick Oct 5, 2021
250cbd0
fix dns rates in ui (#126)
weyrick Oct 7, 2021
33b22c0
Fix Corrade detection and build failures with GCC 11 (#130)
fcelda Oct 22, 2021
f6ae08d
DHCP handler module (#131)
weyrick Oct 25, 2021
1cf15a2
reflection RFC / policies iteration (#132)
weyrick Oct 28, 2021
97f46dd
Feature/133 mock traffic (#134)
weyrick Nov 1, 2021
315dd4b
get all policies at once (#136)
weyrick Nov 2, 2021
c27bc55
check for duplicate policy sooner (#137)
weyrick Nov 3, 2021
4134fb9
simplify docker tag and push (#138)
weyrick Nov 9, 2021
d421cd6
include list of available interfaces in info json and error msgs (#139)
weyrick Nov 10, 2021
0ebd07f
bump 3.3.0-rc
weyrick Nov 15, 2021
138e061
Merge branch 'master' into release
weyrick Nov 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 6 additions & 21 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -177,35 +177,20 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build + push VERSION - pktvisor
- name: Build + push - pktvisor
env:
IMAGE_NAME: ns1labs/pktvisor
run: |
docker build . --file docker/Dockerfile --tag ${{ env.IMAGE_NAME }}:${{ env.VERSION }}
docker push ${{ env.IMAGE_NAME }}:${{ env.VERSION }}
docker build . --file docker/Dockerfile --tag ${{ env.IMAGE_NAME }}:${{ env.VERSION }} --tag ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}
docker push -a ${{ env.IMAGE_NAME }}

- name: Tag + push docker image with ref tag (cached build) - pktvisor
env:
IMAGE_NAME: ns1labs/pktvisor
run: |
docker build . --file docker/Dockerfile --tag ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}
docker push ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}

- name: Build + push VERSION - pktvisor-prom-write
env:
IMAGE_NAME: ns1labs/pktvisor-prom-write
working-directory: ${{github.workspace}}/centralized_collection/prometheus/docker-grafana-agent
run: |
docker build . --file Dockerfile --build-arg PKTVISOR_TAG=${{ env.REF_TAG }} --tag ${{ env.IMAGE_NAME }}:${{ env.VERSION }}
docker push ${{ env.IMAGE_NAME }}:${{ env.VERSION }}

- name: Tag + push docker image with ref tag (cached build) - pktvisor-prom-write
- name: Build + push - pktvisor-prom-write
env:
IMAGE_NAME: ns1labs/pktvisor-prom-write
working-directory: ${{github.workspace}}/centralized_collection/prometheus/docker-grafana-agent
run: |
docker build . --file Dockerfile --build-arg PKTVISOR_TAG=${{ env.REF_TAG }} --tag ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}
docker push ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}
docker build . --file Dockerfile --build-arg PKTVISOR_TAG=${{ env.REF_TAG }} --tag ${{ env.IMAGE_NAME }}:${{ env.VERSION }} --tag ${{ env.IMAGE_NAME }}:${{ env.REF_TAG }}
docker push -a ${{ env.IMAGE_NAME }}

- name: Generate AppImage
env:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ integration_tests/external
golang/pkg/client/version.go
docs/internals/html
appimage/*.AppImage
/test-config*.yaml
localconfig/
2 changes: 1 addition & 1 deletion 3rd/rng/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ target_compile_features(rng INTERFACE cxx_std_11)

target_sources(rng
INTERFACE
${CMAKE_CURRENT_SOURCE_DIR}/randutils.hpp
${CMAKE_CURRENT_SOURCE_DIR}/jsf.h
)
179 changes: 179 additions & 0 deletions 3rd/rng/jsf.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
#ifndef JSF_HPP_INCLUDED
#define JSF_HPP_INCLUDED 1

/*
* A C++ implementation of a Bob Jenkins Small Fast (Noncryptographic) PRNGs
*
* The MIT License (MIT)
*
* Copyright (c) 2018 Melissa E. O'Neill
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
* DEALINGS IN THE SOFTWARE.
*/

/* Based on code published by Bob Jenkins in 2007, adapted for C++ */

#include <cstdint>

namespace jsf_detail {

template <typename itype, typename rtype,
unsigned int p, unsigned int q, unsigned int r>
class jsf {
protected:
itype a_, b_, c_, d_;

static constexpr unsigned int ITYPE_BITS = 8*sizeof(itype);
static constexpr unsigned int RTYPE_BITS = 8*sizeof(rtype);

static itype rotate(itype x, unsigned int k)
{
return (x << k) | (x >> (ITYPE_BITS - k));
}

public:
using result_type = rtype;
using state_type = itype;

static constexpr result_type min() { return 0; }
static constexpr result_type max() { return ~ result_type(0); }

jsf(itype seed = itype(0xcafe5eed00000001ULL))
: a_(0xf1ea5eed), b_(seed), c_(seed), d_(seed)
{
for (unsigned int i=0; i<20; ++i)
advance();
}

void advance()
{
itype e = a_ - rotate(b_, p);
a_ = b_ ^ rotate(c_, q);
b_ = c_ + (r ? rotate(d_, r) : d_);
c_ = d_ + e;
d_ = e + a_;
}


rtype operator()()
{
advance();
return rtype(d_);
}

bool operator==(const jsf& rhs)
{
return (a_ == rhs.a_) && (b_ == rhs.b_)
&& (c_ == rhs.c_) && (d_ == rhs.d_);
}

bool operator!=(const jsf& rhs)
{
return !operator==(rhs);
}

// Not (yet) implemented:
// - arbitrary jumpahead (doable, but annoying to write).
// - I/O
// - Seeding from a seed_seq.
};

} // end namespace

///// ---- Specific JSF Generators ---- ////
//
// Each size has variations corresponding to different parameter sets.
// Each variant will create a distinct (and hopefully statistically
// independent) sequence.
//

// - 128 state bits, 32-bit output
//
// The constants are all those suggested by Bob Jenkins. The n variants
// perform only two rotations, the r variants perform three.

using jsf32na = jsf_detail::jsf<uint32_t, uint32_t, 27, 17, 0>;
using jsf32nb = jsf_detail::jsf<uint32_t, uint32_t, 9, 16, 0>;
using jsf32nc = jsf_detail::jsf<uint32_t, uint32_t, 9, 24, 0>;
using jsf32nd = jsf_detail::jsf<uint32_t, uint32_t, 10, 16, 0>;
using jsf32ne = jsf_detail::jsf<uint32_t, uint32_t, 10, 24, 0>;
using jsf32nf = jsf_detail::jsf<uint32_t, uint32_t, 11, 16, 0>;
using jsf32ng = jsf_detail::jsf<uint32_t, uint32_t, 11, 24, 0>;
using jsf32nh = jsf_detail::jsf<uint32_t, uint32_t, 25, 8, 0>;
using jsf32ni = jsf_detail::jsf<uint32_t, uint32_t, 25, 16, 0>;
using jsf32nj = jsf_detail::jsf<uint32_t, uint32_t, 26, 8, 0>;
using jsf32nk = jsf_detail::jsf<uint32_t, uint32_t, 26, 16, 0>;
using jsf32nl = jsf_detail::jsf<uint32_t, uint32_t, 26, 17, 0>;
using jsf32nm = jsf_detail::jsf<uint32_t, uint32_t, 27, 16, 0>;

using jsf32ra = jsf_detail::jsf<uint32_t, uint32_t, 3, 14, 24>;
using jsf32rb = jsf_detail::jsf<uint32_t, uint32_t, 3, 25, 15>;
using jsf32rc = jsf_detail::jsf<uint32_t, uint32_t, 4, 15, 24>;
using jsf32rd = jsf_detail::jsf<uint32_t, uint32_t, 6, 16, 28>;
using jsf32re = jsf_detail::jsf<uint32_t, uint32_t, 7, 16, 27>;
using jsf32rf = jsf_detail::jsf<uint32_t, uint32_t, 8, 14, 3>;
using jsf32rg = jsf_detail::jsf<uint32_t, uint32_t, 11, 16, 23>;
using jsf32rh = jsf_detail::jsf<uint32_t, uint32_t, 12, 16, 22>;
using jsf32ri = jsf_detail::jsf<uint32_t, uint32_t, 12, 17, 23>;
using jsf32rj = jsf_detail::jsf<uint32_t, uint32_t, 13, 16, 22>;
using jsf32rk = jsf_detail::jsf<uint32_t, uint32_t, 15, 25, 3>;
using jsf32rl = jsf_detail::jsf<uint32_t, uint32_t, 16, 9, 3>;
using jsf32rm = jsf_detail::jsf<uint32_t, uint32_t, 17, 9, 3>;
using jsf32rn = jsf_detail::jsf<uint32_t, uint32_t, 17, 27, 7>;
using jsf32ro = jsf_detail::jsf<uint32_t, uint32_t, 19, 7, 3>;
using jsf32rp = jsf_detail::jsf<uint32_t, uint32_t, 23, 15, 11>;
using jsf32rq = jsf_detail::jsf<uint32_t, uint32_t, 23, 16, 11>;
using jsf32rr = jsf_detail::jsf<uint32_t, uint32_t, 23, 17, 11>;
using jsf32rs = jsf_detail::jsf<uint32_t, uint32_t, 24, 3, 16>;
using jsf32rt = jsf_detail::jsf<uint32_t, uint32_t, 24, 4, 16>;
using jsf32ru = jsf_detail::jsf<uint32_t, uint32_t, 25, 14, 3>;
using jsf32rv = jsf_detail::jsf<uint32_t, uint32_t, 27, 16, 6>;
using jsf32rw = jsf_detail::jsf<uint32_t, uint32_t, 27, 16, 7>;

using jsf32n = jsf32na;
using jsf32r = jsf32rq;
using jsf32 = jsf32n;

// - 256 state bits, uint64_t output

using jsf64na = jsf_detail::jsf<uint64_t, uint64_t, 39, 11, 0>;
using jsf64ra = jsf_detail::jsf<uint64_t, uint64_t, 7, 13, 37>;

using jsf64n = jsf64na;
using jsf64r = jsf64ra;
using jsf64 = jsf64r;

// TINY VERSIONS FOR TESTING AND SPECIALIZED USES ONLY
//
// Parameters derived using a variant of rngav.c, originally written by
// Bob Jenkins.

// - 64 state bits, uint16_t output

using jsf16na = jsf_detail::jsf<uint16_t, uint16_t, 13, 8, 0>;

using jsf16 = jsf16na;

// - 32 state bits, uint8_t output

using jsf8na = jsf_detail::jsf<uint8_t, uint8_t, 1, 4, 0>;

using jsf8 = jsf8na;

#endif // JSF_HPP_INCLUDED
Loading