Skip to content
This repository has been archived by the owner on Jan 18, 2024. It is now read-only.

new alert_json plugin with kafka capabilities #88

Open
wants to merge 212 commits into
base: master
Choose a base branch
from
Open

new alert_json plugin with kafka capabilities #88

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
212 commits
Select commit Hold shift + click to select a range
c5efa08
Created alert_json output plugin skeleton, and integrated in banyard2.
Mar 28, 2013
db69fcf
Changed names under spo_alert_json. No funtionality changed except de…
Apr 3, 2013
eb71ecc
Timestamp now printed in milisenconds, instead of string
Apr 3, 2013
ec29434
Each data type add it's own string to a json file now.
Apr 3, 2013
25ca718
FIXED: JSON invalid fields are not written at all. Before, a ",," was…
Apr 3, 2013
0df4cdd
added output-plugins/spo_alert_json.h in src/plugbase.c
Apr 5, 2013
df547e1
Added kafka libraries and header (in a future we will add the entire
Apr 5, 2013
0de1baa
Added kafka output to spo_alert_json. Topic is need to know in compile
Apr 16, 2013
c408120
Kafka topic can be specified adding a '@' after broker's name
Apr 16, 2013
5d886ca
FIX: alert_json can send alerts to a file and a kafka broker at the same
Apr 16, 2013
6d76240
Delayed KafkaLog's handler init in daemon mode (Need to do because a
Apr 18, 2013
615879c
Increased spo_alert_json LOG_BUFFER; Kafka split messages with just 4K
Apr 18, 2013
43a2c43
Changed the way sf_kafka use the buffer. Now it allocate a new one an…
May 3, 2013
b1f4acc
FIX: When sending alerts, sometimes proto was not set, so the json alert
May 3, 2013
de53c97
Bumped: version to 2-1.13-BETA
binf Mar 13, 2013
5796f03
Last minute commit for a long waited needed feature and some little fix.
binf Apr 26, 2013
99c8188
fixed: libwebsocket update collapsed a number of arguments into gener…
May 7, 2013
bcbd6b0
updated: build bump and removal of beta tag pending release.
May 7, 2013
33f49f8
fixed: range logic was inadvertenly inverted.
May 9, 2013
b0ccd22
fixed: lingering reference identifid during HUP operations.
May 9, 2013
317f4be
added: handle situations where map files are not v2.
May 9, 2013
d856c41
fixed: issue with signature insertion and v1/v2 sid-msg.map handling.
May 14, 2013
5fefbe0
Parsed hosts and networks file in order to produce in spo_alert_json
May 15, 2013
1126d59
Added src_name, src_str, dst_name and dst_str to json fields, based on
May 17, 2013
f048121
Added network identifications in source and destination ip
May 17, 2013
0896ef8
Added geoIP support using maxmind GeoIP
May 19, 2013
dd159ed
Fixed a possible memory leak.
May 20, 2013
6072ae5
Fixed some possible memory leaks and enabled --enable-geo-ip in
May 20, 2013
974bef5
Updated librdkafka
May 20, 2013
72b1321
Added to sf_kafka a maximum queue length value
May 20, 2013
7f1de11
Added json_output plugin
May 20, 2013
4d2fbf3
Added json and geoIP libs conditional compile stuff in configure.in.
eugpermar May 29, 2013
563bc89
fix: possible double free's on cleanup when HUP recieved.
May 24, 2013
0459902
FIX: sf_kafka did not compile if --enable-kafka were passed to compile
eugpermar Jun 1, 2013
ee5eca8
Added priority and classification fields to alert_json plugin. Some f…
eugpermar Jun 1, 2013
f4cb1c3
Added sensor_id capability
eugpermar Jun 3, 2013
a80dd89
Added sensor_id and sensor_name passed by params. We will send the co…
eugpermar Jun 4, 2013
c6f50a5
Using sf_ip from sfutils module, what simplifies a bit the name resol…
eugpermar Jun 27, 2013
b03bb29
FIXED: Sendig src_country twice by error
eugpermar Jul 2, 2013
2b98f85
Added spread over kafka partitions capacity.
eugpermar Jul 4, 2013
dfea910
Deleted kafka libs and used system ones
eugpermar Jul 4, 2013
67a8c03
The geoip library path is now passed by param.
eugpermar Jul 8, 2013
44ad609
Readed services and protocols file, and sended in the event
eugpermar Jul 10, 2013
5a8741d
FIX: dst port name was the same as src_port name.
eugpermar Jul 10, 2013
05fcdc2
FIX: Classification field now show the correct classification message
eugpermar Jul 10, 2013
aacf0d9
Created rbutil folder, where it will be all redborder utils.
eugpermar Jul 10, 2013
7e2eda9
Created rbutil folder, where it will be all redborder utils.
eugpermar Jul 10, 2013
cb52266
Amend: fixed src/Makefile.am
eugpermar Jul 10, 2013
dba7696
Warning supressed.
eugpermar Jul 10, 2013
8ac1090
Changed -DJSON_KAFKA and -DJSON_GEOIP to defines in config.h file
eugpermar Jul 10, 2013
3f2a03a
Added payload field
eugpermar Jul 11, 2013
99adef9
Changed the strcmp alerts processing system to a template based one
eugpermar Jul 12, 2013
9a808be
Using the json name given in the template
eugpermar Jul 12, 2013
3e97959
Added a "default value" field in the template
eugpermar Jul 12, 2013
9758863
Update doc
eugpermar Jul 12, 2013
38f3cac
Proto fallback value is now the same value as proto_id
eugpermar Jul 13, 2013
262c850
Added ARP and VLAN parsing.
eugpermar Jul 15, 2013
f8519e9
Resolved VLAN names.
eugpermar Jul 15, 2013
8fd7109
FIX: ipv4 were not passed by ntohl functions
eugpermar Jul 16, 2013
fcdbe8f
Deleted all slow KafkaLog_Print and changed to KafkaLog_Puts. Added a…
eugpermar Jul 17, 2013
de16a9d
changed a snprintf to a strcat in KafkaLog_Write: performance
eugpermar Jul 17, 2013
c7477a5
Changed hosts format to [name addr] (See redmine #522 issue)
eugpermar Jul 17, 2013
529e965
Timestamp resolution moved from miliseconds to seconds
eugpermar Jul 25, 2013
47af71b
alert-json did not work if --enable-ipv6 was present (Redmine issue 6…
eugpermar Jul 30, 2013
b346855
Changed configure.in so it can import rdkafka from any location. Now you
eugpermar Jul 31, 2013
1971669
Added rb_pointers.h header to have specific commands to check pointer…
eugpermar Aug 1, 2013
f7eff47
Geoip libraries can now be specified in configure (--with...)
eugpermar Aug 1, 2013
70ae7b8
Sending action of message (not fully supported). Some numbers sended …
eugpermar Aug 2, 2013
de518d2
IPv4 sended in wrong format. Deleted a warning. fwsam did not compile…
eugpermar Aug 2, 2013
2da769a
Barnyard could wait forever if the kafka broker was down. Fixed.
eugpermar Aug 2, 2013
038d63a
Eth address not printed with all zeroes padding. Fixed.
eugpermar Aug 2, 2013
55859fc
Added kafka 0.8 support
eugpermar Aug 7, 2013
003086d
Workaround to solve bug in librdkafka.
eugpermar Aug 12, 2013
e11344d
Changed some fields name.
eugpermar Aug 22, 2013
0665f85
Deleted sensor_id_snort from default template
eugpermar Aug 23, 2013
9bc12e3
Changed some template elements
eugpermar Aug 23, 2013
a3a04f1
Changed some template names
eugpermar Sep 16, 2013
37c67a5
Deleted kafka 0.8 support. Fixed memory leak. Inserted some likely()
eugpermar Sep 24, 2013
4d4b105
Updated configure.in
eugpermar Oct 9, 2013
3300b02
Added group_id and group_name in the parameters. domain_id renamed to…
eugpermar Nov 5, 2013
a50d5cc
IP packets length and ethernet packets length are now aggregated in g…
eugpermar Nov 19, 2013
8f6035a
Added AS numbers
eugpermar Nov 21, 2013
0f2e165
Changed Fatal errors to error messages.
eugpermar Nov 21, 2013
770955c
Update to current kafka api
eugpermar Nov 28, 2013
d24420d
FIX sometimes print a *0 when printing ethlength.
eugpermar Dec 5, 2013
9bcd10b
FIX: ntohl were not applied in ip.
eugpermar Dec 5, 2013
e823da9
Solved a invalid write valgrind report. Don't sending ip,mac,vlans na…
eugpermar Dec 18, 2013
ab3c3da
Added rb_macs_vendor support
eugpermar Dec 26, 2013
5086948
Cleaned old kafka code
eugpermar Dec 26, 2013
4b33b2f
FIX: ETHDST_VENDOR without mac in switch.
eugpermar Jan 15, 2014
4fa91db
FIX: spo_alert_json didn't compile if not HAVE_GEOIP present
eugpermar Apr 30, 2014
a56f46c
FIX: rb_kafka didn't compile if not --enable-rdkafka present (thanks …
eugpermar Apr 30, 2014
56c8d63
FIX: ethlen puts an extra 0 sometimes
eugpermar May 26, 2014
f272e43
You can pass option to rdkafka directly (See #2250)
eugpermar May 27, 2014
ac57192
Added delivery function callback for every message
eugpermar May 27, 2014
34376d1
Deleted priority name. Now priority is always a name
eugpermar Jul 11, 2014
e3baa9b
FIX: Buffer overflow
eugpermar Jul 14, 2014
37a8067
FIX: Buffer overflow
eugpermar Jul 14, 2014
3ab793e
FIX: priority was sent even when there was no event
eugpermar Jul 15, 2014
d653674
FIX: Bad delivery message callback management
eugpermar Jul 16, 2014
25f5f6f
Barnyard cache is now freeing at the end, instead of start. This avoi…
eugpermar Jul 17, 2014
208ace4
Sanitized spooler.c: some functions made static, some prototypes dele…
eugpermar Jul 17, 2014
a9de004
Enabled lonely events processing. SRC/DST IP are now extracted from e…
eugpermar Jul 18, 2014
7158760
Trying to get proto from event before attempt to extract from the pac…
eugpermar Jul 18, 2014
64f9cfd
src and dst port now extracted from the event instead of the packet. …
eugpermar Jul 21, 2014
0f49c62
ICMP type now tried to extracted from event first than the packet
eugpermar Jul 21, 2014
99fb126
extract icmp code first from event that from packet. ICMP code & type…
eugpermar Jul 21, 2014
f00d29d
Trying to extract vlanId information of the event first of the packet.
eugpermar Jul 21, 2014
e103aac
ipv6 country database loaded separately from ipv4 country database
eugpermar Jul 21, 2014
b0103b1
Autonomous System Numbers ipv6 database are now sepparated from ipv4 …
eugpermar Jul 21, 2014
c3c02f0
Cosmetic changes: SRC_REQ and DST_REQ uses are now encapsulated by ma…
eugpermar Jul 21, 2014
03555b0
FIX: didn't compile if --enable-ipv6 was not present
eugpermar Jul 21, 2014
7601011
rb_pointers added copyright
eugpermar Aug 28, 2014
6d2eb21
Extracted actionOfEvent function, in order to reuse in another output…
eugpermar Aug 28, 2014
5fa044b
Syslog output plugin can print the action taken
eugpermar Aug 28, 2014
9f5c7ad
Syslog output plugin now prints action taken too.
eugpermar Aug 29, 2014
078102a
Added sensor name to output
eugpermar Aug 29, 2014
485110f
Added sensor-group to syslog output plugin
eugpermar Aug 29, 2014
7099fe5
Renamed "sensor name"->"sensor" and "sensor group"->"group"
eugpermar Aug 29, 2014
b46067c
Added rbutils/Makefile.in, and headers guards
eugpermar Sep 1, 2014
b3f9d72
Increased syslog human-readable IP buffers size, in order to accomoda…
eugpermar Sep 2, 2014
a38c808
FIX: Ports were using as uint8_t, instead of uint16_t
eugpermar Oct 30, 2014
6099b36
FIX: bad IPlength
eugpermar Oct 30, 2014
e0ea441
Included Extra Data counts in Records
Mar 26, 2015
13cbf23
Preparing spo_alert_json.c, plugbase.c and plugbase.h. Perhaps we wil…
Mar 30, 2015
411cc8f
Flush cached event after TIME_ALARM seconds: Setting Alarm
Mar 31, 2015
fe432e5
Changing the way spoolerProcessRecord works and including Extra Data …
Apr 7, 2015
8a01fd9
Fire remained events before closing spooler when EOF is reached
Apr 7, 2015
e17f878
rd_unified2.c modified: fixing macro and actionOfEvent
Apr 9, 2015
2e91674
Merge branch 'master' into ExtraData
Apr 10, 2015
1215ad4
spo_alert_json.c modified: getting events from spooler and producing …
Apr 10, 2015
fc3035b
Not sending src/dst net/name if they are empty
eugpermar Apr 10, 2015
e25213d
Changed alert messages to should_drop and cant_drop
eugpermar Apr 10, 2015
fd044e7
Merge branch 'master' into ExtraData
Apr 10, 2015
a1b3487
Solving memory issues
Apr 14, 2015
10a40c1
Fixing/readjusting some pieces of code
Apr 14, 2015
f45af1e
Comment added to explain payload not matching
Apr 14, 2015
6d2b8b2
Merge branch 'master' into ExtraData
Apr 14, 2015
57cf6a7
Firing the last cached event
Apr 14, 2015
2c06993
define RB_EXTRADATA
Apr 14, 2015
c5a3d45
Including macro RB_EXTRADATA in configure. Short change in spoolerExt…
Apr 15, 2015
185858f
Changing the way DEFAULT_JSON is defined. Including code to write URI…
Apr 17, 2015
e0177f3
changed file_sha256 by sha256 in spo_alert_json.c
Apr 28, 2015
9285734
Using X macros to avoid ID/function duplication
eugpermar May 18, 2015
e512180
Added enrich_with output json parameter. Deleted a few deprecated par…
eugpermar May 18, 2015
21edda4
Added KafkaLog_FlushAll in order to flush all messages when barnyard2…
eugpermar May 18, 2015
2238c7e
Cleaning GeoIP databases
eugpermar May 18, 2015
7ce4263
Fixing .gitignore file
May 22, 2015
2baa664
merging from master
May 22, 2015
09c1d96
Updated .gitignore
eugpermar Jul 1, 2015
609383d
SMTP ExtraData fields included
Jul 10, 2015
f6ed494
FIX: Last event discarded when opening a new snort.log file (redmine …
Jul 13, 2015
8271632
Adding rbhttp library in the configure flags
eugpermar Sep 29, 2015
d6548f6
Deleted rb_kafka.{c,h}, since they did too many stuffs. Preparing for…
eugpermar Sep 30, 2015
c550318
Creating a thread in order to do kafka polls
eugpermar Sep 30, 2015
5ab2438
More verbose kafka errors
eugpermar Oct 1, 2015
d1de5ad
Adding http library to json output plugin
eugpermar Oct 1, 2015
c342656
Deleting kafka by default behavior
eugpermar Oct 2, 2015
4098604
FIX: GeoIP_delete called twice
eugpermar Oct 2, 2015
c79b801
Added printbuf to src/rbutil/Makefile.in
eugpermar Oct 2, 2015
9827bd8
New properties http.max_connections and http.max_queued_messages
eugpermar Oct 13, 2015
d6edcea
Adapted barnyard2 to use librbhttp v1.0, and wrapping new properties
eugpermar Oct 14, 2015
284cdb8
Deleting deprecated comment
eugpermar Dec 18, 2015
81801dd
DEFAULT_JSON is now extracted from X_FUNCTION_TEMPLATE
eugpermar Dec 21, 2015
fd503de
Added header to rb_numstrpair_list.h
eugpermar Dec 21, 2015
44ade9c
Cleaning headers of spo_alert_json.c
eugpermar Dec 21, 2015
c4bea65
Deleted deprecated "type" and "trheader" template
eugpermar Jan 13, 2016
63ef7a7
Deleted AlertJSONConfig, never used
eugpermar Jan 14, 2016
d19a0ee
Created output_template functions that abstract TAILQ behavior
eugpermar Jan 14, 2016
55eb651
New extraData: EVENT_INFO_FTP_USER
Jan 25, 2016
852945d
extradata: EVENT_INFO_SMB_UID EVENT_INFO_SMB_IS_UPLOAD
Jan 27, 2016
75da520
Added compiled, configure, Makefile.in, and compile script to git
eugpermar Feb 1, 2016
ab605f7
Updated configure.in to use new librb-http library
eugpermar Feb 1, 2016
ad18a9d
Updating src/rbutil/Makefile.in, deleting rb_kafka.* references
eugpermar Feb 1, 2016
0b1f424
Removed whitespaces
Bigomby Feb 1, 2016
eb3f943
Integrated new version of librbhttp
Bigomby Feb 1, 2016
9bb391b
Merge pull request #4 from Bigomby/Feature/Managing_ExtraData_fields
eugpermar Feb 1, 2016
8a6997b
Added many autoconf.sh generated files
eugpermar Feb 1, 2016
02140d8
Fixed invalid free memory
Feb 4, 2016
5357d16
Merge pull request #5 from anarey/feature/extra_data_bug
eugpermar Feb 4, 2016
4469f7a
Bug: leak memory. Do not free the eth_vendors_db elemnt
Feb 4, 2016
cb2f2bd
Release memory in kafka
Feb 4, 2016
8f0d510
Merge pull request #7 from anarey/feature/extra_data_bug2
eugpermar Feb 15, 2016
76bffce
extradata: deleted "<" and ">" caracteres in email_sender element
Feb 15, 2016
4a1ec75
extradata: email_destination in an array
Feb 15, 2016
150d5fb
Merge pull request #8 from anarey/feature/email-destinations
eugpermar Feb 16, 2016
3fe7c2e
Created printMail function
eugpermar Feb 16, 2016
abd4f8e
Cleaning EMAIL_DESTINATIONS extradata print case
eugpermar Feb 16, 2016
a859ac8
Created printMultipleMails function
eugpermar Feb 16, 2016
888586a
Fix in Alarm Control for spoolerFireLastEvent()
Feb 22, 2016
05ac8f1
Added spoolerPrint*() functions for debug purposes
Feb 22, 2016
9f5a4b3
Protection added when reading from spooler
Feb 22, 2016
615324e
Merge branch 'Feature/Managing_ExtraData_fields' of github.com:redBor…
Feb 23, 2016
bc0e9a7
Added spoolerPrint*Packet() functions for debug purposes
Feb 23, 2016
40d8d26
Some few lines of code fixed
Feb 23, 2016
f23a075
Added more defaults options
Bigomby Mar 8, 2016
d6fb823
Added newline char to ErrorMessage on HTTP error
Bigomby Mar 10, 2016
f327df1
Merge pull request #9 from Bigomby/Feature/Managing_ExtraData_fields
eugpermar Mar 15, 2016
1d0f2e5
Printing file_size as json number
eugpermar Feb 28, 2016
4e9c18f
FIX: printMultipleMails reading beyond limits
eugpermar Apr 27, 2016
12f2a9a
Use printbuf_memappend_escaped in EVENT_INFO_FILE_NAME
eugpermar Apr 27, 2016
c1cddd9
change service script
Apr 29, 2022
705de4f
Adding systemd service
manegron May 11, 2022
ef6ea9a
rpm build
davidredborder Nov 6, 2023
64470a9
make compatible with rhel9
davidredborder Nov 6, 2023
181ff00
update spec file
davidredborder Nov 6, 2023
d104676
Merge pull request #14 from redBorder/Feature/Managing_ExtraData_fields
manegron Nov 8, 2023
06f8575
Merge pull request #15 from redBorder/rhel9
manegron Nov 8, 2023
bd81df0
update process rpm creation
davidredborder Apr 2, 2024
0b4ea63
add makefile
davidredborder Apr 11, 2024
294aa24
Add gcc into buildrequires
manegron Apr 11, 2024
c639a2b
Merge pull request #16 from redBorder/improvement/#16872_rpm_package_…
manegron Apr 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
18 changes: 6 additions & 12 deletions .gitignore
View file
Open in desktop
@@ -1,24 +1,18 @@
*~
*.o
aclocal.m4
autom4te.cache/
cflags.out
/config.*
configure
cppflags.out
install-sh
libtool
ltmain.sh
m4/libtool.m4
m4/lt~obsolete.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
Makefile
Makefile.in
missing
src/barnyard2
src/input-plugins/libspi.a
src/output-plugins/libspo.a
src/sfutil/libsfutil.a
src/rbutil/librbutil.a
stamp-h1
*Makefile.in

#RPM
pkgs
SOURCES
7 changes: 7 additions & 0 deletions Makefile
View file
Open in desktop
@@ -0,0 +1,7 @@
all: rpm

rpm:
$(MAKE) -C packaging/rpm

rpmtest:
$(MAKE) LATEST=`git stash create` -C packaging/rpm
2 changes: 1 addition & 1 deletion Makefile.am
View file
Open in desktop
Expand Up @@ -3,7 +3,7 @@ AUTOMAKE_OPTIONS = foreign no-dependencies

ACLOCAL_AMFLAGS = -I m4

SUBDIRS = src etc doc rpm schemas m4
SUBDIRS = src etc doc schemas m4

INCLUDES = @INCLUDES@

Expand Down