unable to write to mysql database #62
Comments
|
Do you have actual events being written in the unified2 file you monitor? If you have no events, then no events will be logged to the database. |
|
I ended up using Snort 2.9.2 which barely still supports mysql. I'll play with 2.9.4 again when I have time. From: Eric Lauzon [mailto:notifications@github.com] Do you have actual events being written in the unified2 file you monitor? If you have no events, then no events will be logged to the database. Reply to this email directly or view it on GitHub #62 (comment) . |
|
Ok, well if you have events written to a unified2 file it should be fairly straight forward. One thing you have to make sure is that when you configure snort that you use the following line output unified2: filename merged.log, limit 128 and not output unified2: filename snort.log, limit 128, nostamp (the filename prefix and filesize rotation limit could change to suit your needs) but if you have the nostamp option So that would be the first step. Then when this is setup you have to check for the file to grow and have events in it. Once you have this setup and the file grows barnyard2 should be processed by barnyard2 without a problem. |
|
Hi i've got the same issue at our installation. If there is any configuration problem, I would appreciate any input, but we currently can't find the problem. OS is SLES 11 SP2, we've tried it with barnyard2 version 2.1.9, 2.1.10 and 2.1.11. Also debug mode wasn't any more helpful: me@sensor:~> ps -ef | grep snort grep -vw '#' /etc/snort/snort.conf (only relevant part copied here): output unified2: filename snort.u2, limit 128 output alert_syslog: LOG_LOCAL7 LOG_WARNING LOG_NDELAY grep -vw '#' /etc/snort/barnyard2.conf(only relevant part copied here): config reference_file: /etc/snort/reference.config config logdir: /var/log/barnyard2 config alert_with_interface_name input unified2 output log_tcpdump: snort-tcpdump.log before: me@sensor:~> sudo lsof | grep 2109 => it's writing to the correct file, which barnyard2 is reading test was sending 4 pings to a server that is observed by snort root@sensor: root@sensor:~# ls -lR /var/log/snort/ /var/log/snort/eth1: after: root@sensor:~# cat /var/log/snort/eth1/alert [] [1:477:0] Global ICMP Test Packet [] [] [1:477:0] Global ICMP Test Packet [] [] [1:477:0] Global ICMP Test Packet [] root@sensor:~# tcpdump -r /var/log/snort/eth1/snort.log.1359117629 root@sensor:~# lsof | grep 25248 infos from /var/log/messages: root@sensor:~# ls -l /var/log/barnyard2/ root@sensor: => with strace we don't see anything more generated. As the whole setup didn't work with barnyard2 yet (upgrade from old snort version) i can't say what is supposed to happen when packets are received and added to the unified2 file. the error messages in /var/log/messages are now different than the ones posted above, but we also had them. Please let me know if you need more information. |
|
Remove snort -b command line argument and -A (useless for unified2 logging) use the command : file /var/log/snort/snort.log.XXXXX and you should see it. And we highly recommend you to use 2-1.11 |
|
hi, thank you for your fast reply. i removed -b, -A shouldn't make a difference if i understand it right, as full is default. unfortunately it doesn't make a difference: root@sensor:~# ps -ef | grep snort root@sensor:~# strace -p 5341 Jan 25 14:55:53 sensor barnyard2[25250]: Failed to archive file "/var/log/snort/eth2/snort.log.1359117630" to "/var/log/snort/eth2/archive/snort.log.1359117630": No such file or directory version used is 2-1.11. and reading the tcpdump file wasn't the problem, the problem is the output to the db, the tcpdump was more for testing if it works at all. |
|
you need to delete your old unified2 file that where actually pcap file and not unified2 file. Once you have deleted them it will work, if you do not delete them and reprocess old file you wont see a difference. Also note the by2 error message. Jan 25 14:56:36 sensor barnyard2[25250]: ERROR database: [UpdateLastCid()]: Error commiting transaction And since your using mysql you need to have InnoDB storage and not MyIASM. |
|
thank you, i deleted the old files and it created a new one after my current test. I'm currently adjusting the DB format and than try again. |
|
Well I finally got it to work, following this doc: "Snort 2.9.3 and Snort Report 1.3.3 on Ubuntu 12.04 LTS Installation Guide by David Gullett"; with some modifications, mainly some needed packages that weren't listed. From: Eric Lauzon [mailto:notifications@github.com] Ok, well if you have events written to a unified2 file it should be fairly straight forward. One thing you have to make sure is that when you configure snort that you use the following line output unified2: filename merged.log, limit 128 and not output unified2: filename snort.log, limit 128, nostamp (the filename prefix and filesize rotation limit could change to suit your needs) but if you have the nostamp option So that would be the first step. Then when this is setup you have to check for the file to grow and have events in it. Once you have this setup and the file grows barnyard2 should be processed by barnyard2 without a problem. Reply to this email directly or view it on GitHub #62 (comment) . |
|
Well packages dependancies where probably not related to by2 but i am happy On Fri, Jan 25, 2013 at 6:22 PM, rush01 notifications@github.com wrote:
|
|
so, apparently the db is in innoDB format (sorry, different responsibility people), but i'm wondering about that error message, cause the only thing i find is this one: WARNING database: Defaulting Reconnect/Transaction Error limit to 10 still googeling what that is supposed to mean and if that requires me to do any changes. i also removed the full -A tag, just in case, as well as all the old snort.log.* files including the ones in the archive. i'm right now waiting for another engineer (again, different responsibilities) to package u2spewfoo to verify that the output is in the right format. is there anything else i can look for in the meantime? |
|
ok, i'm back, sorry for still bothering, i unfortunately again need your help: root@sensor:/home/me# ps -ef | grep snort root@sensor:/home/me# lsof | grep 25113 root@sensor:/home/me# u2spewfoo /var/log/snort/eth1/snort.u2.1359620700 (Event) Packet (Event) Packet (Event) Packet (Event) Packet is there any other location where barnyard caches the old information? |
|
On Thu, Jan 31, 2013 at 3:41 AM, d3sre notifications@github.com wrote:
use the -f command line argument to barnyard2 to specify a unified2 prefix. ex: you will provide barnyard2 -f abcd.unified2 command line argument. In your case it would be -f snort.u2 using your command line : /usr/bin/barnyard2 -D -c /etc/snort/barnyard2.conf -d /var/log/snort/eth1 And it should be working. |
|
hi binf thanks for your help, but -f snort.u2 is already specified (or does it need to be at the end of the command?). Update: it's only 1 interface where the barnyard process crashes, the other one is fine and is now correctly reading the .u2 file, i just can't test on that interface :/ |
|
On Thu, Jan 31, 2013 at 4:07 AM, d3sre notifications@github.com wrote:
My bad didin't saw it. (no it can be anywhere in the command line
Yes if the waldo was containing the old file prefix, you needed to delete it.
If you run barnyard2 in console what messages do you get? I would suggest that you do so, so you can capture what its reporting, -elz |
|
ok, it aborts the startup of eth1 when it enters the for loop for every interface (that's line 45), but i didn't change anything there, i only deleted the old waldo file between it working (but accessing the wrong file) and now not working anymore for 1 interface (eth1, eth2 is fine): root@sensor:/var/log/snort# /etc/init.d/barnyard2 start Initializing Input Plugins! database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM data WHERE sid='1';] ______ -> Barnyard2 <-
Using waldo file '/var/log/snort/eth1/barnyard2.waldo': Initializing Input Plugins! database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM data WHERE sid='3';] ______ -> Barnyard2 <-
Using waldo file '/var/log/snort/eth2/barnyard2.waldo': my init.d/barnyard2 file's start looks like this: RETVAL=0 start() { /usr/bin/barnyard2 points to the barnyard2.nodebug-2.1.10 version. was there such a bug that was solved with 2.1.11? btw, the file does exist, but it's only the oldest (i guess it's supposed to access the others after the oldest): is there a problem with a check and the files having same sizes? just wondering about this output from the log above: |
|
Upgrade to current master alot of little issue have been fixed. On Thu, Jan 31, 2013 at 4:48 AM, d3sre notifications@github.com wrote:
|
|
thank you, very much, it's working now. there was a little issue still cause i didn't have a rev: defined for my test rule but after adjusting that i now finally had my first db entries! :D thank you again! |
|
On Thu, Jan 31, 2013 at 6:12 AM, d3sre notifications@github.com wrote:
Good stuff. I would also encourage you to join the barnyard2 mailing list so if you Look for barnyard2-users on google groups. |
|
i will do that.. also thinking about writing a how-to on all the stuff i stumbled upon.. so many small things, that would have helped to know and understand from the beginning.. and i guess it's difficult to see those things if you've been working on the project for a while :/ |
|
On Thu, Jan 31, 2013 at 7:23 AM, d3sre notifications@github.com wrote:
Sure, well you already did some good things like separating your log Also the by2 mailing list is a good place to ask question and help people. |
|
Hi , all . I get some problems like these , and I 've no idea now . environment info: Ok. I've checked these above:
;# Additional configuration for specific types of installs Additional, with barnyard2 command: barnyard2 -c barnyard2.conf -o c:\Snort\log\log.u2.1367683785, i get result: Initializing Input Plugins! ______ -> Barnyard2 <-
Processing 1 files...
step2: for 3) and more: show create table sensor; step 3: for 4) when i running barnyard2 in Continuous mode with command: Initializing Input Plugins! ______ -> Barnyard2 <-
Waiting for new spool file |
|
maybe u would advise to run barnyard like this: and then i touch a barnyard2.waldo file, it started with msg: Maybe , the key point is waldo file( now my derivation ), but i don't know what should i do. U experts have any ideas ? |
|
The problem seems to lie in the fact that snort is not generating any event. Did you try to run snort with command line argument -k none to disable For further assistance,debugging please uses the barnyard2 mailing list : -elz On Sat, May 4, 2013 at 9:27 PM, liebazi notifications@github.com wrote:
|
|
i use snort with -k none arguments, and run barnyard2 in batch mode ,get as follows: Initializing Input Plugins! ______ -> Barnyard2 <-
Processing 1 files... It shows 257 events and 0 packets , but i also find a strange msg: Last event seen for sid 8 was 0 and then i run barnyard2 in Continuous mode, still "waiting for new spool file" |
|
from #17 i check my alert.u2 & log.u2, found that alert.u2 just have only events, log.u2 only packets. ;# Additional configuration for specific types of installs then run snort , generate merge.log.xxxx but Running barnyard2 in Continuous mode, still "wait for new spool file" |
|
So this is my problem I run barnyard using this command and I get this in syslog Sep 13 13:57:39 VMLSnort snort[1773]: Commencing packet processing (pid=1773) so Im using a mysql server and snort writes the file as mergerd.u2 timestamp the files appear and grow. However the database is empty and no waldo file ever appers so I'm really new to snort but I think I need to test if barnyard2 is reading the file and then look at maybe the mysql schemas that came with barnyard2. cat > /etc/snort/barnyard2.conf << EOFconfig reference_file: /etc/snort/reference.config So What might be the problem? If need be how can I test barnyard2 and mysql? Also you mentioned above updating the master but I have no clue what you mean. Would updating the master fix it and if so how do I do that? |
It appears barnyard2 is unable to write to mysql as I have it configured. Snort is running OK on CentOS 6.3 as per a doc on snort.org; I follow directions I found at http://polaris.umuc.edu/~sgantz/Install.html as for the barnyard config, yet I still see this message:
--== Initializing Barnyard2 ==--
Initializing Input Plugins!
Initializing Output Plugins!
Parsing config file "/etc/snort/barnyard2.conf"
Barnyard2 spooler: Event cache size set to [2048]
Log directory = /var/log/barnyard2
INFO database: Defaulting Reconnect/Transaction Error limit to 10
INFO database: Defaulting Reconnect sleep time to 5 second
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM data
WHERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM event
WHERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM icmph
dr WHERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM iphdr
WHERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM opt W
HERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM tcphd
r WHERE sid='1';]
database: [SynchronizeEventId()]: Problems executing [SELECT MAX(cid) FROM udphd
r WHERE sid='1';]
[SignatureReferencePullDataStore()]: No Reference found in database ...
database: compiled support for (mysql)
database: configured to use mysql
database: schema version = 107
database: host = localhost
database: user = snort
database: database name = snort
database: sensor name = localhost:eth0
database: sensor id = 1
database: sensor cid = 1
database: data encoding = hex
database: detail level = full
database: ignore_bpf = no
database: using the "log" facility
______ -> Barnyard2 <-
/ ,,_ \ Version 2.1.11 (Build 317)
|o" )~| By Ian Firns (SecurixLive): http://www.securixlive.com/
WARNING: Ignoring corrupt/truncated waldofile '/var/log/snort/barnyard2.waldo'
Waiting for new spool file
and it doesn't appear to be writing to mysql.
The text was updated successfully, but these errors were encountered: