New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error: Snorby::Jobs::DailyCacheJob failed with DataObjects::DataError: Data too long for column 'src_ips' at row 1 - 2 failed attempts #109
Comments
Can you log into the DB and post some back of examples of dst and src IPs On Tue, Aug 9, 2011 at 10:19 AM, dmerida <
|
Here are ten from the bottom of the iphdr table. I grouped them in descending order and then limited by 10. I also converted the fields back into IPs in the two right columns. This CSV format was the best that I could display here.
The issue seems to stem from multiple ips loading into an object during a cache job. I can see the full error in verbose mode, but it's too great to post here. |
When I say verbose mode, I'm referring to running the cache jobs manually via "rails c." It's like so: The output from the error in this case is many thousands of lines long and therefore too long to post in issues. I could email the full error if necessary. |
Bump Any other comments on a fix for this terracatta? |
Hi, I got the same issue today, turns out the reason is that the columns "signature_metrics", "severity_metrics", "dst_ips" and "src_ips" from the tables "caches" and "daily_caches" are from type TEXT, being thus limited to 65,535 bytes. What solved my problem was updating these columns to MEDIUMTEXT, so they support up to ~16MB text. So, a quick fix is login into your mysql server and run: alter table daily_caches change signature_metrics signature_metrics mediumtext; alter table caches change signature_metrics signature_metrics mediumtext; I'm working on a definitive fix for this, I'll probably push it in the next few days. |
Closing this one, commit 09db614 fixes it. New Snorby installations will have the fields of type Object mapped automatically to MEDIUMTEXT on mysql. |
Snorby::Jobs::DailyCacheJob failed with DataObjects::DataError: Data too long for column 'src_ips' at row 1 - 0 failed attempts
Snorby::Jobs::SensorCacheJob completed after 25.3841
Snorby::Jobs::DailyCacheJob failed with DataObjects::DataError: Data too long for column 'src_ips' at row 1 - 1 failed attempts
Snorby::Jobs::SensorCacheJob completed after 24.4008
Snorby::Jobs::DailyCacheJob failed with DataObjects::DataError: Data too long for column 'src_ips' at row 1 - 2 failed attempts
PERMANENTLY removing Snorby::Jobs::DailyCacheJob because of 3 consecutive failures.
Snorby::Jobs::SensorCacheJob completed after 7200.1682
6 jobs processed at 0.0005 j/s, 3 failed ...
This is the issue I'm currently seeing in Snorby 2.3.6 on RHEL6. The DB is on a separate server running RHEL5.6 with MySQL 5.
The SensorCacheJob also crashes after this (or at least it no longer appears on the worker page). I've tried everything I can think of. I recreated the database and it worked okay with < 15,000 alerts. I've also rebuilt the cache from stratch and tried updating different modules of the install. I've also tried running straight from rails without passenger to see if that makes a different (I can at least control delayed_jobs through the interface now. You can't start/stop/restart/ in passenger).
Thank you for your assistance. I really like Snorby's dashboard and email summaries when they're working well (and so does my director!).
The text was updated successfully, but these errors were encountered: