Fail2ban Integration via syslog #6

garyemiller · 2021-11-29T22:21:22Z

I am new to SentryPeer, but not to honeypots, firewalls etc.

My favorite tools are netfilter and fail2ban. I would like SentryPeer to work easily with fail2ban. That should be simple. Just syslog() the attackes, let syslog add the date stamp.

A typical one line log might look like:

sentrypeer: fromip: 192.168.1.1 type: UDP message: {REGISTER|OPTIONS} agent: sipvicious

With that I can write a fail2ban rule, block them, and report them, automagically.

The text was updated successfully, but these errors were encountered:

ghenry · 2021-11-29T23:33:54Z

Hi Gary,

Thanks for this idea and taking the time to read everything. How did you find out about SentryPeer?

I have actually mentioned this in the README:

SDKs/libs for external access - CGRateS to start with and maybe Fail2Ban or our own with nftables

Also, there is a transport_type which is currently UDP, but TCP and TLS coming. Yesterday I added a collected_method, i.e passive or responsive. That means, as of right now, passive, as we don't reply to SIP probes, but I plan to reply as that's when the proper attacks start and we can harvest phone numbers. Those probes/attacks will be marked as responsive. This allows you to filter by noise versus really bad things. I want the passive version of data for analytics and modelling / predicting things, if possible.

Make sense?

Gavin.

garyemiller · 2021-11-30T01:04:59Z

Thanks for this idea and taking the time to read everything.

Everything? Not even close.

How did you find out about SentryPeer?

It stuck in my mind (and browser bar) for a week before I installed it. I think I saw it mentioned on the NANOG email list. NANOG usually has good ideas.

I have actually mentioned this in the README:

Yes, but not near the top.

Make sense?

Yes, but no responsive to my issue. :-)

Looking at your code, I think I can submit some small patches to make this happen now.

Attached is part 1 of 2 to add syslog()ing.

diff.txt

You may want a getopt() to toggle syslog() on and off.

garyemiller · 2021-11-30T01:08:50Z

Ignore that. vsyslog() should be syslog(). gilhub fighting me about uploading the fix...

diff1.txt

garyemiller · 2021-11-30T01:15:25Z

Part 2 of 2. That is all fail2ban needs.

diff2.txt

But I'm guessing this needs a getopt().

garyemiller · 2021-11-30T02:09:56Z

Small change to diff2.txt. the message has \n it, so:

diff3.txt

I'll let the logs build up a bit, then write a fail2ban rule.

ghenry · 2021-11-30T12:20:20Z

Thanks for all this @garyemiller

I think I'll add this feature if that's OK as I need to update the docs, readme, man pages and tests etc.

If you are using just syslog, would you need the data saved in sqlite at all?

I was thinking:

-s enable logging via syslog
-S disable logging to database and use syslog only

ghenry · 2021-11-30T17:56:57Z

Hi Gary, I don't think we should log parsing command line options to syslog. Thanks.

ghenry · 2021-11-30T17:56:59Z

Hi Gary, Do we want to log a complete SIP message to syslog? Thanks.

ghenry · 2021-11-30T17:57:03Z

Hi Gary, Where are we closing syslog? Have you ever done a Pull Request before on GitHub? https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request I'd be tempted to do something like this: https://github.com/owntracks/recorder/blob/fd3f4631f16da09706eb1cde84a65322a24c31cb/util.c#L282 and add some tests etc. I'm also going to look at switching to getopt_long so we can toggle all these features with better options. Thanks.

garyemiller · 2021-11-30T19:53:17Z

think I'll add this feature if that's OK as I need to update the docs, readme, man pages and tests etc.

Yeah, a simple upgrade is never simple.

If you are using just syslog, would you need the data saved in sqlite at all?

Maybe, maybe not. I have not looked at the sqlite part yet. So smart to make them options and let the users decide.

I don't think we should log parsing command line options to syslog.

Not parsing, failure to parse. Some expect all errors to go into the logs. And it makes it easier for you to debug for others: just ask for the entire log. Complication is that the error may be before the user enables logging, if he enables logging. But, your code, your choice.

Where are we closing syslog?

I did not add that. as the man page says: "The use of closelog() is optional."

Can't hurt.

Do we want to log a complete SIP message to syslog?

That one is hard. syslog messages are one line, but SIP messages have \n in them. You would have to strip the \n, or change it to "\n" to fit on one line. fail2ban just needs the date (added by syslog), the offending IP, and maybe a snippet of what is being logged.

Have you ever done a Pull Request before on GitHub?

Nope. I looked at the doc and it looks way more complicated than on GitLab. And GitLab merges mess up a lot. So I prefer just sending git diffs.

getopt_long()

I'm a fan of getopt_long(), but it is not portable. They are GNU only. So you need to test for it, and fall back to getopt() if the local libc is not GNU.

ghenry · 2021-11-30T20:20:57Z

think I'll add this feature if that's OK as I need to update the docs, readme, man pages and tests etc.

Yeah, a simple upgrade is never simple.

:-)

If you are using just syslog, would you need the data saved in sqlite at all?

Maybe, maybe not. I have not looked at the sqlite part yet. So smart to make them options and let the users decide.

Agreed. I could see this running just in that type of agent/probe mode, also when I add other transport types.

I don't think we should log parsing command line options to syslog.

Not parsing, failure to parse. Some expect all errors to go into the logs. And it makes it easier for you to debug for others: just ask for the entire log. Complication is that the error may be before the user enables logging, if he enables logging. But, your code, your choice.

True. Also if this is running in systemd or some other way where cli args are passed other ways or via ENV.

Where are we closing syslog?

I did not add that. as the man page says: "The use of closelog() is optional."

Can't hurt.

syslog() works without openlog() too. I'll read up on that.

Do we want to log a complete SIP message to syslog?

That one is hard. syslog messages are one line, but SIP messages have \n in them. You would have to strip the \n, or change it to "\n" to fit on one line. fail2ban just needs the date (added by syslog), the offending IP, and maybe a snippet of what is being logged.

Yeah, I'll leave it out until someone asks for it.

Have you ever done a Pull Request before on GitHub?

Nope. I looked at the doc and it looks way more complicated than on GitLab. And GitLab merges mess up a lot. So I prefer just sending git diffs.

:-)

getopt_long()

I'm a fan of getopt_long(), but it is not portable. They are GNU only. So you need to test for it, and fall back to getopt() if the local libc is not GNU.

I'm searching to see where it's not portable to and I wonder if that's a use case we need to worry about.

Lastly @garyemiller

Thank you very much again for sharing your thoughts, ideas and trying SentryPeer out.

garyemiller · 2021-11-30T20:35:39Z

syslog() works without openlog() too. I'll read up on that.

Yes, but then you are stuck with defaults.

Yeah, I'll leave it out until someone asks for it.

That will not take long. Then you'll know their use case.

I'm searching to see where it's not portable to and I wonder if that's a use case we need to worry about.

There are a lot of POSIX compatible systems, that are not GNU. Now your code would work find on *BSD, osX, OpenWRT, maybe even WSL. They do not use glibc (GNU libc) Smaller systems reject glibc as too big. Mandating glibc will flush them out for you. The experience will not be pleasant.

Thank you very much again..

I'm just scrathing my own itch! :-).

ghenry · 2021-11-30T20:41:58Z

syslog() works without openlog() too. I'll read up on that.

Yes, but then you are stuck with defaults.

Ah, true.

Yeah, I'll leave it out until someone asks for it.

That will not take long. Then you'll know their use case.

Agreed.

I'm searching to see where it's not portable to and I wonder if that's a use case we need to worry about.

There are a lot of POSIX compatible systems, that are not GNU. Now your code would work find on *BSD, osX, OpenWRT, maybe even WSL. They do not use glibc (GNU libc) Smaller systems reject glibc as too big. Mandating glibc will flush them out for you. The experience will not be pleasant.

I look forward to the demand!

Thank you very much again..

I'm just scrathing my own itch! :-).

Me too 👍

ghenry · 2021-11-30T20:47:22Z

@garyemiller Could you upload a sample jail.conf etc. for sentrypeer once I add it?

garyemiller · 2021-11-30T20:51:19Z

Could you upload a sample jail.conf etc. for sentrypeer once I add it?

Creating that file is on my TODO list for the next few days. I'm already logging hack attempts to have something to test my new rule against.

ghenry · 2021-11-30T20:51:48Z

Could you upload a sample jail.conf etc. for sentrypeer once I add it?

Creating that file is on my TODO list for the next few days. I'm already logging hack attempts to have something to test my new rule against.

Cool. What is SentryPeer running on?

garyemiller · 2021-11-30T21:01:41Z

What is SentryPeer running on?

This is on my Gentoo Stable firewall. Console (ssh) only. No desktop, no X. It does have nginx, but I have not configured your web tool yet.

ghenry · 2021-11-30T21:03:20Z

Awesome!

…honeypot data - Added build requirement for libmicrohttpd and jansson - Added syslog support for use with [Fail2Ban](https://www.fail2ban.org/wiki/index.php/Main_Page) as per #6

ghenry · 2021-11-30T23:07:39Z

@garyemiller Please try main branch. Thanks.

garyemiller · 2021-12-01T00:22:16Z

Looks good.

Gentoo package name differ from yours, so messages like this do me little good:

configure: error: MHD_start_daemon() is not available. libmicrohttpd-dev / libmicrohttpd-devel is required.

configure: error: json_object() is not available. libjansson-dev / jansson-devel is required.

Gentoo calls those: net-libs/libmicrohttpd, dev-libs/jansson

Package names change fast, and vary from distro to distro. Hard to keep up.

Compiled and running fine.

I'm getting messages like this:

| ERROR | 2021-11-30 16:19:30.1459 <osip_messa:  886> Could not parse start line of message.
Cannot parse incoming SIP message.

Might be nice to log those for fail2ban as well.

Now I need some hackers to generate some more logs for me.

I might have used a slightly different coding style.

Instead of:

   fprintf(stderr, "  -h,      Print this help\n");
   fprintf(stderr, "  -V,      Print version\n");
   fprintf(stderr, "  -s,      Enable syslog logging\n");
   fprintf(stderr, "  -v,      Enable verbose logging\n");
   fprintf(stderr, "  -d,      Enable debug mode\n");

This:

   fprintf(stderr, "  -h,      Print this help\n"
                        "  -V,      Print version\n"
                        "  -s,      Enable syslog logging\n";
                        "  -v,      Enable verbose logging\n"
                        "  -d,      Enable debug mode\n");

Fewer function calls. Note GitHub messes up the indents...

And instead of this:

    if (config.debug_mode || config.verbose_mode) {
            fprintf(stderr, "Starting %s...\n", PACKAGE_NAME);
            if (config.syslog_mode) {
                   syslog(LOG_ERR, "Starting %s...\n", PACKAGE_NAME);
           }
   }

This:

    if (config.debug_mode || config.verbose_mode) {
            const char *fmt = "Starting %s...\n";
            fprintf(stderr, fmt, PACKAGE_NAME);
            if (config.syslog_mode) {
                   syslog(LOG_ERR, fmt, PACKAGE_NAME);
           }
   }

The messages can not get out of sync that way

garyemiller · 2021-12-01T01:18:17Z

I'm testing this now:

filter.d/sentrypeer.conf:

[INCLUDES]

before = common.conf

[Definition]
# Source IP: 216.244.81.66, Method: OPTIONS, Agent: friendly-scanner
failregex="Source IP: <HOST>, Method:.*$"
ignoreregex=

And in my jail.conf:

[sentrypeer]
enabled = true
filter=sentrypeer
logpath=/var/log/messages
action=nftables-allports[name=sentrypeer, bantime=1day]
maxretry = 1
findtime=1day
bantime=1week

As root, this is a simple way to check your failregex can find sentrypeer in the log file:

# fail2ban-regex /var/log/messages "Source IP: <HOST>, Method:.*$"

Still testing, so there may be more tweaks.

ghenry · 2021-12-01T13:29:49Z

Thanks. I'll await your testing feedback.

ghenry · 2021-12-01T22:02:22Z

That's all the latest changes pushed. SIP that has failed to parse has just the source IP logged via syslog too.

Let me know how you get on. I've had to make some changes to autotools to find libmicrohttpd on the GitHub Action ubuntu-lastest runners as I've installed it via Homebrew as it's too old via apt. I think I'll revert them actually as they don't seem right.

Thanks.

ghenry · 2021-12-10T07:31:32Z

Hi Gary,

How's your jail conf testing going?

I've added a few things too:

https://github.com/SentryPeer/SentryPeer/blob/main/CHANGELOG.md#unreleased

Thanks.

garyemiller · 2021-12-10T18:51:43Z

How's your jail conf testing going?

Not well. "fail2ban-regex " was working, but when I try against the conf file, "fail2ban-regex sentrypeer" it does not: Also not working live. After beating my head against the wall, I got pulled onto another bug. Sleeping on it usually helps me when I hit a wall like that. Gonna be something simple and stupid.

ghenry · 2021-12-10T21:01:22Z

Thanks for the update. No rush. Just wanted to check in with you. Gavin.

ghenry · 2022-01-27T08:25:41Z

Morning Gary,

Tried v1?

Thanks.

garyemiller added the enhancement New feature or request label Nov 29, 2021

ghenry self-assigned this Nov 29, 2021

ghenry changed the title ~~SentryPeer~~ Fail2ban integration Nov 29, 2021

ghenry changed the title ~~Fail2ban integration~~ Fail2ban Integration Nov 29, 2021

ghenry mentioned this issue Nov 30, 2021

Record dest_ip and being bound to multiple IPs #3

Closed

ghenry changed the title ~~Fail2ban Integration~~ Fail2ban Integration via syslog Nov 30, 2021

ghenry closed this as completed Feb 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fail2ban Integration via syslog #6

Fail2ban Integration via syslog #6

garyemiller commented Nov 29, 2021

ghenry commented Nov 29, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021 via email

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021

garyemiller commented Dec 1, 2021 •

edited

garyemiller commented Dec 1, 2021

ghenry commented Dec 1, 2021

ghenry commented Dec 1, 2021

ghenry commented Dec 10, 2021

garyemiller commented Dec 10, 2021

ghenry commented Dec 10, 2021

ghenry commented Jan 27, 2022

Fail2ban Integration via syslog #6

Fail2ban Integration via syslog #6

Comments

garyemiller commented Nov 29, 2021

ghenry commented Nov 29, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021 via email

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021

garyemiller commented Nov 30, 2021

ghenry commented Nov 30, 2021 via email

ghenry commented Nov 30, 2021

garyemiller commented Dec 1, 2021 • edited

garyemiller commented Dec 1, 2021

ghenry commented Dec 1, 2021

ghenry commented Dec 1, 2021

ghenry commented Dec 10, 2021

garyemiller commented Dec 10, 2021

ghenry commented Dec 10, 2021

ghenry commented Jan 27, 2022

garyemiller commented Dec 1, 2021 •

edited