New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mmanon rewrite #1723

Open
jgerhards opened this Issue Aug 16, 2017 · 3 comments

Comments

Projects
None yet
5 participants
@jgerhards
Contributor

jgerhards commented Aug 16, 2017

several improvements for mmanon. This is a meta issue tracker used for referencing all the corresponding issues and information.

Background Articles

Issues

For the time being, i consider this done. However, i will leave the issue open so i can later implement the other requested options.

@duritong

This comment has been minimized.

Show comment
Hide comment
@duritong

duritong Aug 19, 2017

So I'm using it with:

# cat /etc/rsyslog.d/mmanon.conf 
module(load="mmanon")
action(type="mmanon")

=> default config.

What is important, that it catches ALL ip entries. :-)

I also observed the following by using the default config, which is given by the nature of only wiping 16bits by default:

Certain ISPs have PTR records with the reverse IP address in them. Meaning, if I have the IP 1.2.3.4 my ISP might have added the following PTR record: 4.3.2.1.dialup.myisp.com, which means after mmanon we get:

IP: 1.2.x.x
PTR: 4.3.x.x.dialup.myisp.com

=> It kinda makes the whole work obsolete. You can find such entries e.g. in maillog.

You could choose a ipv4.bits > 16, to address this, but then sometimes the 16bits are nice to detect pattern in attacks. On the other hand, there are also ISPs with PTR entries such as: 1-2-3-4.dialup.myisp.com which is not catched by mmanon at all. Maybe we could also try to "catch" these? Which I would find more important than trying to address the reverse problem.

duritong commented Aug 19, 2017

So I'm using it with:

# cat /etc/rsyslog.d/mmanon.conf 
module(load="mmanon")
action(type="mmanon")

=> default config.

What is important, that it catches ALL ip entries. :-)

I also observed the following by using the default config, which is given by the nature of only wiping 16bits by default:

Certain ISPs have PTR records with the reverse IP address in them. Meaning, if I have the IP 1.2.3.4 my ISP might have added the following PTR record: 4.3.2.1.dialup.myisp.com, which means after mmanon we get:

IP: 1.2.x.x
PTR: 4.3.x.x.dialup.myisp.com

=> It kinda makes the whole work obsolete. You can find such entries e.g. in maillog.

You could choose a ipv4.bits > 16, to address this, but then sometimes the 16bits are nice to detect pattern in attacks. On the other hand, there are also ISPs with PTR entries such as: 1-2-3-4.dialup.myisp.com which is not catched by mmanon at all. Maybe we could also try to "catch" these? Which I would find more important than trying to address the reverse problem.

@kargig

This comment has been minimized.

Show comment
Hide comment
@kargig

kargig Aug 19, 2017

I mostly use the following configuration:

mail.*              { action(type="mmanon" ipv4.bits="32" mode="simple" replacementChar="*")
                      /var/log/mail.log
                    }

Regarding dashes between hostnames, I once made the following patch which seems to work at least for Debian shipping rsyslog version 8.23. It's a hack that doesn't deal with the reverse order of strings contained in PTR records, so it's fine for my use case just because I use ipv4.bits=32.

--- a/plugins/mmanon/mmanon.c	2017-08-19 17:44:19.731721637 +0300
+++ b/plugins/mmanon/mmanon.c	2017-08-19 17:47:17.843153029 +0300
@@ -304,17 +304,17 @@
 	/* got digit, let's see if ip */
 	ipstart[0] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr = octet << 24;
 	++i;
 	ipstart[1] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr |= octet << 16;
 	++i;
 	ipstart[2] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr |= octet << 8;
 	++i;
 	ipstart[3] = i;
@@ -333,7 +333,7 @@
 		else /* due to our checks, this *must* be 32 */
 			j = ipstart[0];
 		while(j < i) {
-			if(msg[j] != '.')
+			if(msg[j] != '.' && msg[j] != '-')
 				msg[j] = pData->replChar;
 			++j;
 		}

https://paste.debian.net/982130/

I could submit a PR but since you're rewriting stuff I don't know if it makes any sense to you. Tell me if you're interested so I can do a PR against the master.

kargig commented Aug 19, 2017

I mostly use the following configuration:

mail.*              { action(type="mmanon" ipv4.bits="32" mode="simple" replacementChar="*")
                      /var/log/mail.log
                    }

Regarding dashes between hostnames, I once made the following patch which seems to work at least for Debian shipping rsyslog version 8.23. It's a hack that doesn't deal with the reverse order of strings contained in PTR records, so it's fine for my use case just because I use ipv4.bits=32.

--- a/plugins/mmanon/mmanon.c	2017-08-19 17:44:19.731721637 +0300
+++ b/plugins/mmanon/mmanon.c	2017-08-19 17:47:17.843153029 +0300
@@ -304,17 +304,17 @@
 	/* got digit, let's see if ip */
 	ipstart[0] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr = octet << 24;
 	++i;
 	ipstart[1] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr |= octet << 16;
 	++i;
 	ipstart[2] = i;
 	octet = getnum(msg, lenMsg, &i);
-	if(octet > 255 || msg[i] != '.') goto done;
+	if(octet > 255 || (msg[i] != '.' && msg[i] != '-')) goto done;
 	ipv4addr |= octet << 8;
 	++i;
 	ipstart[3] = i;
@@ -333,7 +333,7 @@
 		else /* due to our checks, this *must* be 32 */
 			j = ipstart[0];
 		while(j < i) {
-			if(msg[j] != '.')
+			if(msg[j] != '.' && msg[j] != '-')
 				msg[j] = pData->replChar;
 			++j;
 		}

https://paste.debian.net/982130/

I could submit a PR but since you're rewriting stuff I don't know if it makes any sense to you. Tell me if you're interested so I can do a PR against the master.

@mejo-

This comment has been minimized.

Show comment
Hide comment
@mejo-

mejo- Aug 19, 2017

I use it with the following configuration:

$ModLoad mmanon
action(type="mmanon" ipv4.bits="32" mode="rewrite")

I agree with @duritong, that it's most important that IPs are anonymized. In my case, the IPs shall be fully anonymized.

mejo- commented Aug 19, 2017

I use it with the following configuration:

$ModLoad mmanon
action(type="mmanon" ipv4.bits="32" mode="rewrite")

I agree with @duritong, that it's most important that IPs are anonymized. In my case, the IPs shall be fully anonymized.

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Aug 23, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Aug 23, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Aug 24, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Aug 24, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 1, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 2, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 2, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 2, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 2, 2017

jgerhards added a commit to jgerhards/rsyslog that referenced this issue Sep 2, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment