127.0.0.1 on firehol_level4

[wolfsden3]

I found a curious thing happen with att.com, the firewall I use was blocking it and saying it was coming from the level4 block list. It turns out that 127.0.0.1 (localhost) is in the level4 block list for some reason.

Can you take that off the list? I don't think it's supposed to be on the list.

Thanks.

[ktsaou]

Hm...

right now update-ipsets does not filter-out any IPs.
I resist to the idea of filtering out IPs, since this may significantly alter the effectiveness of certain lists. After all, update-ipsets should not get any such decisions.

Could you please explain how you use it?

I understand that you could use any of these strategies:

1. match the blacklist only on the internet interface. If you do this right, you should want to block packets from/to 127.0.0.1 since they are by definition bad packets.
2. use a whitelist on your firewall, and only blacklist IPs that are not in the whitelist. This is also a good strategy, given that bad ip list maintenance may block you out of your systems.

The last resort could be to use iprange to filter out the whitelisted IPs from the ip feeds you use. To do this, create a text file with the IPs you want to whitelist and run:

iprange blacklist --exclude-next whitelist >accepted_blacklist

However I strongly suggest to follow both 1 and 2. If you need help achieving this, post here and I'll help you.

[wolfsden3]

You're not understanding what I'm saying I guess.

if you look at the list: https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level4.netset

...and do a cntrl + find > then type in: 127.0.0.1

I don't believe that localhost address should be in the list as "localhost" or "127.0.0.1" is clearly in the list.

Am I wrong?

Thanks.

[ktsaou]

ok. I think I perfectly understood you already. So, let me show you.

Check what else firehol_level4 includes:

 # iprange firehol_level4.netset --common bogons.netset
10.1.0.250
10.24.0.165
10.33.0.102
10.107.0.186
10.240.0.10
10.255.255.200
127.0.0.1
172.16.0.101
192.168.0.1
192.168.0.7
192.168.1.1
192.168.1.8
192.168.1.31
192.168.2.4
192.168.3.100
192.168.30.94
192.168.33.201
192.168.51.2
192.168.52.28
192.168.65.8
192.168.100.73
192.168.151.0
192.168.224.1
238.209.5.182

All these are private IPs, unroutable to the internet. Shall I remove them?

Since they are listed on firehol_level4 someone got an attack from these IPs. He was receiving packets from the internet with source IP, one of these IPs.

These IPs should not be a problem for you, if your firewall was blacklisting IPs the right way.

Apparently you are blocking all traffic from/to firehol_level4 without checking anything else. This is the problem. You should only block firehol_level4 on your internet interface. This way the IP 127.0.0.1 should have been blocked only if it was found on the internet interface, and you should want this to happen.

Is it more clear now?

[wolfsden3]

I would think the private blocks wouldn't be on a public block list and
that a separate list of bogons is what's used (Link
https://github.com/firehol/blocklist-ipsets/blob/master/iblocklist_cidr_report_bogons.netset).
Do your 4 or so levels / lists include bogons or is that separate?

If someone has been attacked from a bogon I'd think that was their problem
and not the rest of us or anyone using a public block list.

I'd suggest you filter out bogons on the general level lists because those
networks / IP ranges are on a separate bogon list that people can download
an use - if they had, they wouldn't have been attacked. :-)

Thanks!

On Mon, Jun 6, 2016 at 3:00 PM, Costa Tsaousis notifications@github.com
wrote:

ok. I think I perfectly understood you already. So, let me show you.

Check what else firehol_level4 includes:

iprange firehol_level4.netset --common bogons.netset

10.1.0.250
10.24.0.165
10.33.0.102
10.107.0.186
10.240.0.10
10.255.255.200
127.0.0.1
172.16.0.101
192.168.0.1
192.168.0.7
192.168.1.1
192.168.1.8
192.168.1.31
192.168.2.4
192.168.3.100
192.168.30.94
192.168.33.201
192.168.51.2
192.168.52.28
192.168.65.8
192.168.100.73
192.168.151.0
192.168.224.1
238.209.5.182

All these are private IPs, unroutable to the internet. Shall I remove them?

Since they are listed on firehol_level4 someone got an attack from these
IPs. He was receiving packets from the internet with source IP, one of
these IPs.

These IPs should not be a problem for you, if your firewall was
blacklisting IPs the right way.

Apparently you are blocking all traffic from/to firehol_level4 without
checking anything else. This is the problem. You should only block
firehol_level4 on your internet interface. This way the IP 127.0.0.1
should have been blocked only if it was found on the internet interface,
and you should want this to happen.

Is it more clear now?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#143 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ARvgc0JSS1PbX6wjCBS8goAOCebmTqwCks5qJG5XgaJpZM4IvF1P
.

Rafael

[ktsaou]

Thanks! However, it is more complex than that. firehol_level1 for example, includes bogons by design. I intentionally added it for blocking all the IPs that should never appear on your internet interface.

firehol_level4 should be used on top of level3, level2 and level1. Each additional level (starting from level1) filters out more IPs with the risk of more false positives.

firehol_level4 is a very risky IP list. As its description says it may include a large number of false positives.

I have designed these levels so that under normal conditions you should only use level1. If you face an attack, you include level2 (attacks of the last 48 hours). If it is not effective include level3 (attacks of the last 30 days). If, even all these are not effective, then include level4. With level4, you risk blacklisting some of your legit users but it may help you to stop the attack.

Of course you are not forced to use these. Just combine the ones you need yourself (and even exclude IP lists you don't want included). I have provided all the tools for this, so you can do it. Of course if you believe you have combined a list that can be useful to others too, just give me the rules and I'll add them to update-ipsets so that others can download the final product of it.

[wolfsden3]

No problem - I can filter them out on my side.

Thanks,

Rafael

On Tue, Jun 7, 2016 at 12:59 PM, Costa Tsaousis notifications@github.com
wrote:

Thanks! However, it is more complex than that. firehol_level1 for example,
includes bogons by design. I intentionally added it for blocking all the
IPs that should never appear on your internet interface.

firehol_level4 should be used on top of level3, level2 and level1. Each
additional level (starting from level1) filters out more IPs with the risk
of more false positives.

firehol_level4 is a very risky IP list. As its description says it may
include a large number of false positives.

I have designed these levels so that under normal conditions you should
only use level1. If you face an attack, you include level2 (attacks of the
last 48 hours). If it is not effective include level3 (attacks of the last
30 days). If, even all these are not effective, then include level4. With
level4, you risk blacklisting some of your legit users but it may help you
to stop the attack.

Of course you are not forced to use these. Just combine the ones you need
yourself (and even exclude IP lists you don't want included). I have
provided all the tools for this, so you can do it. Of course if you believe
you have combined a list that can be useful to others too, just give me the
rules and I'll add them to update-ipsets so that others can download the
final product of it.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#143 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ARvgc-qbgcGdmcPO4hBpjBp_AFEeBKBWks5qJaNWgaJpZM4IvF1P
.

Rafael

[Steve-Newcomb]

I have run into a similar problem with localhost's IP address(es). Ktsaou's solution -- to subject only internet interfaces to internet blacklisting -- seems very correct but it adds complexity to the maintenance of various workstations and servers. The extra complexity comes from the fact that interface names differ with hardware/driver combinations, and they are subject to considerable variation and change.

My current solution is to use firehol_level1, but to omit enabling the "fullbogons" ipset. I don't see any convenient way to use fullbogons, even though I would like to, since it includes 127.0.0.0/16 and there's no convenient way to exclude it.

[ktsaou]

hm... which firewall do you use? how do you apply the blocklist?
If it is that hard to do it at your firewall, you could use iprange to exclude all private IPs from an ipset.

[Steve-Newcomb]

We use firehol. (As we have for many years.) The relevant lines in our firehol.conf are:

ipv4 ipset create firehol_level1 hash:net
ipv4 ipset addfile firehol_level1 /etc/firehol/ipsets/firehol_level1.netset
blacklist4 full ipset:firehol_level1

If there's a way to exclude only 127.0.0.0/16 from the full firehol_level1 while using update-ipsets, we would like to know how to do it.

[ktsaou]

ok, then it is easy:

blacklist4 full ipset:firehol_level1 except src "${PRIVATE_IPS}"

Here is what it does:

# firehol explain

FireHOL 3.1.4_master
(C) Copyright 2003-2015 Costa Tsaousis <costa@tsaousis.gr>
(C) Copyright 2012-2015 Phil Whineray <phil@firehol.org>
FireHOL is distributed under the GPL v2+.
Home Page: http://firehol.org

-------------------------------------------------------------------------
Get notified of new FireHOL releases by subscribing to the mailing list:
    http://lists.firehol.org/mailman/listinfo/firehol-support/
-------------------------------------------------------------------------

You can now start typing firehol configuration directives.
Special interactive commands: help, show, quit

# FireHOL [:] > blacklist4 full ipset:firehol_level1 except src "${PRIVATE_IPS}"
# \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
# Cmd Line : 1@Interactive User Input
# Command  : blacklist4 full ipset:firehol_level1 except src "${PRIVATE_IPS}"

# Blacklist input chain
/sbin/iptables -t filter -N BLACKLIST.bi.1.in 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 10.0.0.0/8 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 169.254.0.0/16 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 172.16.0.0/12 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 192.0.2.0/24 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 192.88.99.0/24 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -s 192.168.0.0/16 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -j LOG --log-level warning --log-prefix=\"BLACKLIST-IN:\" 
/sbin/iptables -t filter -A BLACKLIST.bi.1.in -j DROP 

# Blacklist input
/sbin/iptables -t filter -A INPUT -m set --match-set firehol_level1 src \! --update-counters \! --update-subcounters -j BLACKLIST.bi.1.in 
/sbin/iptables -t filter -A FORWARD -m set --match-set firehol_level1 src \! --update-counters \! --update-subcounters -j BLACKLIST.bi.1.in 

# Blacklist output chain
/sbin/iptables -t filter -N BLACKLIST.bi.1.out 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 10.0.0.0/8 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 169.254.0.0/16 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 172.16.0.0/12 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 192.0.2.0/24 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 192.88.99.0/24 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -d 192.168.0.0/16 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -j LOG --log-level warning --log-prefix=\"BLACKLIST-OUT:\" 
/sbin/iptables -t filter -A BLACKLIST.bi.1.out -j REJECT --reject-with icmp-host-unreachable 

# Bidirectional blacklist rules
/sbin/iptables -t filter -A FORWARD -m set --match-set firehol_level1 dst \! --update-counters \! --update-subcounters -j BLACKLIST.bi.1.out 
/sbin/iptables -t filter -A OUTPUT -m set --match-set firehol_level1 dst \! --update-counters \! --update-subcounters -j BLACKLIST.bi.1.out 

# > OK <

As you can see, it first matches the ipset and then it excudes the private IPs (src in input, dst on output).

[ktsaou]

Instead of using PRIVATE_IPS I suggest to use the IP address space you know you use, like this:

blacklist4 full ipset:firehol_level1 except src "127.0.0.0/8 10.0.0.0/8"

So, allow the blacklist to match private IPs you know you don't use.

[ktsaou]

And you can also whitelist interfaces like this:

blacklist4 full ipset:firehol_level1 except inface lo or src "127.0.0.0/8 10.0.0.0/8"

I see there is bug in firehol. We should be able to say this (which is more efficient compared to the above):

blacklist4 full ipset:firehol_level1 inface not lo except src "127.0.0.0/8 10.0.0.0/8"

but the parsing of inface is wrong in the blacklist helper. I'll make a PR to fix it.

[ktsaou]

fixed it. Will merge in a while.

After it is merged, you can also use:

blacklist4 full inface not lo src ipset:firehol_level1 except src "10.0.0.0/8"

which produces this:

# ./sbin/firehol explain

FireHOL 3.1.4_master
(C) Copyright 2003-2015 Costa Tsaousis <costa@tsaousis.gr>
(C) Copyright 2012-2015 Phil Whineray <phil@firehol.org>
FireHOL is distributed under the GPL v2+.
Home Page: http://firehol.org

-------------------------------------------------------------------------
Get notified of new FireHOL releases by subscribing to the mailing list:
    http://lists.firehol.org/mailman/listinfo/firehol-support/
-------------------------------------------------------------------------

You can now start typing firehol configuration directives.
Special interactive commands: help, show, quit

# FireHOL [:] > blacklist4 full inface not lo src ipset:firehol_level1 except src "10.0.0.0/8"
# \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
# Cmd Line : 3@Interactive User Input
# Command  : blacklist4 full inface not lo src ipset:firehol_level1 except src "10.0.0.0/8"

# Blacklist input chain
/sbin/iptables -t filter -N BLACKLIST.bi.3.in 
/sbin/iptables -t filter -A BLACKLIST.bi.3.in -s 10.0.0.0/8 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.3.in -j LOG --log-level warning --log-prefix=\"BLACKLIST-IN:\" 
/sbin/iptables -t filter -A BLACKLIST.bi.3.in -j DROP 

# Blacklist input
/sbin/iptables -t filter -A INPUT \! -i lo -m set --match-set firehol_level1 src \! --update-counters \! --update-subcounters -j BLACKLIST.bi.3.in 
/sbin/iptables -t filter -A FORWARD \! -i lo -m set --match-set firehol_level1 src \! --update-counters \! --update-subcounters -j BLACKLIST.bi.3.in 

# Blacklist output chain
/sbin/iptables -t filter -N BLACKLIST.bi.3.out 
/sbin/iptables -t filter -A BLACKLIST.bi.3.out -d 10.0.0.0/8 -j RETURN 
/sbin/iptables -t filter -A BLACKLIST.bi.3.out -j LOG --log-level warning --log-prefix=\"BLACKLIST-OUT:\" 
/sbin/iptables -t filter -A BLACKLIST.bi.3.out -j REJECT --reject-with icmp-host-unreachable 

# Bidirectional blacklist rules
/sbin/iptables -t filter -A FORWARD \! -o lo -m set --match-set firehol_level1 dst \! --update-counters \! --update-subcounters -j BLACKLIST.bi.3.out 
/sbin/iptables -t filter -A OUTPUT \! -o lo -m set --match-set firehol_level1 dst \! --update-counters \! --update-subcounters -j BLACKLIST.bi.3.out 

# > OK <

[ktsaou]

merged it.