New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IPv6 support master plan #1123

Open
yarikoptic opened this Issue Jul 22, 2015 · 29 comments

Comments

Projects
None yet
@yarikoptic
Member

yarikoptic commented Jul 22, 2015

I think that #39 grew out of the proportions to be any longer efficient. So I have decided to organize a new ticket to discuss the high-level development pieces which could largely be taken in steps. Let's continue discussion here at that level, and whenever we finalize the possible plan, detail specific steps in separate issues.
So, logical workflow to move off the dead point could IMHO could be be

  1. Provide basic support for IPv6 addresses in Fail2Ban, at the beginning assuming very conservative prefix (/128). For that -- review/adopt previous work (https://github.com/grooverdan/fail2ban/tree/ipv6, https://github.com/grooverdan/fail2ban/tree/ipv6_2 etc ), without touching actual action definitions, incorporating prefix within the ban ticket (for possible future treatment in 2.) and possibly adopting/using https://github.com/google/ipaddr-py .
  2. Investigate more and decide on how to deal with variable across use-cases prefixes. Suggested approaches were to
    • variable bantime (@sebres #39 (comment)) depending on the number of failed attempts within the subnet

    • automatic shrinking of the prefix (@rotanid #39 (comment)) given growing number of attacks

    • adhere to some very conservative default while maintaining a custom list of network prefixes for known providers/addresses

      Also @neirbowj information in #39 (comment) .

  3. Look into establishing definitions for support of both ipv4 and ipv6 within action files. See my summary of considered approaches in #39 (comment)

Although every next step depends on the previous one, I think they have enough of independence to continue discussion in separate issues. ATM
I just want to hear from other @fail2ban/contributors and the community if this sounds reasonable.

@brianjmurrell

This comment has been minimized.

Show comment
Hide comment
@brianjmurrell

brianjmurrell Jul 22, 2015

I think this reflects the separation of issues that I was previously advocating. Basic, equivalent-to-current-IPv4 support for IPv6. Land it, release it.

Then, and only then, start to look at general wider prefix handling generic enough to be applicable to both IPv4 and IPv6. Land it, release it.

And then go from there.

brianjmurrell commented Jul 22, 2015

I think this reflects the separation of issues that I was previously advocating. Basic, equivalent-to-current-IPv4 support for IPv6. Land it, release it.

Then, and only then, start to look at general wider prefix handling generic enough to be applicable to both IPv4 and IPv6. Land it, release it.

And then go from there.

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Jul 22, 2015

Member

I am not sure we will land/release 1 alone since, as was stated, it could immediately be exploited by an attacker to cause resources exhaustion/DoS. May be only if treatment of IPv6 addresses would be made optional with a big fat warning on possible ramifications.

Member

yarikoptic commented Jul 22, 2015

I am not sure we will land/release 1 alone since, as was stated, it could immediately be exploited by an attacker to cause resources exhaustion/DoS. May be only if treatment of IPv6 addresses would be made optional with a big fat warning on possible ramifications.

@mrc0mmand

This comment has been minimized.

Show comment
Hide comment
@mrc0mmand

mrc0mmand Jul 22, 2015

Member

Maybe something like gradual banning of larger subnets would help? Eg. if f2b bans 25 IPs from /121 subnet, it will ban entire /120 and remove all previous bans for /121. If it later bans another 50 from /120, it will again remove all bans for /120 and ban entire /119. This could be still exploited in some way, but it wouldn't be so easy/fast. (Also I'm not sure if it's technically correct/possible, it's just a thought).

Member

mrc0mmand commented Jul 22, 2015

Maybe something like gradual banning of larger subnets would help? Eg. if f2b bans 25 IPs from /121 subnet, it will ban entire /120 and remove all previous bans for /121. If it later bans another 50 from /120, it will again remove all bans for /120 and ban entire /119. This could be still exploited in some way, but it wouldn't be so easy/fast. (Also I'm not sure if it's technically correct/possible, it's just a thought).

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Jul 22, 2015

Member

@yarikoptic I can make a IPv6 version as described in your plan, but:

  • I will make it iptables/debian only, need help from another contributers for other systems;
  • unfortunately I have no interest to make it, as long as #716 and my feature branch based on it + #1116 are not completed resp. both are not mainline (not master), because I see too many conflicts hereafter;
  • I see the subnet support as a necessary part of the IPv6, also because of memory exhaustion, increase factor for failure count or time of ban should be applied for subnet's (single IP would be a subset of it), and many other things, why it is really necessary.
Member

sebres commented Jul 22, 2015

@yarikoptic I can make a IPv6 version as described in your plan, but:

  • I will make it iptables/debian only, need help from another contributers for other systems;
  • unfortunately I have no interest to make it, as long as #716 and my feature branch based on it + #1116 are not completed resp. both are not mainline (not master), because I see too many conflicts hereafter;
  • I see the subnet support as a necessary part of the IPv6, also because of memory exhaustion, increase factor for failure count or time of ban should be applied for subnet's (single IP would be a subset of it), and many other things, why it is really necessary.
@rotanid

This comment has been minimized.

Show comment
Hide comment
@rotanid

rotanid Jul 22, 2015

the plan sounds good - and i would prefer the optional "1" with default to off and big warning being released. there are use cases, where this is better than f2b doing nothing for IPv6 at all.
@mrc0mmand please read the ticket! as far as i understand it, "automatic shrinking of the prefix" does exactly what you are suggesting.

rotanid commented Jul 22, 2015

the plan sounds good - and i would prefer the optional "1" with default to off and big warning being released. there are use cases, where this is better than f2b doing nothing for IPv6 at all.
@mrc0mmand please read the ticket! as far as i understand it, "automatic shrinking of the prefix" does exactly what you are suggesting.

@mrc0mmand

This comment has been minimized.

Show comment
Hide comment
@mrc0mmand

mrc0mmand Jul 23, 2015

Member

@rotanid I honestly don't know how I could miss that, but now I see it. So sorry and thanks - it's exactly what I mean.

Also, even with 'automatic prefix shrinking' there would be a huge amount of IP addresses, which would be kind of uncomfortable or even slow if we store them in iptables. Wouldn't be eg. ipset better 'default' solution for that? I know even ipset has its limits, but it would be much faster (and probably convenient) than storing that load of IPv6 addresses in iptables.

Member

mrc0mmand commented Jul 23, 2015

@rotanid I honestly don't know how I could miss that, but now I see it. So sorry and thanks - it's exactly what I mean.

Also, even with 'automatic prefix shrinking' there would be a huge amount of IP addresses, which would be kind of uncomfortable or even slow if we store them in iptables. Wouldn't be eg. ipset better 'default' solution for that? I know even ipset has its limits, but it would be much faster (and probably convenient) than storing that load of IPv6 addresses in iptables.

@mfechner

This comment has been minimized.

Show comment
Hide comment
@mfechner

mfechner Jul 28, 2015

Could it be possible to implement at first only a block for the specific IPv6 IP?
I get tons of logs in my server that fail2ban cannot block the address, because it is an IPv6 address.
You can add the /64 or /56 block feature later.

mfechner commented Jul 28, 2015

Could it be possible to implement at first only a block for the specific IPv6 IP?
I get tons of logs in my server that fail2ban cannot block the address, because it is an IPv6 address.
You can add the /64 or /56 block feature later.

@Xoib

This comment has been minimized.

Show comment
Hide comment
@Xoib

Xoib Jul 28, 2015

@idefix6 It has already been discussed in #39. Btw, this new topic is to discuss the proper way to implement this, if you need a workaround there are plenty in #39.

Xoib commented Jul 28, 2015

@idefix6 It has already been discussed in #39. Btw, this new topic is to discuss the proper way to implement this, if you need a workaround there are plenty in #39.

@buanzo

This comment has been minimized.

Show comment
Hide comment
@buanzo

buanzo Jul 28, 2015

Contributor

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

Contributor

buanzo commented Jul 28, 2015

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Jul 28, 2015

Member

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

I don't think it was discussed, but I am also not yet familiar with this
so can't stay how feasible it will be.

Member

yarikoptic commented Jul 28, 2015

Currently we have one table per jail with all banned IPs. We could apply a
hierarchical approach, based on ipv6 grouping, to ease the load? Sorry if
this has been discussed already.

I don't think it was discussed, but I am also not yet familiar with this
so can't stay how feasible it will be.

@Elompenta

This comment has been minimized.

Show comment
Hide comment
@Elompenta

Elompenta Dec 1, 2015

Do you have any new information to support IPv6 banning?

Elompenta commented Dec 1, 2015

Do you have any new information to support IPv6 banning?

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Dec 2, 2015

Member

Short - no, we have not. But I work on the solution, unfortunately fewer as secondary priority (there much to do).
@fail2ban/developers any thoughts?

Member

sebres commented Dec 2, 2015

Short - no, we have not. But I work on the solution, unfortunately fewer as secondary priority (there much to do).
@fail2ban/developers any thoughts?

@f0o

This comment has been minimized.

Show comment
Hide comment
@f0o

f0o Dec 4, 2015

👍 to IPv6 Support :)

f0o commented Dec 4, 2015

👍 to IPv6 Support :)

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Dec 4, 2015

Member

+1
....
Did it help? Not really... We know that it is a needed feature, and this issue is not to seek feedback on its desirability, but about the plan. Provide feedback accordingly. Or better -- submit PRs addressing specific parts of the plan for review so feature actually gets implemented. Thanks

edit 1: all non-constructive/0-bit-of-relevant-information comments get removed from this thread

Member

yarikoptic commented Dec 4, 2015

+1
....
Did it help? Not really... We know that it is a needed feature, and this issue is not to seek feedback on its desirability, but about the plan. Provide feedback accordingly. Or better -- submit PRs addressing specific parts of the plan for review so feature actually gets implemented. Thanks

edit 1: all non-constructive/0-bit-of-relevant-information comments get removed from this thread

@GRMrGecko

This comment has been minimized.

Show comment
Hide comment
@GRMrGecko

GRMrGecko Apr 16, 2016

So, as I understand it:

  1. Get fail2ban to parse IPv6 addresses from logs. I don't think they are logged differently from IPv4.
  2. At first, have it detect only the /128 prefix for attacks and later have features where it'll see that multiple IPv6 addresses from a similar prefix is attacking and block that prefix.
    From my research, ISPs normally give a /64 prefix to customers, but would be good to be able to grow the prefix depending on the range of attacks.
  3. At first, do not modify actions for IPv6, but later maybe have the option to have actionban6 and actionunban6 to specify what to do specifically for IPv6 addresses.
    As far as I know, UFW will support banning IPv6 in the same way you ban IPv4, so an actionban6 type thing is not needed for it, but I can see where you may need something for say example ip6tables.
    I think having an actionban6 type variable would be easy to implement.

I believe that getting the basic support can be done rather quickly, but as I have not reviewed the code in full I could be mistaken.

I believe IPv6 support is important as many hosting companies provide IPv6 (such as my own). I would be happy to help implement support for IPv6 once this semester in school is over.

To cut down on spam, I think people should thumb up the post using githubs "Add your reaction" button instead of writing a post saying +1 or something. I for one want IPv6 support, so I have thumbed up the post.

Thank you for your work on this project fail2ban team.

GRMrGecko commented Apr 16, 2016

So, as I understand it:

  1. Get fail2ban to parse IPv6 addresses from logs. I don't think they are logged differently from IPv4.
  2. At first, have it detect only the /128 prefix for attacks and later have features where it'll see that multiple IPv6 addresses from a similar prefix is attacking and block that prefix.
    From my research, ISPs normally give a /64 prefix to customers, but would be good to be able to grow the prefix depending on the range of attacks.
  3. At first, do not modify actions for IPv6, but later maybe have the option to have actionban6 and actionunban6 to specify what to do specifically for IPv6 addresses.
    As far as I know, UFW will support banning IPv6 in the same way you ban IPv4, so an actionban6 type thing is not needed for it, but I can see where you may need something for say example ip6tables.
    I think having an actionban6 type variable would be easy to implement.

I believe that getting the basic support can be done rather quickly, but as I have not reviewed the code in full I could be mistaken.

I believe IPv6 support is important as many hosting companies provide IPv6 (such as my own). I would be happy to help implement support for IPv6 once this semester in school is over.

To cut down on spam, I think people should thumb up the post using githubs "Add your reaction" button instead of writing a post saying +1 or something. I for one want IPv6 support, so I have thumbed up the post.

Thank you for your work on this project fail2ban team.

@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Apr 17, 2016

Member

There is a perspective PR #1374 so everyone interested is welcome to review/try

Member

yarikoptic commented Apr 17, 2016

There is a perspective PR #1374 so everyone interested is welcome to review/try

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres May 13, 2016

Member

We have introduced a new branch with IPv6 support: 0.10, and the new PR #1410
The decision is at least firstly - 0.9 will never get IPv6-support.

Everyone interested to test 0.10 is welcome to review resp. try this new vesion.

Member

sebres commented May 13, 2016

We have introduced a new branch with IPv6 support: 0.10, and the new PR #1410
The decision is at least firstly - 0.9 will never get IPv6-support.

Everyone interested to test 0.10 is welcome to review resp. try this new vesion.

@mjholtkamp

This comment has been minimized.

Show comment
Hide comment
@mjholtkamp

mjholtkamp Sep 8, 2016

Maybe I'm joining a bit late in the discussion, but I would also like to have IPv6 support in fail2ban and I have a potential contribution.

I tried to find a solution for the second item mentioned in the beginning of this issue: "Investigate more and decide on how to deal with variable across use-cases prefixes". I chose to implement "automatic shrinking of the prefix" and I think I have a nice working library:

https://github.com/mjholtkamp/py-iptree

Are people interested in using this library? I do not know if it's a good match, since iptree uses the ipaddress package (included in py3.5, separately installed for py2.7), instead of relying on the socket interface that fail2ban currently uses.

iptree is tested under both python 2.7 and 3.5.

A nice bonus is that the automatic shrinking also works for IPv4 addresses, if you choose to use that.

mjholtkamp commented Sep 8, 2016

Maybe I'm joining a bit late in the discussion, but I would also like to have IPv6 support in fail2ban and I have a potential contribution.

I tried to find a solution for the second item mentioned in the beginning of this issue: "Investigate more and decide on how to deal with variable across use-cases prefixes". I chose to implement "automatic shrinking of the prefix" and I think I have a nice working library:

https://github.com/mjholtkamp/py-iptree

Are people interested in using this library? I do not know if it's a good match, since iptree uses the ipaddress package (included in py3.5, separately installed for py2.7), instead of relying on the socket interface that fail2ban currently uses.

iptree is tested under both python 2.7 and 3.5.

A nice bonus is that the automatic shrinking also works for IPv4 addresses, if you choose to use that.

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Sep 8, 2016

Member

Thx, I will take a look into your library...

I would also like to have IPv6 support in fail2ban

Well, we have it in 0.10 (even pre-released and yet without subnets, but ... :))

Member

sebres commented Sep 8, 2016

Thx, I will take a look into your library...

I would also like to have IPv6 support in fail2ban

Well, we have it in 0.10 (even pre-released and yet without subnets, but ... :))

@mjholtkamp

This comment has been minimized.

Show comment
Hide comment
@mjholtkamp

mjholtkamp Sep 24, 2016

Of course I meant to have IPv6 support with subnet support ;)

Did you get a chance to take a look into the library yet? If you have suggestions on changes, I'm open to it and I wouldn't mind putting some effort into integrating the library in fail2ban (I think there might be some changes internally), but I want to know if it's desired. If you already now that it will never be merged I think it will be a waste of effort :)

mjholtkamp commented Sep 24, 2016

Of course I meant to have IPv6 support with subnet support ;)

Did you get a chance to take a look into the library yet? If you have suggestions on changes, I'm open to it and I wouldn't mind putting some effort into integrating the library in fail2ban (I think there might be some changes internally), but I want to know if it's desired. If you already now that it will never be merged I think it will be a waste of effort :)

@basd82

This comment has been minimized.

Show comment
Hide comment
@basd82

basd82 Mar 15, 2018

When will support for ipv6 be implemented ?

basd82 commented Mar 15, 2018

When will support for ipv6 be implemented ?

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Mar 15, 2018

Member

@basd82 It is already implemented in 0.10.x (ca. one and a half years).

Member

sebres commented Mar 15, 2018

@basd82 It is already implemented in 0.10.x (ca. one and a half years).

@basd82

This comment has been minimized.

Show comment
Hide comment
@basd82

basd82 Mar 15, 2018

@sebres is there debian/ubuntu repo for that version ?

basd82 commented Mar 15, 2018

@sebres is there debian/ubuntu repo for that version ?

@costela

This comment has been minimized.

Show comment
Hide comment
@costela

costela Mar 15, 2018

@basd82 both debian testing and ubuntu bionic have it:

$ rmadison -u debian fail2ban
(...)
fail2ban   | 0.10.2-1       | testing            | source, all
fail2ban   | 0.10.2-1       | unstable           | source, all

and

$ rmadison -u ubuntu fail2ban
(...)
 fail2ban | 0.10.2-1                     | bionic/universe           | source, all

Also, a github issue isn't the best place to ask these questions.

costela commented Mar 15, 2018

@basd82 both debian testing and ubuntu bionic have it:

$ rmadison -u debian fail2ban
(...)
fail2ban   | 0.10.2-1       | testing            | source, all
fail2ban   | 0.10.2-1       | unstable           | source, all

and

$ rmadison -u ubuntu fail2ban
(...)
 fail2ban | 0.10.2-1                     | bionic/universe           | source, all

Also, a github issue isn't the best place to ask these questions.

@szepeviktor

This comment has been minimized.

Show comment
Hide comment
@szepeviktor

szepeviktor Mar 15, 2018

Member

What parts of IPv6 support are missing? As this issues is open.

Member

szepeviktor commented Mar 15, 2018

What parts of IPv6 support are missing? As this issues is open.

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Mar 15, 2018

Member

As this issues is open.

  • subnet
  • all banning actions must provide IPv6 capability
Member

sebres commented Mar 15, 2018

As this issues is open.

  • subnet
  • all banning actions must provide IPv6 capability
@yarikoptic

This comment has been minimized.

Show comment
Hide comment
@yarikoptic

yarikoptic Mar 15, 2018

Member

@basd82 backports of 0.10.2-1 are available from "secret" debian-devel of neurodebian: http://neuro.debian.net/debian-devel/pool/main/f/fail2ban/ I was not yet brave enough to push them into the main repo of neurodebian... but soon I guess ;)

Member

yarikoptic commented Mar 15, 2018

@basd82 backports of 0.10.2-1 are available from "secret" debian-devel of neurodebian: http://neuro.debian.net/debian-devel/pool/main/f/fail2ban/ I was not yet brave enough to push them into the main repo of neurodebian... but soon I guess ;)

@f0o

This comment has been minimized.

Show comment
Hide comment
@f0o

f0o Mar 16, 2018

@sebres How can I, as a banning action provider, detect if I should ban a IPv4 or IPv6 without having to run the provided IP through regex?
Can I provide two separate ban/unban actions for v4 vs v6 targets?

f0o commented Mar 16, 2018

@sebres How can I, as a banning action provider, detect if I should ban a IPv4 or IPv6 without having to run the provided IP through regex?
Can I provide two separate ban/unban actions for v4 vs v6 targets?

@sebres

This comment has been minimized.

Show comment
Hide comment
@sebres

sebres Mar 16, 2018

Member

@f0o this is very easy: we've a conditional section that allows to differentiate between IP families.
For example take a look into any action supporting IPv6, e. g.

Short self explaining example:

[Definition]
...
actionban = printf "%%b\n" "ban <_ip_type> <_ip_var>:<port>"
actionunban = printf "%%b\n" "unban <_ip_type> <_ip_var>:<port>"

[Init]
_ip_var = <ip>
_ip_type = IPv4

[Init?family=inet6]
_ip_var = [<ip>]
_ip_type = IPv6

Of course it is not necessary to use some parameters extra (like in the example above), you can simply overwrite actionban, actionunban, etc. in section [Init?family=inet6].
So yes, you can provide two separate ban/unban actions for v4 vs v6 targets, but IMHO it is better to use parameters, at least saves time (and code) and more dynamical, e. g. if action will be extended then you must not adjust entries in both sections (if the same syntax).

Additionally, you've a new run-time tag <family> which contains the family of the IP (ban/unban only). So you can use it like this (or similar):

actionban = if [ '<family>' = 'inet4' ]; then ...; fi;
Member

sebres commented Mar 16, 2018

@f0o this is very easy: we've a conditional section that allows to differentiate between IP families.
For example take a look into any action supporting IPv6, e. g.

Short self explaining example:

[Definition]
...
actionban = printf "%%b\n" "ban <_ip_type> <_ip_var>:<port>"
actionunban = printf "%%b\n" "unban <_ip_type> <_ip_var>:<port>"

[Init]
_ip_var = <ip>
_ip_type = IPv4

[Init?family=inet6]
_ip_var = [<ip>]
_ip_type = IPv6

Of course it is not necessary to use some parameters extra (like in the example above), you can simply overwrite actionban, actionunban, etc. in section [Init?family=inet6].
So yes, you can provide two separate ban/unban actions for v4 vs v6 targets, but IMHO it is better to use parameters, at least saves time (and code) and more dynamical, e. g. if action will be extended then you must not adjust entries in both sections (if the same syntax).

Additionally, you've a new run-time tag <family> which contains the family of the IP (ban/unban only). So you can use it like this (or similar):

actionban = if [ '<family>' = 'inet4' ]; then ...; fi;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment