New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore original remote_ip algorithm. #7980

Merged
merged 1 commit into from Jan 2, 2013

Conversation

Projects
None yet
10 participants
@steveklabnik
Member

steveklabnik commented Oct 18, 2012

We incorrectly were using the first address, rather than the last one.

Fixes #7979

/cc @indirect @gazay

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Oct 18, 2012

Member

I am 99% sure this is right, the tests were new, so I wasn't sure how to 'revert' them. Please check this request well before merging. ;)

Member

steveklabnik commented Oct 18, 2012

I am 99% sure this is right, the tests were new, so I wasn't sure how to 'revert' them. Please check this request well before merging. ;)

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

I put some notes into the extended commit messages, but the upshot is that it works correctly now, it ignores private IPv6 IPs now (which it didn't do before), and the code is much shorter. Yay. :)

Member

indirect commented Oct 18, 2012

I put some notes into the extended commit messages, but the upshot is that it works correctly now, it ignores private IPv6 IPs now (which it didn't do before), and the code is much shorter. Yay. :)

@rafaelfranca

View changes

Show outdated Hide outdated actionpack/lib/action_dispatch/middleware/remote_ip.rb
@rafaelfranca

View changes

Show outdated Hide outdated actionpack/test/dispatch/request_test.rb
@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Oct 18, 2012

Member

I did some comments inline. The feature seems fine but we need to remove the comment tests (if they are not needed) or uncomment they (if they are needed)

Member

rafaelfranca commented Oct 18, 2012

I did some comments inline. The feature seems fine but we need to remove the comment tests (if they are not needed) or uncomment they (if they are needed)

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

Oops! Fixing, thank you.

On Oct 17, 2012, at 6:30 PM, Rafael Mendonça França notifications@github.com wrote:

I did some comments inline. The feature seems fine but we need to remove the comment tests (if they are not needed) or uncomment they (if they are needed)


Reply to this email directly or view it on GitHub.

Member

indirect commented Oct 18, 2012

Oops! Fixing, thank you.

On Oct 17, 2012, at 6:30 PM, Rafael Mendonça França notifications@github.com wrote:

I did some comments inline. The feature seems fine but we need to remove the comment tests (if they are not needed) or uncomment they (if they are needed)


Reply to this email directly or view it on GitHub.

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 18, 2012

Contributor

I'm not sure that this is right to use last address from header
X-Forwarded-For, in every specification about this header written that
proxy should add initiator's IP addres to the end of list, so if we go
through proxy1, proxy2, proxy3 we get list like clientIP, proxy1IP,
proxy2IP, proxy3IP. You can see it in wikipedia and in referenced links
from there, also from w3c specification.
http://www.w3.org/TR/2009/WD-ct-guidelines-20091006/#sec-additional-headers

Here is the usage of mod_extract_forwarded for apache for example:
http://www.openinfo.co.uk/apache/index.html

Sorry, I have problems with accessing github.com right now, so I comment
from mail.

On Thu, Oct 18, 2012 at 9:41 AM, André Arko notifications@github.comwrote:

Oops! Fixing, thank you.

On Oct 17, 2012, at 6:30 PM, Rafael Mendonça França <
notifications@github.com> wrote:

I did some comments inline. The feature seems fine but we need to remove
the comment tests (if they are not needed) or uncomment they (if they are
needed)


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com/rails/rails/pull/7980#issuecomment-9550556.

Contributor

gazay commented Oct 18, 2012

I'm not sure that this is right to use last address from header
X-Forwarded-For, in every specification about this header written that
proxy should add initiator's IP addres to the end of list, so if we go
through proxy1, proxy2, proxy3 we get list like clientIP, proxy1IP,
proxy2IP, proxy3IP. You can see it in wikipedia and in referenced links
from there, also from w3c specification.
http://www.w3.org/TR/2009/WD-ct-guidelines-20091006/#sec-additional-headers

Here is the usage of mod_extract_forwarded for apache for example:
http://www.openinfo.co.uk/apache/index.html

Sorry, I have problems with accessing github.com right now, so I comment
from mail.

On Thu, Oct 18, 2012 at 9:41 AM, André Arko notifications@github.comwrote:

Oops! Fixing, thank you.

On Oct 17, 2012, at 6:30 PM, Rafael Mendonça França <
notifications@github.com> wrote:

I did some comments inline. The feature seems fine but we need to remove
the comment tests (if they are not needed) or uncomment they (if they are
needed)


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHubhttps://github.com/rails/rails/pull/7980#issuecomment-9550556.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

Ohhh, I understand what the problem is now. Many proxies don't actually modify the X-Forwarded-For header. Instead, they add another X-Forwarded-For header. According to the HTTP spec, that means that the values should be concatenated, joined by a comma. If you're using a ruby server that understands repeated headers, the correct IP will then be the last value. If the proxy conforms to the W3C spec that you linked, the correct IP will be the first value.

Um. How can we resolve this? FWIW, I've only ever seen the "adds a second header" proxy type in the wild (I know HAProxy does this).

Member

indirect commented Oct 18, 2012

Ohhh, I understand what the problem is now. Many proxies don't actually modify the X-Forwarded-For header. Instead, they add another X-Forwarded-For header. According to the HTTP spec, that means that the values should be concatenated, joined by a comma. If you're using a ruby server that understands repeated headers, the correct IP will then be the last value. If the proxy conforms to the W3C spec that you linked, the correct IP will be the first value.

Um. How can we resolve this? FWIW, I've only ever seen the "adds a second header" proxy type in the wild (I know HAProxy does this).

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

In addition, other blog posts I have read on the subject seem to indicate that the last IP is the correct one in real-world scenarios.

Member

indirect commented Oct 18, 2012

In addition, other blog posts I have read on the subject seem to indicate that the last IP is the correct one in real-world scenarios.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

And indeed, the Apache bug on this subject also seems to imply we should be using the last IP address in the concatenated list.

Member

indirect commented Oct 18, 2012

And indeed, the Apache bug on this subject also seems to imply we should be using the last IP address in the concatenated list.

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 18, 2012

Contributor

I did my pull last time because I used this functionality on groupon
russia, and there getting first IP address from header was right solution.
Maybe we can do preconfigured usage of remote_ip with your algorithm as
default?

On Thu, Oct 18, 2012 at 10:45 AM, André Arko notifications@github.comwrote:

And indeed, the Apache bug on this subjecthttps://issues.apache.org/bugzilla/show_bug.cgi?id=50453also seems to imply we should be using the last IP address in the
concatenated list.


Reply to this email directly or view it on GitHubhttps://github.com/rails/rails/pull/7980#issuecomment-9551581.

Contributor

gazay commented Oct 18, 2012

I did my pull last time because I used this functionality on groupon
russia, and there getting first IP address from header was right solution.
Maybe we can do preconfigured usage of remote_ip with your algorithm as
default?

On Thu, Oct 18, 2012 at 10:45 AM, André Arko notifications@github.comwrote:

And indeed, the Apache bug on this subjecthttps://issues.apache.org/bugzilla/show_bug.cgi?id=50453also seems to imply we should be using the last IP address in the
concatenated list.


Reply to this email directly or view it on GitHubhttps://github.com/rails/rails/pull/7980#issuecomment-9551581.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

It seems like an option on the middleware is the way to go if this is a common need, and a replacement middleware that removes the calls to reverse is the way to go if its not common. WDYT, people who can merge this?

Member

indirect commented Oct 18, 2012

It seems like an option on the middleware is the way to go if this is a common need, and a replacement middleware that removes the calls to reverse is the way to go if its not common. WDYT, people who can merge this?

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik
Member

steveklabnik commented Oct 18, 2012

Ugh.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 18, 2012

Member

@steveklabnik inorite? 😭

Member

indirect commented Oct 18, 2012

@steveklabnik inorite? 😭

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Oct 20, 2012

Member

I don't like the idea to add Yet Another Configuration Option, but I don't know if this is not common

Member

rafaelfranca commented Oct 20, 2012

I don't like the idea to add Yet Another Configuration Option, but I don't know if this is not common

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Oct 20, 2012

Member

I always prefer what is correct to what is not. It's not our job to fix bugs in other projects, we should follow the spec.

Then again, I don't have anything currently deployed in production that depends on this in either direction ;)

Member

steveklabnik commented Oct 20, 2012

I always prefer what is correct to what is not. It's not our job to fix bugs in other projects, we should follow the spec.

Then again, I don't have anything currently deployed in production that depends on this in either direction ;)

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 20, 2012

Member

If it helps at all, this pull does fix at least one bug in the trusted proxies regex, and also restores the same behaviour as e.g. Apache. I personally think that's the way it should work, but hey. I'm biased. :)

On Oct 20, 2012, at 12:00 PM, Steve Klabnik notifications@github.com wrote:

I always prefer what is correct to what is not. It's not our job to fix bugs in other projects, we should follow the spec.

Then again, I don't have anything currently deployed in production that depends on this in either direction ;)


Reply to this email directly or view it on GitHub.

Member

indirect commented Oct 20, 2012

If it helps at all, this pull does fix at least one bug in the trusted proxies regex, and also restores the same behaviour as e.g. Apache. I personally think that's the way it should work, but hey. I'm biased. :)

On Oct 20, 2012, at 12:00 PM, Steve Klabnik notifications@github.com wrote:

I always prefer what is correct to what is not. It's not our job to fix bugs in other projects, we should follow the spec.

Then again, I don't have anything currently deployed in production that depends on this in either direction ;)


Reply to this email directly or view it on GitHub.

@ehoch

This comment has been minimized.

Show comment
Hide comment
@ehoch

ehoch Oct 22, 2012

Figured I'd weigh in here. I use Google Page Speed Service in front of Heroku. Not the craziest, most unheard of Rails environment. There, the client's IP is the FIRST in the X-Forwarded-For csv.

Can we make this configurable?

ehoch commented Oct 22, 2012

Figured I'd weigh in here. I use Google Page Speed Service in front of Heroku. Not the craziest, most unheard of Rails environment. There, the client's IP is the FIRST in the X-Forwarded-For csv.

Can we make this configurable?

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 22, 2012

Member

I guess the upside is that it's always going to be first or last? Unless you combine Google Page Speed and HAProxy, then it'll be in the middle. -_-

I'm ready to implement an option that lets you tell the middleware whether you want the first or last IP. I'm not sure which option should be the default, though, since the "spec correct" one is first and the "works like Apache etc" one is last. :|

Member

indirect commented Oct 22, 2012

I guess the upside is that it's always going to be first or last? Unless you combine Google Page Speed and HAProxy, then it'll be in the middle. -_-

I'm ready to implement an option that lets you tell the middleware whether you want the first or last IP. I'm not sure which option should be the default, though, since the "spec correct" one is first and the "works like Apache etc" one is last. :|

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 22, 2012

Contributor

I think that default should be last IP (your implementation), because long time nobody noticed that, and my pull was opened about 8 months before merged in. So I think first IP is like exception.

I can help you with implementation

Contributor

gazay commented Oct 22, 2012

I think that default should be last IP (your implementation), because long time nobody noticed that, and my pull was opened about 8 months before merged in. So I think first IP is like exception.

I can help you with implementation

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 22, 2012

Member

Makes sense to me. Thanks for offering to help! I imagine @steveklabnik can get you hooked up with push to his repo so you can add to this pull request if you have some patches.

On Oct 22, 2012, at 8:39 AM, Alexey Gaziev notifications@github.com wrote:

I think that default should be last IP (your implementation), because long time nobody noticed that, and my pull was opened about 8 months before merged in. So I think first IP is like exception.

I can help you with implementation


Reply to this email directly or view it on GitHub.

Member

indirect commented Oct 22, 2012

Makes sense to me. Thanks for offering to help! I imagine @steveklabnik can get you hooked up with push to his repo so you can add to this pull request if you have some patches.

On Oct 22, 2012, at 8:39 AM, Alexey Gaziev notifications@github.com wrote:

I think that default should be last IP (your implementation), because long time nobody noticed that, and my pull was opened about 8 months before merged in. So I think first IP is like exception.

I can help you with implementation


Reply to this email directly or view it on GitHub.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Oct 22, 2012

Member

Yes, @gazay let me know if you have some commits and I can give you access to my repo.

Member

steveklabnik commented Oct 22, 2012

Yes, @gazay let me know if you have some commits and I can give you access to my repo.

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 22, 2012

Contributor

Great, thanks guys!

Contributor

gazay commented Oct 22, 2012

Great, thanks guys!

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 22, 2012

Contributor

So, I'm ready to push some commits. I did the same option usage as with ip_spoofing_check. And I wrote some tests to check different order usage

And btw I think that we shouldn't filter values from X_FORWARDED_FOR and give to user unexpected IP of unknown proxy server. We should give him only first or last ip from list as we decided here. I changed code in remote_ip.rb and fix tests for it. What do you think @indirect, @steveklabnik?

Contributor

gazay commented Oct 22, 2012

So, I'm ready to push some commits. I did the same option usage as with ip_spoofing_check. And I wrote some tests to check different order usage

And btw I think that we shouldn't filter values from X_FORWARDED_FOR and give to user unexpected IP of unknown proxy server. We should give him only first or last ip from list as we decided here. I changed code in remote_ip.rb and fix tests for it. What do you think @indirect, @steveklabnik?

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Oct 22, 2012

Member

You're added to my repo.

Member

steveklabnik commented Oct 22, 2012

You're added to my repo.

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 22, 2012

Contributor

I've added commits. I'm not sure about name of option - x_forwarded_for_last_ip, but that's all I could think of

Contributor

gazay commented Oct 22, 2012

I've added commits. I'm not sure about name of option - x_forwarded_for_last_ip, but that's all I could think of

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 22, 2012

Member

I think the ability to pass in proxies to be filtered is very important, because its the only way to get the real IP in a situation where one proxy adds to the front but another proxy adds to the end. So I think we should keep it. Since the option is already scoped to the remote IP middleware, and applies to more than one header, I would suggest naming it "first_ip" or "first_client_ip".

On Oct 22, 2012, at 11:09 AM, Alexey Gaziev notifications@github.com wrote:

So, I'm ready to push some commits. I did the same option usage as with ip_spoofing_check. And I wrote some tests to check different order usage

And btw I think that we shouldn't filter values from X_FORWARDED_FOR and give to user unexpected IP of unknown proxy server. We should give him only first or last ip from list as we decided here. I changed code in remote_ip.rb and fix tests for it. What do you think @indirect, @steveklabnik?


Reply to this email directly or view it on GitHub.

Member

indirect commented Oct 22, 2012

I think the ability to pass in proxies to be filtered is very important, because its the only way to get the real IP in a situation where one proxy adds to the front but another proxy adds to the end. So I think we should keep it. Since the option is already scoped to the remote IP middleware, and applies to more than one header, I would suggest naming it "first_ip" or "first_client_ip".

On Oct 22, 2012, at 11:09 AM, Alexey Gaziev notifications@github.com wrote:

So, I'm ready to push some commits. I did the same option usage as with ip_spoofing_check. And I wrote some tests to check different order usage

And btw I think that we shouldn't filter values from X_FORWARDED_FOR and give to user unexpected IP of unknown proxy server. We should give him only first or last ip from list as we decided here. I changed code in remote_ip.rb and fix tests for it. What do you think @indirect, @steveklabnik?


Reply to this email directly or view it on GitHub.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 22, 2012

Member

And of course when I just suggested "first_ip", I actually meant "last_ip". :P

On Oct 22, 2012, at 11:33 AM, Alexey Gaziev notifications@github.com wrote:

I've added commits. I'm not sure about name of option - x_forwarded_for_last_ip, but that's all I could think of


Reply to this email directly or view it on GitHub.

Member

indirect commented Oct 22, 2012

And of course when I just suggested "first_ip", I actually meant "last_ip". :P

On Oct 22, 2012, at 11:33 AM, Alexey Gaziev notifications@github.com wrote:

I've added commits. I'm not sure about name of option - x_forwarded_for_last_ip, but that's all I could think of


Reply to this email directly or view it on GitHub.

@gazay

This comment has been minimized.

Show comment
Hide comment
@gazay

gazay Oct 23, 2012

Contributor

About filtering – I see the problem, you right, I'll change it back. About naming - last_ip or last_client_ip not really clear when someone will read this code or try to change option. It's like if ip_spoofing_check will be just ip_check - not sure what this option allow to check. Maybe last_forwardable_ip?

Contributor

gazay commented Oct 23, 2012

About filtering – I see the problem, you right, I'll change it back. About naming - last_ip or last_client_ip not really clear when someone will read this code or try to change option. It's like if ip_spoofing_check will be just ip_check - not sure what this option allow to check. Maybe last_forwardable_ip?

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Oct 23, 2012

Member

Ahh, I see. I was thinking the option should apply to REMOTE_ADDR too, but it seems that that header will only ever have a list of IPs due to duplicated headers, and so reverse is always the correct direction.

How about calling the option "last_forwarded_ip"? That includes both the important part of the header name, and literally describes what you will get if you turn the option on -- the last IP from the proxy's list of forwarded IPs.

Member

indirect commented Oct 23, 2012

Ahh, I see. I was thinking the option should apply to REMOTE_ADDR too, but it seems that that header will only ever have a list of IPs due to duplicated headers, and so reverse is always the correct direction.

How about calling the option "last_forwarded_ip"? That includes both the important part of the header name, and literally describes what you will get if you turn the option on -- the last IP from the proxy's list of forwarded IPs.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Nov 18, 2012

Member

@gingerlime ahh, you make a very good point. I didn't connect the dots during the earlier discussion, but the option shouldn't be needed at all if you can explicitly list the proxies whose addresses you don't want to count. @gazay, would that work in your situation?

Given the potential for IP spoofing vulnerabilities, I think we might want to ultimately revert to the original behaviour without an option, and suggest in the comments/docs that anyone who needs something custom should remove this middleware and insert their own instead.

Member

indirect commented Nov 18, 2012

@gingerlime ahh, you make a very good point. I didn't connect the dots during the earlier discussion, but the option shouldn't be needed at all if you can explicitly list the proxies whose addresses you don't want to count. @gazay, would that work in your situation?

Given the potential for IP spoofing vulnerabilities, I think we might want to ultimately revert to the original behaviour without an option, and suggest in the comments/docs that anyone who needs something custom should remove this middleware and insert their own instead.

@NZKoz

This comment has been minimized.

Show comment
Hide comment
@NZKoz

NZKoz Nov 18, 2012

Member

Yes, while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks. I'd suggest just adding something like:

  request.forwarded_ips # => ['x.x.x.x', 'y.y.y.y', 'z.z.z.z']

Then the config just becomes a question of first or last from that method.

FWIW, if you're using remote_ip in a security setting you're completely crazy ;)

Member

NZKoz commented Nov 18, 2012

Yes, while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks. I'd suggest just adding something like:

  request.forwarded_ips # => ['x.x.x.x', 'y.y.y.y', 'z.z.z.z']

Then the config just becomes a question of first or last from that method.

FWIW, if you're using remote_ip in a security setting you're completely crazy ;)

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Nov 18, 2012

Member

The config in question is pretty much exactly that (mod some filtering of obviously not useful IPs). So that seems okay, I guess.

On Nov 18, 2012, at 12:35 PM, Michael Koziarski notifications@github.com wrote:

Yes, while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks. I'd suggest just adding something like:

request.forwarded_ips # => ['x.x.x.x', 'y.y.y.y', 'z.z.z.z']
Then the config just becomes a question of first or last from that method.

FWIW, if you're using remote_ip in a security setting you're completely crazy ;)


Reply to this email directly or view it on GitHub.

Member

indirect commented Nov 18, 2012

The config in question is pretty much exactly that (mod some filtering of obviously not useful IPs). So that seems okay, I guess.

On Nov 18, 2012, at 12:35 PM, Michael Koziarski notifications@github.com wrote:

Yes, while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks. I'd suggest just adding something like:

request.forwarded_ips # => ['x.x.x.x', 'y.y.y.y', 'z.z.z.z']
Then the config just becomes a question of first or last from that method.

FWIW, if you're using remote_ip in a security setting you're completely crazy ;)


Reply to this email directly or view it on GitHub.

@gingerlime

This comment has been minimized.

Show comment
Hide comment
@gingerlime

gingerlime Nov 22, 2012

Contributor

@NZKoz I just re-read your comment:

while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks

This is not entirely accurate. If you trust your proxy server(s), and by trust I also mean you trust them to handle X-Forwarded-For appropriately (and barring any vulnerabilities those might have). Then you should be able to trust a portion of the header's content, which should produce the IP address of the device connected to your outer-most trusted proxy.

If you don't use any (trusted) proxy servers, then you should simply ignore the X-Forwarded-For header completely.

Contributor

gingerlime commented Nov 22, 2012

@NZKoz I just re-read your comment:

while we read from X-Forwarded-For in the implementation of remote_ip then you'll always be potentially vulnerable to spoofing attacks

This is not entirely accurate. If you trust your proxy server(s), and by trust I also mean you trust them to handle X-Forwarded-For appropriately (and barring any vulnerabilities those might have). Then you should be able to trust a portion of the header's content, which should produce the IP address of the device connected to your outer-most trusted proxy.

If you don't use any (trusted) proxy servers, then you should simply ignore the X-Forwarded-For header completely.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Nov 28, 2012

Member

So: what exactly needs to be done to move this forward?

Member

steveklabnik commented Nov 28, 2012

So: what exactly needs to be done to move this forward?

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Nov 28, 2012

Member

...better...docs?

Member

indirect commented Nov 28, 2012

...better...docs?

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Nov 28, 2012

Member

I've thought about this some more, and I'd like to propose a different final solution: how about we remove the first/last IP option, revert to the (safe) last IP, and beef up the docs explaining how to use the trusted proxy option to remove IPs from the end until what's left is the IP that you're interested in?

On Nov 28, 2012, at 2:01 PM, Steve Klabnik notifications@github.com wrote:

So: what exactly needs to be done to move this forward?


Reply to this email directly or view it on GitHub.

Member

indirect commented Nov 28, 2012

I've thought about this some more, and I'd like to propose a different final solution: how about we remove the first/last IP option, revert to the (safe) last IP, and beef up the docs explaining how to use the trusted proxy option to remove IPs from the end until what's left is the IP that you're interested in?

On Nov 28, 2012, at 2:01 PM, Steve Klabnik notifications@github.com wrote:

So: what exactly needs to be done to move this forward?


Reply to this email directly or view it on GitHub.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Nov 28, 2012

Member

I can see that too. @NZKoz what do you think?

Member

steveklabnik commented Nov 28, 2012

I can see that too. @NZKoz what do you think?

@NZKoz

This comment has been minimized.

Show comment
Hide comment
@NZKoz

NZKoz Nov 28, 2012

Member

sounds good to me

Member

NZKoz commented Nov 28, 2012

sounds good to me

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Nov 28, 2012

Member

Great. I'll try to get that implemented today.

Member

indirect commented Nov 28, 2012

Great. I'll try to get that implemented today.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 18, 2012

Member

Ping! @indirect did you get a chance to play with this?

Member

steveklabnik commented Dec 18, 2012

Ping! @indirect did you get a chance to play with this?

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Dec 18, 2012

Member

Oh man. Totally fell off my list. Today or tomorrow, I hope.

Member

indirect commented Dec 18, 2012

Oh man. Totally fell off my list. Today or tomorrow, I hope.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Dec 18, 2012

Member

No worries bro, know you've been busy.

Member

steveklabnik commented Dec 18, 2012

No worries bro, know you've been busy.

@jfirebaugh

This comment has been minimized.

Show comment
Hide comment
@jfirebaugh

jfirebaugh Dec 24, 2012

Contributor

I agree with reverting to last IP and documenting the trusted proxy option.

What I would like to see is an option to ignore HTTP_CLIENT_IP. In an EC2+ELB environment that header is effectively garbage and should be ignored (ELB only uses X-Forwarded-For, not Client-IP). The fact that it is used to try to detect spoofing results in legitimate users getting locked out. If not an option, then at least extract some finer-grained methods so I can monkey patch more easily.

def client_ip
  @env['HTTP_CLIENT_IP']
end

def forwarded_ips
  ips_from('HTTP_X_FORWARDED_FOR')
end

def remote_addrs
  ips_from('REMOTE_ADDR').reverse
end

def candidate_ips
  [client_ip, forwarded_ips, remote_addrs].flatten.compact
end

def remote_ip
  remove_proxies(candidate_ips).first || remote_addrs.first
end
Contributor

jfirebaugh commented Dec 24, 2012

I agree with reverting to last IP and documenting the trusted proxy option.

What I would like to see is an option to ignore HTTP_CLIENT_IP. In an EC2+ELB environment that header is effectively garbage and should be ignored (ELB only uses X-Forwarded-For, not Client-IP). The fact that it is used to try to detect spoofing results in legitimate users getting locked out. If not an option, then at least extract some finer-grained methods so I can monkey patch more easily.

def client_ip
  @env['HTTP_CLIENT_IP']
end

def forwarded_ips
  ips_from('HTTP_X_FORWARDED_FOR')
end

def remote_addrs
  ips_from('REMOTE_ADDR').reverse
end

def candidate_ips
  [client_ip, forwarded_ips, remote_addrs].flatten.compact
end

def remote_ip
  remove_proxies(candidate_ips).first || remote_addrs.first
end
@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Dec 24, 2012

Member

That seems reasonable to me.

On Dec 24, 2012, at 11:46 AM, John Firebaugh notifications@github.com wrote:

I agree with reverting to last IP and documenting the trusted proxy option.

What I would like to see is an option to ignore HTTP_CLIENT_IP. In an EC2+ELB environment that header is effectively garbage and should be ignored (ELB only uses X-Forwarded-For, not Client-IP). The fact that it is used to try to detect spoofing results in legitimate users getting locked out. If not an option, then at least extract some finer-grained methods so I can monkey patch more easily.

def client_ip
@env['HTTP_CLIENT_IP']
end

def forwarded_ips
ips_from('HTTP_X_FORWARDED_FOR')
end

def remote_addrs
ips_from('REMOTE_ADDR').reverse
end

def candidate_ips
[client_ip, forwarded_ips, remote_addrs].flatten.compact
end

def remote_ip
remove_proxies(candidate_ips).first || remote_addrs.first
end

Reply to this email directly or view it on GitHub.

Member

indirect commented Dec 24, 2012

That seems reasonable to me.

On Dec 24, 2012, at 11:46 AM, John Firebaugh notifications@github.com wrote:

I agree with reverting to last IP and documenting the trusted proxy option.

What I would like to see is an option to ignore HTTP_CLIENT_IP. In an EC2+ELB environment that header is effectively garbage and should be ignored (ELB only uses X-Forwarded-For, not Client-IP). The fact that it is used to try to detect spoofing results in legitimate users getting locked out. If not an option, then at least extract some finer-grained methods so I can monkey patch more easily.

def client_ip
@env['HTTP_CLIENT_IP']
end

def forwarded_ips
ips_from('HTTP_X_FORWARDED_FOR')
end

def remote_addrs
ips_from('REMOTE_ADDR').reverse
end

def candidate_ips
[client_ip, forwarded_ips, remote_addrs].flatten.compact
end

def remote_ip
remove_proxies(candidate_ips).first || remote_addrs.first
end

Reply to this email directly or view it on GitHub.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Jan 2, 2013

Member

@jfirebaugh after editing this, I'm not sure why you want to be able to monkeypatch this at all. You can completely resolve your stated use-case by adding a middleware farther out that strips the "garbage" header, and then disable the spoof checking using the option that is provided.

I personally would like to remove the spoof checking, because AFAIK there aren't any proxies that set both Client-Ip and X-Forwarded-For, but I'm sure that's a totally separate PR and discussion. :)

Member

indirect commented Jan 2, 2013

@jfirebaugh after editing this, I'm not sure why you want to be able to monkeypatch this at all. You can completely resolve your stated use-case by adding a middleware farther out that strips the "garbage" header, and then disable the spoof checking using the option that is provided.

I personally would like to remove the spoof checking, because AFAIK there aren't any proxies that set both Client-Ip and X-Forwarded-For, but I'm sure that's a totally separate PR and discussion. :)

@jfirebaugh

This comment has been minimized.

Show comment
Hide comment
@jfirebaugh

jfirebaugh Jan 2, 2013

Contributor

Figuring out the correct IP is the responsibility of the RemoteIp middleware, right? Folks shouldn't have to write a second middleware that deals in the same protocol details as RemoteIp in order to get RemoteIp to work correctly in what is a very common environment.

I'm all for removing the spoof checking as well, but I think there needs to be a configuration option to say which of Client-Ip and X-Forwarded-For is accurate.

Contributor

jfirebaugh commented Jan 2, 2013

Figuring out the correct IP is the responsibility of the RemoteIp middleware, right? Folks shouldn't have to write a second middleware that deals in the same protocol details as RemoteIp in order to get RemoteIp to work correctly in what is a very common environment.

I'm all for removing the spoof checking as well, but I think there needs to be a configuration option to say which of Client-Ip and X-Forwarded-For is accurate.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Jan 2, 2013

Member

The job of this middleware is to return the IP address reported in the headers of the request. IMO, if you want to ignore one of those headers, then yes, you should have to write code that deletes that header. Have you asked the Amazon guys about fixing their headers so that code isn't needed?

On Jan 1, 2013, at 5:16 PM, John Firebaugh notifications@github.com wrote:

Figuring out the correct IP is the responsibility of the RemoteIp middleware, right? Folks shouldn't have to write a second middleware that deals in the same protocol details as RemoteIp in order to get RemoteIp to work correctly in what is a very common environment.

I'm all for removing the spoof checking as well, but I think there needs to be a configuration option to say which of Client-Ip and X-Forwarded-For is accurate.


Reply to this email directly or view it on GitHub.

Member

indirect commented Jan 2, 2013

The job of this middleware is to return the IP address reported in the headers of the request. IMO, if you want to ignore one of those headers, then yes, you should have to write code that deletes that header. Have you asked the Amazon guys about fixing their headers so that code isn't needed?

On Jan 1, 2013, at 5:16 PM, John Firebaugh notifications@github.com wrote:

Figuring out the correct IP is the responsibility of the RemoteIp middleware, right? Folks shouldn't have to write a second middleware that deals in the same protocol details as RemoteIp in order to get RemoteIp to work correctly in what is a very common environment.

I'm all for removing the spoof checking as well, but I think there needs to be a configuration option to say which of Client-Ip and X-Forwarded-For is accurate.


Reply to this email directly or view it on GitHub.

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Jan 2, 2013

Member

Okay. I have coded and documented the fuck out of this thing. :shipit: /cc @steveklabnik

Member

indirect commented Jan 2, 2013

Okay. I have coded and documented the fuck out of this thing. :shipit: /cc @steveklabnik

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
@rafaelfranca

rafaelfranca Jan 2, 2013

Member

needs rebase

Member

rafaelfranca commented Jan 2, 2013

needs rebase

@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Jan 2, 2013

Member

@rafaelfranca bro, I just rebased this 4 minutes ago. It needs another one?!

Member

indirect commented Jan 2, 2013

@rafaelfranca bro, I just rebased this 4 minutes ago. It needs another one?!

Restore original remote_ip algorithm.
Proxy servers add X-Forwarded-For headers, resulting in a list of IPs. We
remove trusted IP values, and then take the last given value, assuming that
it is the most likely to be the correct, unfaked value. See [1] for a very
thorough discussion of why that is the best option we have at the moment.

[1]: http://blog.gingerlime.com/2012/rails-ip-spoofing-vulnerabilities-and-protection/

Fixes #7979
@indirect

This comment has been minimized.

Show comment
Hide comment
@indirect

indirect Jan 2, 2013

Member

Huh, stupid CHANGELOG. :P Rebased! Again! QUICK SOMEBODY PUSH THE MERGE BUTTON

Member

indirect commented Jan 2, 2013

Huh, stupid CHANGELOG. :P Rebased! Again! QUICK SOMEBODY PUSH THE MERGE BUTTON

guilleiguaran added a commit that referenced this pull request Jan 2, 2013

Merge pull request #7980 from steveklabnik/issue_7979
Restore original remote_ip algorithm.

@guilleiguaran guilleiguaran merged commit c79fb2a into rails:master Jan 2, 2013

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik

steveklabnik Jan 3, 2013

Member

🤘

Thanks guys. ❤️

Member

steveklabnik commented Jan 3, 2013

🤘

Thanks guys. ❤️

@steveklabnik steveklabnik deleted the steveklabnik:issue_7979 branch Jan 3, 2013

@amoose

This comment has been minimized.

Show comment
Hide comment
@amoose

amoose commented Apr 17, 2013

o_O

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment