Skip to content
This repository has been archived by the owner on Feb 2, 2023. It is now read-only.

Add Google IP ranges #14

Closed
wants to merge 3 commits into from
Closed

Add Google IP ranges #14

wants to merge 3 commits into from

Conversation

ktheory
Copy link

@ktheory ktheory commented Feb 27, 2013

This dataset of Google IP ranges comes from:

$ dig -t TXT _netblocks.google.com @ns1.google.com

This is useful for identifying requests from Google AppEngine, which I've noticed often uses the 74.125.. range.

This dataset of Google IP ranges comes from running

    $ dig -t TXT _netblocks.google.com @ns1.google.com
Per `dig -t TXT _netblocks.google.com @ns1.google.com`
@ktheory
Copy link
Author

ktheory commented Jul 27, 2015

@client9 👋 I just updated with some new Google IP ranges. Would love to have this as part of the official datacenters.csv.

It's helpful for verifying Googlebot requests (i.e. the User-agent matches, and it's from a Google IP address), since the official method requires DNS lookups.

(NB: App Engine is included in these datacenters too, so that's not a completely sound way to verify Googlebot; but it's a start.)

ktheory added a commit to kickstarter/ipcat-ruby that referenced this pull request Jul 27, 2015
From my fork, see client9/ipcat#14

In a console, I ran:

```
IPCat.load_csv!('https://raw.githubusercontent.com/ktheory/ipcat/d9a21a6
d91432d28f08d776a793a0492f32bbab4/datacenters.csv');

File.open("data/datacenters", 'w') {|f| f << Marshal.dump(IPCat.ranges)
}
```
@jimdigriz
Copy link

Apparently it is better to parse _cloud-netblocks.googleusercontent.com which will catch URLFetch too:

https://cloud.google.com/appengine/kb/#static-ip

@shawnps
Copy link
Contributor

shawnps commented Jan 6, 2017

catch URLFetch too:

@jimdigriz sorry, could you clarify for my own understanding, does this mean it would catch calls to URL Fetch from a GAE app for example? My familiarity with URL Fetch is that it's used in GAE apps for tracking outbound requests.

@shawnps
Copy link
Contributor

shawnps commented Jan 6, 2017

Seems like _netblocks.google.com contains ranges that do not appear in _cloud-netblocks.googleusercontent.com, so we may need to get both.

@shawnps
Copy link
Contributor

shawnps commented Apr 24, 2017

We have AppEngine ranges in here now so I'll close this out.

@shawnps shawnps closed this Apr 24, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants