Skip to content

Commit

Permalink
Initial release
Browse files Browse the repository at this point in the history
  • Loading branch information
jcowley committed Sep 9, 2017
1 parent ee23999 commit d5d7bae
Show file tree
Hide file tree
Showing 6 changed files with 1,410 additions and 1 deletion.
15 changes: 15 additions & 0 deletions .travis.yml
@@ -0,0 +1,15 @@
language: node_js
node_js:
- "4"

node_js:
- stable

install:
- npm install

script:
- npm run cover

# Send coverage data to Coveralls
after_script: "cat ./coverage/lcov.info | ./node_modules/coveralls/bin/coveralls.js"
35 changes: 34 additions & 1 deletion README.md
@@ -1,2 +1,35 @@
# googlebot-verify
Verify that a request is from Google using Google's DNS verification steps
[![Build Status](https://travis-ci.org/jcowley/googlebot-verify.svg?branch=master)](https://travis-ci.org/jcowley/googlebot-verify)
[![Coverage Status](https://coveralls.io/repos/github/jcowley/googlebot-verify/badge.svg?branch=master)](https://coveralls.io/github/jcowley/googlebot-verify?branch=master)
## Synopsis

Verify that an IP address is from Google using Google's recommended DNS verification steps.

## Motivation

You may wish to verify that a web crawler accessing your server is Googlebot (or another Google user-agent) and not spammers or other bots scraping your site while claiming to be Googlebot. Since you cannot rely on the `User-Agent` header which is easily spoofed, you need to use DNS look up to verify that the IP address belongs to Google.

This library implements Google's own verification steps outlined here: https://support.google.com/webmasters/answer/80553?hl=en

## Code Example

Usage in an express application:

```javascript
const isGoogle = require('googlebot-verify');
const ipAddress = req.headers['x-forwarded-for'] || req.connection.remoteAddress;
if (isGoogle(ipAddress)) {
...
```
## Installation
`npm install googlebot-verify`
## Tests
`npm test`
## License
[MIT](https://github.com/pillarjs/parseurl/blob/master/LICENSE)
29 changes: 29 additions & 0 deletions index.js
@@ -0,0 +1,29 @@
'use strict';
const dns = require('dns');
const url = require('url');

module.exports = (ip, callback) => {
callback = callback || function () {}

return new Promise(function (resolve, reject) {

dns.reverse(ip, (error, hosts) => {
if (error) {
reject(error);
return callback(error);
}

const tld = hosts[0] && hosts[0].split('.').slice(-2, -1)[0];
if (tld !== "google" && tld !== "googlebot") {
resolve(false);
return callback(null, false);
}

dns.lookup(hosts[0], (error, address) => {
const match = address === ip;
resolve(match);
return callback(error, match);
});
});
});
}

0 comments on commit d5d7bae

Please sign in to comment.