Skip to content

🕷️ This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md



crawlerdetect.io

npm version contributions welcome

Version 3

The new version now do include a middleware for expressjs, please read the documentation for the latest updates.

About ES6-CrawlerDetect

This Library is an ES6 version of the original PHP class @CrawlerDetect, it helps you detect bots/crawlers and spiders only by scanning the user-agent string or from the global request.headers.

Installation

npm install es6-crawler-detect

Usage

ECMAScript 6 (ES6)

'use strict';

const express = require('express')
const {Crawler, middleware} = require('es6-crawler-detect')

const app = express()

app.get('your/route', function async (request, response) {
  // create a new Crawler instance
  var CrawlerDetector = new Crawler(request)
  
  // check the current visitor's useragent
  if ( CrawlerDetector.isCrawler() )
  {
    // true if crawler user agent detected
  }
  
  // or check a user agent string
  if ( CrawlerDetector.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)') )
  {
    // true if crawler user agent detected
  }
  
  // Output the name of the bot that matched (if any)
  response.send(CrawlerDetector.getMatches())
})

/**
 * Or by using the middleware
*/
app.use(middleware)

app.get('/crawler', function async (request, response) {
  // or check a user agent string
  request.Crawler.isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')
  
  // Output the name of the bot that matched (if any)
  response.send(request.Crawler.getMatches())
})

ECMAScript 5 (ES5)

<head>
	<!-- Metas -->
	<meta name="title" content="ES6-CrawlerDetect">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
	<meta name="description" content="">
	<meta name="keywords" content="">
	<title>CrawlerDetect - the web crawler detection library</title>
	<!-- Scripts -->
	<script type="text/javascript" src="./your/path/to/main.bundle.js"></script>
</head>
  // create a new Crawler instance
  var CrawlerDetector = new Crawler.Crawler();
  var userAgentString = navigator.userAgent;
  
  // check the current visitor's useragent
  if ( CrawlerDetector.isCrawler(userAgentString) )
  {
    // true if crawler user agent detected
  }
  
  // Output the name of the bot that matched (if any)
  console.debug(CrawlerDetector.getMatches());

Contributing

If you find a bot/spider/crawler user agent that CrawlerDetect fails to detect, please submit a pull request with the regex pattern added to the data array in ./crawler/crawlers.js.

About

🕷️ This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

Topics

Resources

License

Sponsor this project

Packages

No packages published
You can’t perform that action at this time.