Skip to content
Night Watch crawler
JavaScript Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src
.eslintrc.json
.gitignore
.prettierrc.json
Dockerfile
LICENSE
README.md
package.json

README.md

Night Watch crawler

INSTALLATION

Requirements

  • Node.js >= 8
  • Dotenv files: .env.production and/or .env.development

Instructions

$ yarn install
$ yarn start # yarn dev for development

DOCUMENTATION

Environment Variables

  • NODE_ENV (string): "development" or "production" environment
  • PORT (number): Port number to run the server
  • NOTIFICATION_SERVICE_ADDRESS (string): Address of notification service
  • MONGODB_URI (string): MongoDB URI
  • MONGODB_DB_NAME (string): Database name

Routes

GET /

Just for testing

Response body
  • iam: "/"

POST /

Execute a watch session

Request body
  • url (string): URL to crawl
  • cssSelectors (object): Mapping from CSS selectors to their types. Currently, only "string" is a valid type.
You can’t perform that action at this time.