Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Site Mapper

Look at Your Website the Way Google Does

This is a demo of several of Algorithmia's website inspection and text analysis tools, including:

  • Get Links - examines a webpage and returns links to other pages
  • Page Rank - a simple implementation of the PageRank algorithm
  • URL2Text - extracts the main content from a webpage
  • Summarizer - creates a summary by extracting key topic sentences
  • Auto-Tag - generates keywords via Latent Dirichlet Allocation
  • D3 - a JavaScript library for generating graphs and visualizations

With this demo, one can quickly crawl an entire website, rank its pages, summarize and extract tags and sentiment from ech page, and view the sitemap in a D3 force-directed graph.

See this demo in action

This demo can be viewed at

Run it yourself

This demo only contains frontend code and requires no specialized hosting (or even a server).

  1. download the repository
  2. edit /JavaScript/sitemap/public/js/main.js to place your own API Key (free signup w/ 5k credits monthly) in the line containing "Algorithmia.client()"
  3. if you have not already, run the setup steps outlined in /
  4. build the demo: grunt build:sitemap
  5. open /build/sitemap/index.html in a web browser

Find more information in the Algorithmia Developer Center or the API Docs.