Archive Team

grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

ArchiveBot

ArchiveBot, an IRC bot for archiving websites

NewsGrabber-Warrior

NewsGrabber

Grabbing all news.

NewsGrabber-Services

The services for NewsGrabber.

archiveteam-megawarc-factory

Some scripts to process ArchiveTeam uploads

NewsGrabber-Discovery

The scripts for a discovery server for NewsGrabber.

wpull

Wget-compatible web downloader and crawler.

sketch-grab

terroroftinytown

URLTeam's second generation of URL shortener archiving tools

terroroftinytown-client-grab

The Seesaw pipeline grab script for the URLTeam (terroroftinytown) project

flickr-grab

Grabbing Flickr images.

warrior-dockerfile

A Dockerfile for the ArchiveTeam Warrior

zippcast-items

Managing items for zippcast-grab.

ludios_wpull

wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved

facepunch-grab

Archiving Facepunch Forums.

reddit-grab

Grabbing everything from reddit.

googleplus-grab

Archiving Google+.

Ubuntu-Warrior

Scripts to build and boot warrior virtual machine containing Docker

googleplus-items

Managing items for googleplus-grab and googleplus2-grab.

googleplus2-grab

Archiving communities and collection from Google+.

universal-tracker

A configurable, reusable tracker with dashboard

seesaw-kit

Making a reusable toolkit for writing seesaw scripts

flickr-items

Managing items for flickr-grab.

standalone-readme-template

Readme instructions template for manually running pipeline grab scripts outside the warrior

googlecode-items

Managing items for googlecode-grab.

ancestry-grab

Grabbing Ancestry sites/pages: http://archiveteam.org/index.php?title=Ancestry.com

uolforums-grab

Archiving the UOL forums.

uolforums-items

Managing items for uolforums-grab.

warrior-code2

Boot scripts for the ArchiveTeam Warrior 2