Skip to content

Web Archiving Community

Nick Sweeting edited this page Oct 8, 2026 · 215 revisions

Web Archiving Community

๐Ÿ’ฌ Join us on our new ArchiveBox community chat server: https://Zulip.ArchiveBox.io

๐Ÿ”ข Just getting started and want to learn more about why Web Archiving is important?
ย  ย ย  Check out this article: On the Importance of Web Archiving.


The internet archiving community is surprisingly far-reaching and almost universally friendly! It has some overlap with the scraping and OSINT worlds, but it's also kinda its own thing.

Whether you want to learn which organizations are the big players in the web archiving space, want to find a specific open source tool for your web archiving need, or just want to see where archivists hang out online, this is my attempt at an index of the entire web archiving community. The alternatives and recent reading list were reviewed in October 2026; older resources are retained for historical context.


The Master Lists

Indexes of archiving institutions and software maintained by other people. If there's anything archivists love doing, it's making lists.


Web Archiving Projects

ย  ย  ย  ย  ย  ย 

Legend
Cost ๐Ÿ†“ free option ยท ๐Ÿ’ฐ paid option (trials excluded)
Runs ๐Ÿ  local / self-hostable ยท โ˜๏ธ hosted option; both where available (sync-only labeled)
Capture ๐Ÿ“ธ static / no JavaScript ยท ๐Ÿš€ browser / JavaScript
Outputs / modes ๐Ÿ“œ article text ยท ๐ŸŽฅ embedded media ยท ๐Ÿ‡ผ WARC / WACZ ยท ๐Ÿงฉ specialized plugins / content modes
Inputs ๐Ÿ”– bookmark import; other input types named in rows
Coverage ? unverified ยท โ€” not applicable or no documented feature among the listed icons; HTML/PDF and other formats remain in descriptions

Individual bookmarkers and collectors

Project Cost Runs Capture Outputs / modes Inputs
Linkwarden Modern bookmarking UI with singlefile archiving ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿš€ ๐Ÿ“œ URLs, ๐Ÿ”– bookmarks, browser, RSS, files
Karakeep (formerly Hoarder) Self-hosted bookmark manager for links, notes, images and PDFs, with saved page content, search and optional AI tagging. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ“œ ๐ŸŽฅ URLs, ๐Ÿ”– bookmarks, RSS, files
Readeck Self-hosted read-later app preserving readable page content, with highlights, collections, full-text search and EPUB export. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs, ๐Ÿ”– bookmarks
Grimoire Local-first bookmark and content manager with readable content extraction, notes, tags and keyword/semantic search. ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ“œ ๐Ÿงฉ URLs, ๐Ÿ”– bookmarks, files
Django Link Archive Self-hosted link database and RSS reader with public-archive integration and yt-dlp downloads. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿš€ ๐ŸŽฅ ๐Ÿงฉ URLs, RSS, files
Wallabag / Wallabag.it Self-hostable web archiving server that can import via RSS ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿ“ธ ๐Ÿ“œ URLs, ๐Ÿ”– bookmarks
Shaarli Self-hostable bookmark tagging, archiving, and sharing service ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, ๐Ÿ”– bookmarks
Reminiscence extremely similar to ArchiveBox, uses a Django backend + UI and provides auto-tagging and summary features with NLTK ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ“œ ๐ŸŽฅ URLs, ๐Ÿ”– bookmarks, files
Shaarchiver very similar project that archives Firefox, Shaarli, or Delicious bookmarks and all linked media, generating a markdown/HTML index ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿ”– bookmarks
Archivy Python-based self-hosted knowledge base embedded into your filesystem ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs
Shiori Simple bookmark manager + readability archiver built with Go (like a clone of Pocket) ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs, ๐Ÿ”– bookmarks
LinkAce A self-hosted bookmark management tool that saves snapshots to archive.org ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, ๐Ÿ”– bookmarks
LinkDing Self-hosted bookmark manager that is designed be to be minimal, fast, and easy to set up using Docker. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs, ๐Ÿ”– bookmarks, browser, files
LinkWallet A self-hosted bookmark database with full-text page content search and limited archiving features ๐Ÿ†“ ๐Ÿ  ? โ€” URLs, ๐Ÿ”– bookmarks
Espial Bookmark manager and search tool with limited archiving features ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, ๐Ÿ”– bookmarks
Herodotus Django-based web archiving tool with a focus on collecting text-based content ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ“œ URLs
Buku Browser-independent bookmark manager CLI written in Python3 and SQLite3 ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, ๐Ÿ”– bookmarks
NeonLink Simple self-hosted bookmark management + Benotes note-taking app with limited archiving features ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, ๐Ÿ”– bookmarks
Erised Super simple CLI utility to bookmark and archive webpages ๐Ÿ†“ ๐Ÿ  ? โ€” URLs
Wayback Archiving in style like ArchiveBox, but with a chat. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ URLs
Gosuki A lightweight, open-source, privacy-first bookmark manager that unifies bookmarks across multiple browsers; its ArchiveBox integration can automatically archive tagged bookmarks. ๐Ÿ†“ ๐Ÿ  โ€” โ€” browser, ๐Ÿ”– bookmarks
Pinboard Bookmarking tool that provides archiving in a paid version, run by a single independent developer ๐Ÿ’ฐ โ˜๏ธ ? โ€” URLs, ๐Ÿ”– bookmarks
Raindrop Bookmarking tool with archiving in their paid version, run by a company est. 2011 ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ ? ๐Ÿงฉ URLs, ๐Ÿ”– bookmarks, files
Instapaper Bookmarking alternative to Pocket/Pinboard, with permanent article archives in Premium ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ ? ๐Ÿ“œ URLs, files
ReadWise A paid Pocket/Pinboard alternative that includes article snippet and highlight saving ๐Ÿ’ฐ โ˜๏ธ ? ๐Ÿ“œ URLs, RSS, files
Diigo Another brookmarking/annotation service with archiving as a paid feature ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ ? โ€” URLs
Trilium Notes Personal web UI based knowledge-base with web clipping and note-taking ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ“œ files, browser
Perkeep "Perkeep lets you permanently keep your stuff, for life." ๐Ÿ†“ ๐Ÿ  โ€” โ€” files, accounts
TiddlyWiki Non-linear bookmark and note-taking tool with archiving support ๐Ÿ†“ ๐Ÿ  โ€” โ€” files, notes
Joplin Desktop + mobile app for knowledge-base-style info collection and notes (w/ optional plugin for archiving) ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ (sync) ๐Ÿš€ ๐Ÿ“œ browser, files
Muse Paid macOS visual bookmark and media manager with browser clipping, local files, OCR/search and on-device tagging; one-time purchase with a free trial. ๐Ÿ’ฐ ๐Ÿ  ? โ€” browser, ๐Ÿ”– bookmarks, files
https://github.com/karlicoss/promnesia A browser extension that collects and collates all the URLs you visit into a hierarchical/graph structure with metadata ๐Ÿ†“ ๐Ÿ  โ€” โ€” browser, accounts, files
Timelinize Collect and browse personal data locally on a timeline; successor to the archived Timeliner. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿงฉ files, accounts
https://github.com/karlicoss/grasp capture webpages from Firefox and Chrome into Org-mode documents ๐Ÿ†“ ๐Ÿ  โ€” โ€” browser
SingleFile Web Extension / CLI util for Firefox and Chrome to save a web page as a single HTML file ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs, browser
Hoardy-Web Browser extension and local tools that capture HTTP requests/responses for offline replay, mirroring and indexing, including POST traffic. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” browser
WebScrapBook Browser extension for saving, organizing, annotating and editing page captures locally or with a backend server. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” browser
Packrat Chromium extension for archiving browsing history using ArchiveWeb.page capture and ReplayWeb.page replay. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ browser
Diskernet Archiving tool that uses the Chrome debugger protocol to save each page as-loaded in the browser (formerly 22120 by c0fe or i5ik) ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  ๐Ÿš€ โ€” browser
Monolith of Web Chrome extension using Monolith compiled to WebAssembly to save a static page as one HTML file. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” browser
https://github.com/vrtdev/save-page-state A Chrome extension for saving the state of a page in multiple formats ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” browser
StreamStash Windows app for recording livestreams and collecting social-media content in a local library; free tier and paid one-time licenses. See the vendorโ€™s StreamStash vs ArchiveBox comparison. ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ? ๐ŸŽฅ ๐Ÿงฉ URLs, accounts
Munin Archiver Social media archiver for Facebook, Instagram and VKontakte accounts. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ ๐Ÿงฉ accounts
Tube Archivist Self-hosted YouTube archive with downloads, metadata search and playback. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿงฉ URLs, channels, playlists
TubeSync Synchronizes YouTube channels/playlists to local directories and media servers. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿงฉ channels, playlists
ytdl-sub Automates yt-dlp subscription downloads and metadata generation for local media libraries. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿงฉ URLs, subscriptions
gallery-dl Command-line downloader for image galleries and collections, with metadata support. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿงฉ URLs, accounts
Slack Dumper Exports Slack messages, threads, files, channels and users to local storage. ๐Ÿ†“ ๐Ÿ  ? ๐Ÿงฉ accounts
tdl Telegram toolkit with file downloads and JSON export of messages and member/subscriber lists. ๐Ÿ†“ ๐Ÿ  ? ๐Ÿงฉ accounts
forum-dl Alpha-stage forum and mailing-list archiver exporting threads/boards to JSONL, mailbox formats or WARC. ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ‡ผ ๐Ÿงฉ URLs
TwitchDownloader Downloads Twitch VODs, clips and chat, with chat replay rendering. ๐Ÿ†“ ๐Ÿ  ? ๐ŸŽฅ ๐Ÿงฉ URLs, IDs
cobalt Self-hostable multi-site media downloader with a web UI and API. ๐Ÿ†“ ๐Ÿ  โ˜๏ธ ? ๐ŸŽฅ ๐Ÿงฉ URLs
https://archiveofourown.org/ โ€” โ€” โ€” โ€” works
https://github.com/HelloZeroNet/ZeroNet (super cool project) ๐Ÿ†“ ๐Ÿ  โ€” โ€” sites, files
Pocket Premium Bookmarking tool that provides an archiving service in their paid version, run by Mozilla (historical) โ€” โ€” โ€” โ€” โ€”
Sheetsee-Pocket project that provides a pretty auto-updating index of your Pocket links (without archiving them) (historical) โ€” โ€” โ€” โ€” โ€”
Pocket -> IFTTT -> Dropbox Post by Christopher Su on his Pocket saving IFTTT recipe (historical) โ€” โ€” โ€” โ€” โ€”
https://en.wikipedia.org/wiki/Furl (historical) โ€” โ€” โ€” โ€” โ€”

Journalists, lawyers and archivists

Project Cost Runs Capture Outputs / modes Inputs
ArchiveWeb.page Chrome extension for manual, interactive archiving of websites as you browse the web. Good for capturing high-fidelity complex interactions ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ browser
warcreate a Chrome extension for creating WARCs from any webpage ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ browser
Conifer by Rhizome.org An open-source personal archiving server that uses pywb under the hood. Previously affiliated with Webrecorder; legacy self-hosting available, hosted capture discontinued in June 2026. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ ๐Ÿงฉ browser, URLs
Scoop Create high-fidelity WARC/WACZ captures using a playwright browser, with support for signing, media extraction, PDFs, etc. (by the Perma.cc team) ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐ŸŽฅ ๐Ÿ‡ผ URLs
Hunchly A paid web archiving / session recording tool designed for OSINT ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿš€ โ€” browser
Auto Archiver Bellingcat's archiver for webpages, social posts, images and videos, with CLI/CSV/Google Sheets inputs and local or cloud storage. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐ŸŽฅ ๐Ÿ‡ผ ๐Ÿงฉ URLs, CSV, Sheets
Memex by Worldbrain.io a beautiful, user-friendly browser extension that archives all history with full-text search, annotation support, and more ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ (sync) ? ๐Ÿงฉ browser, ๐Ÿ”– bookmarks, history, PDFs
Hypothes.is a web/pdf/ebook annotation tool that also archives content ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ โ€” โ€” web, PDFs, images, video
Zotero collect, organize, cite, and share research (mainly for technical/scientific papers & citations) ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ (library/sync) ๐Ÿš€ ๐Ÿงฉ browser, files, identifiers
https://www.stillio.com ๐Ÿ’ฐ โ˜๏ธ ? โ€” URLs
SnapshotArchive Hosted scheduled screenshots and visual change monitoring, with PDF/HTML capture and archive retention depending on the plan. ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ ? โ€” URLs
https://github.com/dgtlmoon/changedetection.io Change detection and monitoring of web page content changes ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿ“ธ ๐Ÿš€ โ€” URLs, browser
Archive.org The O.G. Wayback Machine provided publicly by the Internet Archive (Archive.org) ๐Ÿ†“ โ˜๏ธ ๐Ÿš€ ๐Ÿ‡ผ URLs, browser, RSS
https://archive.is / https://archive.today ? โ˜๏ธ ? โ€” URLs
https://ghostarchive.org ? โ˜๏ธ ? โ€” URLs
https://perma.cc ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿš€ ๐Ÿ‡ผ URLs, browser, images, PDFs
Arquivo.pt Portuguese web archive with a services catalog and open-source tools. ๐Ÿ†“ โ˜๏ธ ? ๐Ÿ‡ผ URLs
https://archive.st ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ ๐Ÿš€ โ€” URLs
https://theoldnet.com/ ? โ˜๏ธ โ€” โ€” URLs, dates
https://timetravel.mementoweb.org/ ? ? โ€” โ€” ?
https://freezepage.com/ ๐Ÿ†“ โ˜๏ธ ? โ€” URLs, browser
https://webcitation.org/archive โ€” โ˜๏ธ โ€” โ€” URLs
https://megalodon.jp/ ? โ˜๏ธ ? โ€” URLs
https://www.webarchive.org.uk/ukwa/ ? ? ? โ€” ?
Google, Bing, DuckDuckGo, and other search engine caches (historical) โ€” โ€” โ€” โ€” URLs
Archive.it commercial Wayback Machine solution ๐Ÿ’ฐ โ˜๏ธ ๐Ÿ“ธ ๐Ÿš€ ๐ŸŽฅ ๐Ÿ‡ผ URLs
https://www.pagefreezer.com ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿš€ ๐ŸŽฅ ๐Ÿงฉ URLs, browser, social accounts
https://www.smarsh.com ๐Ÿ’ฐ โ˜๏ธ ? โ€” communications, websites
https://preservica.com/digital-archive-software-1/active-digital-preservation For-profit company offering a digital preservation software suite ๐Ÿ†“ ๐Ÿ’ฐ โ˜๏ธ โ€” โ€” files, Microsoft 365
Browsertrix Fully integrated (self hostable) SaaS web archiving platform ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿš€ ๐Ÿ‡ผ ๐Ÿงฉ URLs, WACZ files
WAIL Web archiver GUI using Heritrix and OpenWayback ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ URLs
WAIL (Electron) Electron app version of the original wail for creating and interacting with web archives ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ URLs
Sosse Self-hosted Selenium crawler and search engine with recurring crawls, authenticated browsing, locally rewritten HTML archives and downloaded assets. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿš€ โ€” URLs
Archivematica web GUI for institutional long-term archiving of web and other content ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ โ€” โ€” files
Heritrix The king of internet archiving crawlers, powers the Wayback Machine ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ URLs
Brozzler chrome headless crawler + WARC archiver maintained by Archive.org ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐ŸŽฅ ๐Ÿ‡ผ URLs
Grab-Site An easy preconfigured web crawler designed for backing up websites ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐ŸŽฅ ๐Ÿ‡ผ URLs
Zeno Go crawler for broad crawls or individual pages, recording HTTP traffic to WARC. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ URLs
kage CLI website mirror using headless Chrome to save static DOM snapshots and assets for offline viewing. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs, mirror files
ReadableWebProxy A proxying archiver that downloads content from sites and can snapshot multiple versions of sites over time ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ“œ ๐Ÿงฉ URLs
WWWofle old proxying recorder software similar to ArchiveBox ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs, browser proxy
Fossilo A commercial archiving solution that appears to be very similar to ArchiveBox ? ? ? โ€” ?

Developers and integrators

Project Cost Runs Capture Outputs / modes Inputs
ReplayWeb.page Web archive viewer that runs entirely in the browser and doesn't require any server-hosted component to view WARC and WACZ files. Also available as a standalone electron app for local desktop use ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC/WACZ files
pywb aka Python Wayback, the open source toolkit forked from archive.org for self-hosting your own wayback machine among other web archiving tools ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ‡ผ URLs, WARC/ARC files
ipwb A distributed web archiving solution using pywb with IPFS for storage ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC files
OpenWayback Open source project developing core Wayback Machine components ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC/ARC files
AUT Archives Unleashed Toolkit for analyzing web archives (formerly WarcBase) ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC/ARC files
Warclight A Rails engine for finding and searching web archives ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC/ARC indexes
WarcDB Converts WARC crawl data into SQLite databases for querying. ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC/WACZ files
WARC-GPT Experimental retrieval-augmented generation pipeline and web UI for exploring WARC collections. ๐Ÿ†“ ๐Ÿ  โ€” โ€” WARC files
sist2 Indexes saved file collections with text/metadata extraction, thumbnails, OCR and a search web interface; early development. ๐Ÿ†“ ๐Ÿ  โ€” โ€” files
abx-dl Standalone downloader using ArchiveBox plugins for webpage assets, media, PDFs, screenshots and other outputs. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿš€ ๐Ÿ“œ ๐ŸŽฅ ๐Ÿ‡ผ ๐Ÿงฉ URLs, URL lists
Sliver Archives small URL collections or copies existing captures using shot-scraper through pywb, preserving WARC/WACZ provenance. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ URLs, archive URLs
shot-scraper Browser screenshots, video recordings and JavaScript extraction from the command line. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs
WPull A pure python implementation of wget with WARC saving ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ URLs
Frozen Soup Python library and CLI that inline resources into self-contained HTML pages. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs
singlepage Go library and CLI that bundle CSS, JavaScript and images into standalone HTML. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs
webarchive-to-singlefile Converts Safari .webarchive files into resource-embedded HTML; requires Chrome and is tested on macOS. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” Safari webarchive files
Monolith CLI tool for saving complete web pages as a single HTML file ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs, HTML files/stdin
Obelisk Go package and CLI tool for saving web page as single HTML file ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs
Percollate A command-line tool to turn web pages into beautiful, readable PDF, EPUB, or HTML docs. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs, HTML, RSS/Atom
scrapy-playwright Scrapy download handler for JavaScript-rendered pages using Playwright, retaining Scrapy scheduling and item processing. ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs
Crawlee JavaScript/TypeScript toolkit for browser-rendered and HTTP crawling, link discovery, file downloads and storing extracted results. ๐Ÿ†“ ๐Ÿ’ฐ ๐Ÿ  โ˜๏ธ ๐Ÿ“ธ ๐Ÿš€ โ€” URLs
Browsertrix Crawler Command-line crawling application that powers Browsertrix's core crawling features ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ ๐Ÿงฉ URLs, browser profiles
Squidwarc User-scriptable, archival crawler using Chrome ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ ๐Ÿ‡ผ URLs
Photon a fast crawler with archiving and asset extraction support ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿงฉ URLs
Headless Chrome Crawler distributed web crawler built on puppeteer with screenshots ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs
markdown-crawler Recursively saves website content as Markdown, with depth/domain limits and resumable processing. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs
Courlan Python library and CLI for crawl URL normalization, filtering, deduplication and scheduling. ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, HTML
https://github.com/wkhtmltopdf/wkhtmltopdf Webkit HTML to PDF archiver/saver ๐Ÿ†“ ๐Ÿ  ๐Ÿš€ โ€” URLs, HTML files
ExtractNet Dragnet-based machine-learning extractor for article attributes such as author, headline, date and keywords; its README excludes boilerplate content extraction. ๐Ÿ†“ ๐Ÿ  โ€” โ€” HTML
Inscriptis HTML-to-text library, CLI and service preserving text layout, including nested tables and optional annotations. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ“œ HTML, URLs, files
htmldate Python library and CLI for identifying original and updated publication dates in webpages. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs, HTML
extruct Extracts embedded metadata including JSON-LD, Microdata, Microformats, Open Graph, RDFa and Dublin Core. ๐Ÿ†“ ๐Ÿ  โ€” โ€” HTML
Defuddle Extracts primary content and metadata as cleaned HTML or Markdown; library/CLI developed for Obsidian Web Clipper, currently a work in progress. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ ๐Ÿงฉ URLs, HTML, browser DOM
Newspaper4k Maintained Newspaper3k fork for article scraping, extraction and curation. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ“œ URLs, HTML
https://github.com/mozilla/readability tool for extracting article contents and text ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ“œ HTML DOM
WarcProx WARC proxy recording and playback utility ๐Ÿ†“ ๐Ÿ  ? ๐Ÿ‡ผ HTTP proxy traffic
WarcTools utilities for dealing with WARCs ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ WARC/ARC files
warcit Create a WARC file out of a folder full of assets ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ files, directories, ZIP files
warcio fast streaming asynchronous WARC reader and writer ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿ‡ผ WARC/ARC files, HTTP requests
node-warc Parse And Create Web ARChive (WARC) files with node.js ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ WARC files, HTTP records
JWARC A Java library for reading and writing WARC files. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿš€ ๐Ÿ‡ผ WARC files, URLs
warcat-rs Rust WARC manipulation library and CLI; rewrite of the earlier Python warcat. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ WARC files
Warchaeology Command-line tools to inspect, manipulate and validate WARC files. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ WARC files
Waczerciser Experimental tool to inspect, edit, extract and repackage WARC/WACZ files. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿ‡ผ WARC/WACZ files
archivenow tool that pushes urls into all the online archive services like Archive.is and Archive.org ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ ๐Ÿš€ ๐Ÿ‡ผ URLs
MemGator Memento aggregator providing CLI and server access to captures across configured web archives. ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs
Mink Chrome extension for finding archived versions of live pages and submitting pages to public archives. ๐Ÿ†“ ๐Ÿ  โ€” โ€” browser URLs
LinkChecker Recursively checks websites and local HTML for broken links, with filters and reports. ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs, HTML files
https://github.com/jsvine/waybackpack command-line tool that lets you download the entire Wayback Machine archive for a given URL ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs
https://github.com/hartator/wayback-machine-downloader Download an entire website from the Internet Archive Wayback Machine. ๐Ÿ†“ ๐Ÿ  ๐Ÿ“ธ โ€” URLs
https://github.com/Lifesgood123/prevent-link-rot Replace any broken URLs in some content with Wayback machine URL equivalents ๐Ÿ†“ ๐Ÿ  โ€” โ€” Markdown/text files
https://en.archivarix.com download an archived page or entire site from the Wayback Machine ๐Ÿ’ฐ โ˜๏ธ ๐Ÿ“ธ ๐Ÿงฉ domains, URLs
https://proofofexistence.com prove that a certain file existed at a given time using the blockchain ? โ˜๏ธ โ€” โ€” files
http://squidman.net/squidman/index.html ๐Ÿ†“ ๐Ÿ  โ€” โ€” proxy traffic
https://wordpress.org/plugins/broken-link-checker/ ๐Ÿ†“ ๐Ÿ  โ˜๏ธ โ€” โ€” WordPress site
http://freedup.org/ ? ? โ€” โ€” files
ArchiveBox QuickAdd Desktop utility for submitting URLs to an existing ArchiveBox instance. ๐Ÿ†“ ๐Ÿ  โ€” โ€” URLs
archivebox-reddit Exports Reddit comments, posts, upvotes or saved items to ArchiveBox. ๐Ÿ†“ ๐Ÿ  โ€” ๐Ÿงฉ Reddit accounts
ArchiveboxTelegramBot Telegram bot that submits URLs to ArchiveBox. ๐Ÿ†“ ๐Ÿ  โ€” โ€” Telegram URLs
https://github.com/TheCakeIsNaOH/xbs-to-archivebox A utility to sync xBrowserSync bookmarks with ArchiveBox ๐Ÿ†“ ๐Ÿ  โ€” โ€” xBrowserSync ๐Ÿ”– bookmarks

WACZ file format specification.

Third-party clients can target older ArchiveBox interfaces; check compatibility with your server version.


More tools and indexes.


Reading List

A collection of blog posts and articles about internet archiving, contact me / open an issue if you want to add a link here!


Blogs Friends of ArchiveBox


Articles We Like About Internet Archiving

If any of these links are dead, you can find an archived version on https://archive.sweeting.me or https://web.archive.org.


ArchiveBox-Specific Posts, Tutorials, and Guides

Tutorials describe the version available when written; use the current installation and usage docs for setup. This list favors original articles, practical guides, interviews, and substantial reviews with meaningful ArchiveBox coverage. Brief listicle entries and passing mentions are omitted. Syndicated copies and translated editions of the same article are represented by the original source rather than counted separately.

For other languages, see Articles in Other Languages below.

Recent English coverage (2022โ€“2026)

Additional English articles and ongoing guides

ArchiveBox Discussions in News & Social Media

  • 2024-02-14 ยท English: Web archiving with ArchiveBox โ€” Tom Ryder's 30-slide PLUG presentation and demonstration of ArchiveBox for personal daily archiving.

Articles in Other Languages

Original articles, guides, videos and discussions grouped by language. Dated entries are newest first; older and undated resources follow. Translated editions of the same work remain together.

Chinese ยท French ยท German ยท Italian ยท Japanese ยท Polish ยท Portuguese ยท Russian ยท Spanish

Chinese

French

German

Italian

Japanese

Polish

Portuguese

Russian

Spanish


Communities

Most Active Communities

Internet Archive ย  Webrecorder ย  Rhizome Conifer ย  ODU Web Science and Digital Libraries ย  Archives Unleashed ย  IIPC

Project directories: Internet Archive, Webrecorder, ODU WS-DL, Archives Unleashed, and IIPC.


Web Archiving Communities

Follow these technological and organizational archiving hubs for the latest archiving news.


General Archiving Foundations, Coalitions, Initiatives, and Institutes

Browse archiving foundations, coalitions, initiatives, and institutes

Find your local archiving group in the list and see how you can contribute!

You can find more organizations and initiatives on these other lists:


ArchiveBox Community Resources

ArchiveBox Chat Rooms

ArchiveBox on Social Media

ArchiveBox on Package Distribution Platforms


Reference

Guides

Initial Setup

Features & Integrations

Maintenance

More Info




ย 
ย 

Clone this wiki locally