Pinned repositories

  1. TransparencyToolkit

    Main repository for Transparency Toolkit

    21 5

  2. Harvester

    Web crawling and document processing through a usable interface.

    JavaScript 46 11

  3. LookingGlass

    Intuitive and configurable search interface for document archives.

    Ruby 151 34

  • Intuitive and configurable search interface for document archives.

    Ruby 151 34 GPL-3.0 Updated Aug 1, 2018
  • Universal backend for indexing, storing, and querying documents.

    Ruby 12 4 GPL-3.0 Updated Jul 17, 2018
  • Ansible roles for deployment. In development, expect problems.

    Python Updated Jun 19, 2018
  • Ruby 1 Updated Jun 5, 2018
  • NSA documents in machine readable form

    Ruby 64 27 Updated Jun 5, 2018
  • Scripts for managing scrapers

    Ruby 20 4 GPL-3.0 Updated May 21, 2018
  • OCR server for hosted archiving service

    Ruby 1 1 GPL-3.0 Updated May 14, 2018
  • Test data for Transparency Toolkit development

    HTML Updated May 13, 2018
  • Upload application for documents in archiving service.

    Ruby 1 1 GPL-3.0 Updated May 11, 2018
  • Manages communications over UDP between different parts of the pipeline

    Ruby GPL-3.0 Updated May 10, 2018
  • Ruby 1 1 GPL-3.0 Updated Mar 7, 2018
  • Methods for encrypting and verifying documents. Utility gem for document processing pipeline.

    Ruby GPL-3.0 Updated Feb 25, 2018
  • Web crawling and document processing through a usable interface.

    JavaScript 46 11 GPL-3.0 Updated Jul 22, 2017
  • Backend for processing document suggestions from LookingGlass

    Ruby 1 Updated Jun 30, 2017
  • Raw data and scripts for Surveillance Research Archive

    Ruby 4 1 Updated Jun 1, 2017
  • OCRs document and extracts metadata

    Ruby 4 1 GPL-3.0 Updated May 29, 2017
  • Runs block of code on every file in directory

    Ruby 3 1 GPL-3.0 Updated May 28, 2017
  • Main repository for Transparency Toolkit

    21 5 1 issue needs help Updated May 25, 2017
  • API for calling crawlers

    Ruby 4 3 GPL-3.0 Updated May 22, 2017
  • Collects listings for jobs that require security clearance.

    Ruby 3 1 GPL-3.0 Updated Mar 14, 2017
  • Incremental crawler result reporting for Transparency Toolkit

    Ruby 1 GPL-3.0 Updated Mar 14, 2017
  • Dataspec for cleared job listings

    1 GPL-3.0 Updated Mar 13, 2017
  • A crawler for Twitter

    Ruby 3 1 GPL-3.0 Updated Mar 1, 2017
  • A collection of branding, interfaces, and other visual resources!

    PostScript 6 3 Updated Mar 1, 2017
  • LookingGlass dataspec for tweets

    1 GPL-3.0 Updated Feb 27, 2017
  • Crawls public LinkedIn profiles

    Ruby 8 4 GPL-3.0 Updated Feb 18, 2017
  • Scrapes all pages on any site you specify for keywords.

    Ruby 22 6 GPL-3.0 Updated Feb 18, 2017
  • Resume data and scripts for managing it

    Ruby 73 33 GPL-3.0 Updated Feb 5, 2017
  • A crawler for converting email files on disk to JSON

    Ruby 5 4 Updated Jan 19, 2017
  • Dataspec for emails

    1 1 Updated Jan 4, 2017