Transparency Toolkit

Repositories

• ArchiveAdministrator

  Archive administration system. Handles archive creation and user authentication.
  CSS GPL-3.0 0 1 23 7 Updated May 20, 2021

• DesignAssets

  A collection of branding, interfaces, and other visual resources!
  HTML 3 5 0 0 Updated Apr 26, 2021

• LookingGlass

  Intuitive and configurable search interface for document archives.

  search interface archive document
  Ruby GPL-3.0 36 180 19 0 Updated Nov 11, 2019

• OCRServer

  OCR server for hosted archiving service
  Ruby GPL-3.0 0 1 6 0 Updated Nov 10, 2019

• DocManager

  Universal backend for indexing, storing, and querying documents.

  api elasticsearch document-management
  Ruby GPL-3.0 4 19 13 0 Updated Nov 10, 2019

• DocUpload

  Upload application for documents in archiving service.
  CSS GPL-3.0 0 1 2 0 Updated Nov 3, 2019

• tt-ansible

  Ansible roles for deployment. In development, expect problems.
  Python 0 0 0 0 Updated Jul 3, 2019

• IndexServer

  Receives, decrypts, and verifies data, then indexes with DocManager
  Ruby GPL-3.0 0 0 0 0 Updated Jul 3, 2019

• Catalyst

  Text mining framework.
  Ruby GPL-3.0 1 4 4 0 Updated Apr 30, 2019

• Test-Data

  Test data for Transparency Toolkit development
  HTML 1 0 0 0 Updated Mar 20, 2019

• UtilityScripts

  Scripts for managing scrapers
  Ruby GPL-3.0 5 19 0 0 Updated Nov 12, 2018

• catalyst_test_scripts

  Test scripts for all Catalyst methods.
  Ruby 0 0 0 0 Updated Oct 13, 2018

• DocIntegrityCheck

  Methods for encrypting and verifying documents. Utility gem for document processing pipeline.
  Ruby GPL-3.0 0 0 0 0 Updated Sep 12, 2018

• gj-docs
  Ruby 1 0 0 0 Updated Jun 5, 2018

• NSA-Data

  NSA documents in machine readable form
  Ruby 36 77 1 0 Updated Jun 5, 2018

• Harvester

  Web crawling and document processing through a usable interface.

  api ocr osint web interface crawling document
  JavaScript GPL-3.0 16 60 3 0 Updated Jul 22, 2017

• DocSuggestions

  Backend for processing document suggestions from LookingGlass
  Ruby 1 0 0 0 Updated Jun 30, 2017

• Surveillance-Research-Data

  Raw data and scripts for Surveillance Research Archive
  Ruby 2 6 0 0 Updated Jun 1, 2017

• ParseFile

  OCRs document and extracts metadata
  Ruby GPL-3.0 1 3 0 0 Updated May 29, 2017

• DirCrawl

  Runs block of code on every file in directory
  Ruby GPL-3.0 1 2 0 0 Updated May 28, 2017

• TransparencyToolkit

  Main repository for Transparency Toolkit
  10 43 6 0 Updated May 25, 2017

• CrawlerManager

  API for calling crawlers
  Ruby GPL-3.0 3 3 0 0 Updated May 22, 2017

• TSJobCrawler

  Collects listings for jobs that require security clearance.
  Ruby GPL-3.0 1 3 0 0 Updated Mar 14, 2017

• HarvesterReporter

  Incremental crawler result reporting for Transparency Toolkit
  Ruby GPL-3.0 1 0 0 0 Updated Mar 14, 2017

• dataspec-TsjobCrawl

  Dataspec for cleared job listings
  GPL-3.0 1 0 0 0 Updated Mar 13, 2017

• TwitterCrawler

  A crawler for Twitter
  Ruby GPL-3.0 2 6 0 0 Updated Mar 1, 2017

• dataspec-TwitterCrawl

  LookingGlass dataspec for tweets
  GPL-3.0 1 0 0 0 Updated Feb 27, 2017

• LinkedinCrawler

  Crawls public LinkedIn profiles
  Ruby GPL-3.0 6 8 0 0 Updated Feb 18, 2017

• generalscraper

  Scrapes all pages on any site you specify for keywords.
  Ruby GPL-3.0 7 23 2 0 Updated Feb 18, 2017

• ICWATCH-Data

  Resume data and scripts for managing it
  Ruby GPL-3.0 36 85 0 0 Updated Feb 5, 2017