Grow your team on GitHub
GitHub is home to over 28 million developers working together. Join them to grow your own development teams, manage permissions, and collaborate on projects.Sign up
Data. Together. Let's read about it!
Parallelized web crawler written in Golang
A static-generated website to introduce the Data Together project, built with Hugo.
Websockets-based API backend for our react and redux webapp.
Guide for Deploying the Data Together Reference Platform
Serves the Data Together JSON API
Coordinating technical work & roadmapping additional services
command line tool for storing data together on the distributed web
golang package for creating/working with warc & cdxj archives
Golang package that provides utils for working with dotsql and postgres
Golang WARC (Web ARChive) Library
Golang package implementing the CDXJ file format used by OpenWayback 3.0.0+ to index web archive contents
modify the contents of web-related content types for archival purposes
Library that extracts xmp metadata from a PDF document.
Golang package for parsing Extensible Metadata Platform (XMP) documents
Web application to allow users to add content metadata about crawled resources
Python package for scraping websites into the Data Together pipeline via morph.io
Golang package for making sensible guesses about file formats from Url strings
extract urls of dependant files for displaying a web page
Service for managing & executing archiving tasks written in Golang
Experimental Golang implementation of the ipfs datastore interface for sql databases.
Core Archive Model Definitions
Command line tool for extracting urls from a HTML web page using a jquery-style selector
Service for serving archived content stored on amazon S3.
Learning materials for Data Together
Project for visualizing the status of digital data archiving efforts across various data repositories
Golang package for working with linked data structures, an implementation of the W3C Data Catalog Vocabulary (DCAT)
Open-licensed artwork related to Data Together
User & Identity Management server