Skip to content
This repository has been archived by the owner. It is now read-only.
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

T4 features and code have moved to quilt

docs on_gitbook chat on_slack codecov pypi

Overview

Rethinking S3: Announcing T4, a team data hub.

A team data hub for S3

  • T4 adds search, content preview, versioning, and a Python API to any S3 bucket
  • Every file in T4 is versioned and searchable
  • T4 is for data scientists, data engineers, and data-driven teams

Use cases

  • Collaborate - get everyone on the same page by pointing them all to the same immutable data version
  • Experiment faster - blob storage is schemaless and scalable, so iterations are quick
  • Recover, rollback, and reproduce with immutable packages
  • Understand what's in S3 - plaintext and faceted search over S3

Key features

  • Browse, search any S3 bucket
  • Preview images, Jupyter notebooks, Vega visualizations - without downloading
  • Read/write Python objects to and from S3
  • Immutable versions for objects, immutable packages for collections of objects

Components

  • /catalog (JavaScript) - Search, browse, and preview your data in S3
  • /api/python - Read, write, and annotate Python objects in S3

Roadmap