by ex-googlers, for ex-googlers - a lookup table of similar tech & services
Switch branches/tags
Nothing to show
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
ENG_PRACTICES.md Update ENG_PRACTICES.md Dec 6, 2018
README.md Update README.md Nov 20, 2018

README.md

A handy lookup table of similar technology and services to help ex-googlers survive the real world :)
pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products

Technology

Core Technology

Google Internal Google External Open Source / Real-World
MapReduce Apache Hadoop, Spark
Protocol Buffer Protobuf Cap'n Proto, Thrift, Avro, Amazon Ion
Stubby gRPC Bolt, Thrift
Chubby Apache Zookeeper, etcd
? Apache Kafka, Apache Pulsar

Infrastructure

Google Internal Google External Open Source / Real-World
Borg Kubernetes Apache Mesos
GSLB (load balancer) ELB, HAProxy, Istio, F5, envoy
data center hardware open compute

Storage

Google Internal Google External Open Source / Real-World
GFS/Colossus HDFS, Ceph, GlusterFS
BigTable Cloud BigTable PrestoDB, Cassandra, HBase, Accumulo, DynamoDB
Spanner Cloud Spanner CockroachDB, TiDB
ColumnIO / Capacitor Apache Parquet
sstable levelDB
zippy Snappy lz4

Services

Google Internal Google External Open Source SaaS
Dremel bigquery Apache Drill, Presto, Spark(sort-of), AWS Athena, Redshift Spectrum
Dremel UI Redash
Search (Mustang, Alexandria) Elasticsearch, Solr, Lucene algolia
pubsub pubsub RabbitMQ, PubNub AWS SQS/SNS
Flume (Java) Apache Crunch

DevOps

Google Internal Google External Real-World
Assimilator Dominator
Blaze Bazel Buck, Pants, please.build
Oncall PagerDuty, OpsGenie, VictorOps
varz/borgmon Datadog, Prometheus, librato, newrelic, skylight, scout, Scotty/tricorder, also this and this
Exception/Error Tracking (??) Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps
styleguides google styleguides PEP-8, HoundCI auto-style-reviewer
blaze test / build / CI buildkite, circleCI, travis, jenkins, gitlabCI
continuous delivery / releasing Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI
borg / borgcfg / gcl AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, Jsonnet
logging StackDriver logstash, fluentd, papertrail, cernan
CodeSearch Zoekt Sourcegraph
cider Eclipse Che, Cloud9
buganizer JIRA, bugzilla, github issues
ToTT Google Test Blog

Security

Google Internal Google External Real-World
prodaccess Keymaster
prod secrets management knox

IT / Operations

Google Internal Real-World
free food :(
software engineering at google
valentine 1Password, Lastpass
OWNERS files in repo github CODEOWNERS
snippets Khan/snippets
memegen memegen
edge, people ops training LifeLabs
googlegeist Culture Amp, humu
stuff (SaaS IT management) intello, zylo
stuff (Device Management) Fleetsmith, jamf
go/ links golinks, go

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.