TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

DC/OS - The Datacenter Operating System

Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.

Jupyter magics and kernels for working with remote Spark clusters

OpenQuake Engine: a software for Seismic Hazard and Risk Analysis

Create clusters of VMs on the cloud and configure them with Ansible.

A UI for local Kubernetes clusters created with k3d

Best practices & guides on how to write distributed pytorch training code

短文本聚类预处理模块 Short text cluster

GUI based offensive penetration testing tool (Open Source)

Mirror of Kubernetes Web View: allows to list and view all Kubernetes resources (incl. CRDs) with permalink-friendly URLs in a plain-HTML frontend

face-cluster-by-infomap 一种无监督人脸聚类方法，在开源数据集上取得SOTA效果

Monitor your GPUs whether they are on a single computer or in a cluster

A tool and library for easily deploying applications on Apache YARN

The Workload Analyzer collects Presto® and Trino workload statistics, and analyzes them

ClusterSSH for iTerm2

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

my tools working with redis

a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine

A docker based multi machine render setup