Skip to content

云原生一站式机器学习平台,多租户,数据资产,notebook在线开发,拖拉拽任务流编排,多机多卡分布式训练,超参搜索,推理服务,多集群调度,多项目组资源组,边缘计算,大模型实时训练, ai应用商店

License

tencentmusic/cube-studio

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Nov 29, 2022
Nov 7, 2022
Oct 24, 2022
Oct 9, 2022
Nov 23, 2022
Nov 15, 2021

Cube Studio

English | 简体中文

Infra

image

cube-studio is a one-stop cloud-native machine learning platform open sourced by Tencent Music, Currently mainly includes the following functions

  • 1、data management: feature store, online and offline features; dataset management, structure data and media data, data label platform
  • 2、develop: notebook(vscode/jupyter); docker image management; image build online
  • 3、train: pipeline drag and drop online; open template market; distributed computing/training tasks, example tf/pytorch/mxnet/spark/ray/horovod/kaldi/volcano; batch priority scheduling; resource monitoring/alarm/balancing; cron scheduling
  • 4、automl: nni, ray
  • 5、inference: model manager; serverless traffic control; tf/pytorch/onnx/tensorrt model deploy, tfserving/torchserver/onnxruntime/triton inference; VGPU; load balancing、high availability、elastic scaling
  • 6、infra: multi-user; multi-project; multi-cluster; edge cluster mode; blockchain sharing;

Doc

https://github.com/tencentmusic/cube-studio/wiki

WeChat group

learning、deploy、consult、contribution、cooperation, join group, wechart id luanpeng1234 remark<open source>, construction guide

Job Template

tips:

  • 1、You can develop your own template, Easy to develop and more suitable for your own scenarios
template type describe
linux base Custom stand-alone operating environment, free to implement all custom stand-alone functions
datax import export Import and export of heterogeneous data sources
hadoop data processing hdfs,hbase,sqoop,spark client
sparkjob data processing spark serverless
volcanojob data processing volcano multi-machine distributed framework
ray data processing python ray multi-machine distributed framework
ray-sklearn machine learning sklearn based on ray framework supports multi-machine distributed parallel computing
xgb machine learning xgb model training and inference
tfjob deep learning Multi-machine distributed training of tensorflow
pytorchjob deep learning Multi-machine distributed training of pytorch
horovod deep learning Multi-machine distributed training of horovod
paddle deep learning Multi-machine distributed training of paddle
mxnet deep learning Multi-machine distributed training of mxnet
kaldi deep learning Multi-machine distributed training of kaldi
tfjob-train model train distributed training of tensorflow: plain and runner
tfjob-runner model train distributed training of tensorflow: runner method
tfjob-plain model train distributed training of tensorflow: plain method
tf-model-evaluation model evaluate distributed model evaluation of tensorflow2.3
tf-offline-predict model inference distributed offline model inference of tensorflow2.3
model-register model service register model to platform
model-offline-predict model service distributed offline model inference of framework
deploy-service model service deploy inference service
media-download multimedia data processing Distributed download of media files
video-audio multimedia data processing Distributed extraction of audio from video
video-img multimedia data processing Distributed extraction of pictures from video
object-detection-on-darknet machine vision object-detection with darknet yolov3
ner natural language Named Entity Recognition

Deploy

wiki

cube

Company

image

About

云原生一站式机器学习平台,多租户,数据资产,notebook在线开发,拖拉拽任务流编排,多机多卡分布式训练,超参搜索,推理服务,多集群调度,多项目组资源组,边缘计算,大模型实时训练, ai应用商店

Topics

Resources

License

Stars

Watchers

Forks