Skip to content
Apache Submarine is a unified AI platform
Java Vue Shell JavaScript Python TypeScript Other
Branch: master
Clone or download
liuxunorg and yuanzac SUBMARINE-295. Modify mysql database submarineDB to submarine
### What is this PR for?
Because the mysql database name is case sensitive,
Now submarine has 2 mysql databases. They are `submarineDB` and `submarineDB_test`.
It is easy to write `DB` as `db` in program development, Causes a program error and is difficult to troubleshoot.
So modify the `submarineDB` and `submarineDB_test` database to `submarine` and `submarine_test`.

### What type of PR is it?
Improvement

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/SUBMARINE-295

### How should this be tested?
* [CI Pass](https://travis-ci.org/liuxunorg/submarine/builds/614558566)

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Xun Liu <liuxun@apache.org>

Closes #95 from liuxunorg/SUBMARINE-295 and squashes the following commits:

549c12a [Xun Liu] SUBMARINE-295. Modify mysql database submarineDB to submarine
Latest commit 64ecfe9 Nov 20, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github [SUBMARINE-93] Add github ISSUE_TEMPLATE (#4) Sep 19, 2019
bin SUBMARINE-292. Merge workbench server into submarine server Nov 19, 2019
conf SUBMARINE-292. Merge workbench server into submarine server Nov 19, 2019
dev-support SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
docs SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
licenses-binary [Submarine 169] Revisit license and notice (#67) Sep 19, 2019
submarine-all SUBMARINE-66. Improve TF config env JSON generator + tests Nov 19, 2019
submarine-client SUBMARINE-276. Submarine runtime class config should be defined in th… Nov 16, 2019
submarine-commons SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
submarine-dist SUBMARINE-294. Unified submarine jar package name prefix Nov 20, 2019
submarine-sdk/pysubmarine SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
submarine-server SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
submarine-test SUBMARINE-294. Unified submarine jar package name prefix Nov 20, 2019
submarine-workbench SUBMARINE-294. Unified submarine jar package name prefix Nov 20, 2019
submodules [SUBMARINE-104]. YARN application type should set SUBMARINE Oct 6, 2019
.asf.yaml Hotfix enabled squash merge buttons Nov 8, 2019
.gitignore SUBMARINE-257. Submarine web user manager page with angular Nov 11, 2019
.gitmodules SUBMARINE-137. Introducing linked tony via submodule (#54) Sep 19, 2019
.travis.yml SUBMARINE-295. Modify mysql database submarineDB to submarine Nov 21, 2019
LICENSE [Submarine 169] Revisit license and notice (#67) Sep 19, 2019
LICENSE-binary [Submarine 169] Revisit license and notice (#67) Sep 19, 2019
NOTICE [Submarine 169] Revisit license and notice (#67) Sep 19, 2019
NOTICE-binary [Submarine 169] Revisit license and notice (#67) Sep 19, 2019
README.md SUBMARINE-285. Add .asf.yaml file Nov 7, 2019
pom.xml SUBMARINE-66. Improve TF config env JSON generator + tests Nov 19, 2019

README.md

color_logo_with_text

What is Apache Submarine?

Apache Submarine is a unified AI platform which allows engineers and data scientists to run Machine Learning and Deep Learning workload in distributed cluster.

Goals of Submarine:

  • It allows jobs easy access data/models in HDFS and other storages.
  • Can launch services to serve TensorFlow/PyTorch models.
  • Support run distributed TensorFlow jobs with simple configs.
  • Support run user-specified Docker images.
  • Support specify GPU and other resources.
  • Support launch TensorBoard for training jobs if user specified.
  • Support customized DNS name for roles (like TensorBoard.$user.$domain:6006)

Architecture

image-20190811191220934

Components

Submarine Workbench

Submarine Workbench is a WEB system. Algorithm engineers can perform complete lifecycle management of machine learning jobs in the Workbench.

  • Projects

    Manage machine learning jobs through project.

  • Data

    Data processing, data conversion, feature engineering, etc. in the workbench.

  • Job

    Data processing, algorithm development, and model training in machine learning jobs as a job run.

  • Model

    Algorithm selection, parameter adjustment, model training, model release, model Serving.

  • Workflow

    Automate the complete life cycle of machine learning operations by scheduling workflows for data processing, model training, and model publishing.

  • Team

    Support team development, code sharing, comments, code and model version management.

Submarine Core

The submarine core is the execution engine of the system and has the following features:

  • ML Engine

    Support for multiple machine learning framework access, such as tensorflow, pytorch.

  • Data Engine

    Docking the externally deployed Spark calculation engine for data processing.

  • SDK

    Support Python, Scala, R language for algorithm development, The SDK is provided to help developers use submarine's internal data caching, data exchange, and task tracking to more efficiently improve the development and execution of machine learning tasks.

  • Submitter

    Compatible with the underlying hybrid scheduling system of yarn and k8s for unified task scheduling and resource management, so that users are not aware.

  • Hybrid Scheduler
    • YARN
    • Kubernetes

Quick start

Run mini-submarine in one step

You can use mini-submarine for a quick experience submairne.

This is a docker image built for submarine development and quick start test.

Installation and deployment

Read the Quick Start Guide

Apache Submarine Community

Read the Apache Submarine Community Guide

How to contribute Contributing Guide

License

The Apache Submarine project is licensed under the Apache 2.0 License. See the LICENSE file for details.

You can’t perform that action at this time.