Skip to content
C++ Python Jupyter Notebook Cuda Perl Shell Other
Branch: feature/persis…
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Update PR Template (#9919) Mar 17, 2018
.vscode work on background Apr 2, 2018
3rdparty
R-package fix sth Apr 13, 2018
amalgamation [MXNET-120] Float16 support for distributed training (#10183) Apr 11, 2018
benchmark/python [MXNET-72] Improve sparse.adam_update (#10062) Mar 27, 2018
ci Minor simplifications in ci/build.py (#10496) Apr 11, 2018
cmake Remove Fermi from cmake (#10486) Apr 11, 2018
cpp-package fix sth Apr 13, 2018
docker [Perl] Sparse feature. (#9988) Mar 6, 2018
docs fix sth Apr 13, 2018
example fix sth Apr 13, 2018
include/mxnet fix ptr_ type Apr 13, 2018
make [MXNET-120] Float16 support for distributed training (#10183) Apr 11, 2018
mshadow fix Mar 8, 2018
perl-package fix sth Apr 13, 2018
plugin fix plugin bug (#10221) Mar 23, 2018
python Prevent partially update of ParameterDict (#10515) Apr 12, 2018
scala-package fix sth Apr 13, 2018
setup-utils Script to install on a python virtualenv for development (#9685) Mar 29, 2018
src fix sth Apr 13, 2018
tests
tools fix sth Apr 13, 2018
.gitattributes [R] To ignore R-pkg when releasing on github (#7007) Jul 13, 2017
.gitignore fix sth Apr 13, 2018
.gitmodules move dmlc-core & make" (#10231) Mar 26, 2018
.mxnet_root CI docker revamp; Add Jetson, Raspberry and CentOS 7 build [MXNET-42]… Mar 9, 2018
.travis.yml Add h5py support to NDArrayIter (#6790) Jul 18, 2017
CMakeLists.txt
CODEOWNERS [MXNET-120] Float16 support for distributed training (#10183) Apr 11, 2018
CONTRIBUTORS.md [MXNET-120] Float16 support for distributed training (#10183) Apr 11, 2018
DISCLAIMER Add DISCLAIMER and lxn2 GPG keys (#7344) Aug 5, 2017
Jenkinsfile [MXNET-120] Float16 support for distributed training (#10183) Apr 11, 2018
KEYS Add my key (#9736) Feb 7, 2018
LICENSE Add mkldnn (#10412) Apr 5, 2018
MKL_README.md remove MKL_EXPERIMENTAL and update make files for MKL-DNN (#9810) Feb 25, 2018
Makefile err Apr 13, 2018
NEWS.md [MXNET-67] Sync master with v1.1.0 branch (#10031) Mar 8, 2018
NOTICE Update NOTICE (#9706) Feb 6, 2018
README.md Update README.md Nov 16, 2019
appveyor.yml License Adds - some more (#9559) Jan 26, 2018
banana.png continue work Feb 17, 2018
client_run continue work Feb 15, 2018
config.mk err Apr 13, 2018
prepare_mkl.sh [ReviewRequired] License Fixes post 1.0.0 Release (based on Voting Th… Jan 22, 2018
prepare_mkldnn.sh fix nproc arg (#10404) Apr 4, 2018
readthedocs.yml [docs] add favicon and fix index html title Mar 25, 2016
snap.python Add snapcraft packaging (#4852) Mar 23, 2017
snapcraft.yaml Bump version to 1.2.0 (#9840) Feb 21, 2018

README.md

TRIMS (MXNet with Persistent GPU Memory)

Reference

Dakkak, A., Li, C., De Gonzalo, S. G., Xiong, J., & Hwu, W-M. W. (2019). TrIMS: Transparent and isolated model sharing for low latency deep learning inference in function-as-a-service. In E. Bertino, C. K. Chang, P. Chen, E. Damiani, M. Goul, & K. Oyama (Eds.), Proceedings - 2019 IEEE International Conference on Cloud Computing, CLOUD 2019 - Part of the 2019 IEEE World Congress on Services (pp. 372-382). [8814494] (IEEE International Conference on Cloud Computing, CLOUD; Vol. 2019-July). IEEE Computer Society. https://doi.org/10.1109/CLOUD.2019.00067

Installation

Requirements

Setup your target location (make sure that LD_LIBRARY_PATH and PATH has been updated to include this)

export UPR_INSTALL_PREFIX=$HOME/.usr
export UPR_BASE_DIR=$HOME/carml/data/mxnet

make sure that both directories exist

mkdir -p $UPR_INSTALL_PREFIX
mkdir -p $UPR_BASE_DIR

Remember to update your PKG_CONFIG_PATH

export PKG_CONFIG_PATH=$UPR_INSTALL_PREFIX/lib/pkgconfig/:$PKG_CONFIG_PATH

C-ARES

wget https://c-ares.haxx.se/download/c-ares-1.13.0.tar.gz
tar -xf c-ares-1.13.0.tar.gz
cd c-ares-1.13.0
./buildconf
./configure --prefix=$UPR_INSTALL_PREFIX --disable-dependency-tracking --disable-debug
make install

Protobuf

wget https://github.com/google/protobuf/archive/v3.5.1.tar.gz
tar -xf v3.5.1.tar.gz
cd protobuf-3.5.1
./autogen.sh
./configure --prefix=$UPR_INSTALL_PREFIX --disable-dependency-tracking --disable-debug --with-zlib
make
make install

Protobuf-c

wget https://github.com/protobuf-c/protobuf-c/releases/download/v1.3.0/protobuf-c-1.3.0.tar.gz
tar -xf protobuf-c-1.3.0.tar.gz
cd protobuf-c-1.3.0
./configure --prefix=$UPR_INSTALL_PREFIX --disable-dependency-tracking --disable-debug
make install

GRPC

wget https://github.com/grpc/grpc/archive/v1.9.1.tar.gz
tar -xf v1.9.1
cd v1.9.1
make install prefix=$UPR_INSTALL_PREFIX
make install-plugins prefix=$UPR_INSTALL_PREFIX

OpenCV (Optional)

wget https://github.com/opencv/opencv/archive/3.4.1.tar.gz
tar -xf 3.4.1.tar.gz
cd opencv-3.4.1/
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$UPR_INSTALL_PREFIX -DWITH_CUDA=OFF -DWITH_OPENCL=OFF ..
make
make install

From APT-GET

Other requirements can be installed using APT . The base requirements are listed in the (MXNet installation guide)[https://mxnet.apache.org/install/index.html].

Regenerate GRPC Code

It is not recommended to regenerate the protofile, since it's dependent on the protobuf version. The only time you need to regenerate it is after updating it.

cd src/c_api
make

Build Server

The server is part of the MXNet build process.

Downloading Models

Running

Server

Client

Profiling

Environment Variables

Name Description Default Value
UPR_ENABLED true
UPR_PROFILE_IO only makes sense if UPR_ENABLED=false true
UPR_RUN_ID [undefined]
git build_git_sha
UPR_GIT_BRANCH build_git_branch
UPR_GIT_TIME build_git_time
UPR_CLIENT
UPR_BASE_DIR
UPR_MODEL_NAME
UPR_PROFILE_TARGET profile.json
UPR_INITIALIZE_EAGER false
UPR_INITIALIZE_EAGER_ASYNC false
UPR_INPUT_CHANNELS 3
UPR_INPUT_WIDTH 224
UPR_INPUT_HEIGHT 224
UPR_INPUT_MEAN_R 0
UPR_INPUT_MEAN_G 0
UPR_INPUT_MEAN_B 0
UPR_ENABLE_MEMORY_PROFILE false
UPR_ENABLE_CUDA_FREE false
UPR_SHARING_GRANULARITY model
-------------------------- ----------- -------------
UPRD_EVICTION_POLICY LRU
UPRD_ESTIMATION_RATE 1.0
UPRD_MEMORY_PERCENTAGE 0.8
UPRD_PERSIST_CPU true
UPRD_PERSIST_ONLY_CPU only persist on cpu memory false
UPRD_WRITE_PROFILE write server profile file false
UPRD_ESTIMATE_WITH_INTERNAL_MEMORY use internal memory info for estimate true
You can’t perform that action at this time.