Skip to content

Commit

Permalink
[docs] Update README and index of docs, add get start docs (#15)
Browse files Browse the repository at this point in the history
  • Loading branch information
JackMoriarty committed Dec 28, 2021
1 parent 1e0df0e commit 03f0ad4
Show file tree
Hide file tree
Showing 3 changed files with 117 additions and 6 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ $ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
```
### **Install whl package**
```
$ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2106-cp36-cp36m-linux_x86_64.whl
$ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+${version}-cp36-cp36m-linux_x86_64.whl
```


Expand All @@ -87,18 +87,18 @@ $ pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+deeprec2106-cp36-cp36m-linu
### **Nightly Images**
#### Image for GPU CUDA11.0
```
registry.cn-shanghai.aliyuncs.com/pai-dlc/tensorflow-training:deeprec-nightly-gpu-py36-cu110-ubuntu18.04
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec-nightly-gpu-py36-cu110-ubuntu18.04
```
#### Image for CPU
```
registry.cn-shanghai.aliyuncs.com/pai-dlc/tensorflow-training:deeprec-nightly-cpu-py36-ubuntu18.04
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec-nightly-cpu-py36-ubuntu18.04
```


***
## **User Document (Chinese)**

[https://deeprec.readthedocs.io/en/latest/](https://deeprec.readthedocs.io/en/latest/)
[https://deeprec.rtfd.io](https://deeprec.rtfd.io)

## **License**

Expand Down
103 changes: 103 additions & 0 deletions docs/Compile-And-Install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# DeepRec源代码编译&安装

## 开发环境准备

**CPU Base Docker Image**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-base-cpu-py36-ubuntu18.04
```

**GPU(cuda11.0) Base Docker Image**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-base-gpu-py36-cu110-ubuntu18.04
```

**CPU Dev Docker (with bazel cache)**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-cpu-py36-ubuntu18.04
```

**GPU(cuda11.0) Dev Docker (with bazel cache)**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-developer:deeprec-dev-gpu-py36-cu110-ubuntu18.04
```

## 代码编译

```bash
./configure
```

**GPU/CPU版本编译**

```bash
bazel build -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package
```

**GPU/CPU版本编译+ABI=0**

```bash
bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --host_cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" -c opt --config=opt //tensorflow/tools/pip_package:build_pip_package
```

**编译开启OneDNN + Eigen Threadpool工作线程池版本(CPU)**

```bash
bazel build -c opt --config=opt --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
```

**编译开启OneDNN + Eigen Threadpool工作线程池版本+ABI=0的版本 (CPU)**

```bash
bazel build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" --host_cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0" -c opt --config=opt --config=mkl_threadpool --define build_with_mkl_dnn_v1_only=true //tensorflow/tools/pip_package:build_pip_package
```

## 生成Whl包

```bash
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
```

## 安装Whl包

```bash
pip3 install /tmp/tensorflow_pkg/tensorflow-1.15.5+${version}-cp36-cp36m-linux_x86_64.whl
```

## Nightly 镜像

**GPU CUDA11.0镜像**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec-nightly-gpu-py36-cu110-ubuntu18.04
```

**CPU镜像**

```
registry.cn-shanghai.aliyuncs.com/pai-dlc-share/deeprec-training:deeprec-nightly-cpu-py36-ubuntu18.04
```

## Estimator编译&打包

由于DeepRec新增了分布式grpc++、star_server等protocol,在使用DeepRec配合原生Estimator会存在像grpc++, star_server功能使用时无法通过Estimator检查的问题,因为我们提供了针对DeepRec版本的Estimator,对应代码库:[https://github.com/AlibabaPAI/estimator](https://github.com/AlibabaPAI/estimator)

**代码编译**

```bash
bazel build //tensorflow_estimator/tools/pip_package:build_pip_package
```

**生成Wheel包**

```bash
bazel-bin/tensorflow_estimator/tools/pip_package/build_pip_package /tmp/estimator_pip
```

**安装**

安装DeepRec会默认安装原生的tensorflow-estimator的版本,请重装新编译的tensorflow-estimator即可。
12 changes: 10 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,20 @@
DeepRec
</h1>

# Introduction
# 简介
稀疏模型,是指在模型结构中离散特征计算逻辑占比较高的一类深度学习模型的统称。离散特征通常表现为id、tag、文字、词组等算法不能直接处理的非数值化特征,其广泛应用于搜索、广告、推荐等高价值业务中。当下主流开源深度学习框架,对稀疏模型的支持不足。在稀疏功能的支持、训练性能存在着问题,制约了稀疏模型的探索和发展。

DeepRec(PAI-TF) 支持了淘宝搜索、猜你喜欢、定向、直通车等核心业务,支撑着千亿特征、万亿样本超大规模的稀疏训练。积累了核心的稀疏场景的功能及性能优化。针对稀疏模型在分布式、图优化、算子、Runtime等方面进行了深度的性能优化,同时提供了稀疏场景下特有的动态弹性特征,动态弹性维度,多Hash Embedding,自适应EmbeddingVariable、增量模型导出及加载等一系列功能。

# Contents
# 开始

```{toctree}
:maxdepth: 2
Compile-And-Install
```

# 功能

```{toctree}
:maxdepth: 2
Expand Down

0 comments on commit 03f0ad4

Please sign in to comment.