Permalink
Browse files

add link translation

  • Loading branch information...
1 parent 6b629c2 commit 9ff0301515f828f8e70e7ec70bdbc4bef7cf42be @tqchen tqchen committed Jul 29, 2015
Showing with 60 additions and 28 deletions.
  1. +2 −1 .gitignore
  2. +9 −17 doc/conf.py
  3. +10 −10 doc/guide.md
  4. +39 −0 doc/sphinx_util.py
View
@@ -35,4 +35,5 @@
*.rabit
*.mock
dmlc-core
-recommonmark
+recommonmark
+recom
View
@@ -14,7 +14,7 @@
import sys
import os, subprocess
import shlex
-import recommonmark
+from sphinx_util import MarkdownParser
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -28,7 +28,13 @@
project = u'rabit'
copyright = u'2015, rabit developers'
author = u'rabit developers'
+github_doc_root = 'https://github.com/dmlc/rabit/tree/master/doc/'
+# add markdown parser
+MarkdownParser.github_doc_root = github_doc_root
+source_parsers = {
+ '.md': MarkdownParser,
+}
# Version information.
import rabit
version = rabit.__version__
@@ -147,23 +153,9 @@ def generate_doxygen_xml(app):
if read_the_docs_build:
run_doxygen('..')
-# add markdown parser
-source_parsers = {
- '.md': None,
-}
-
-def setup_path(app):
- read_the_docs_build = os.environ.get('READTHEDOCS', None) == 'True'
- if read_the_docs_build:
- subprocess.call('cd ..; git clone https://github.com/tqchen/recommonmark recommonmark-customized;' +
- 'ln -s recommonmark-customized/recommonmark recom', shell=True)
- sys.path.insert(0, os.path.abspath('..'))
- from recom import parser
- global source_parsers
- source_parsers['.md'] = parser.CommonMarkParser
-
def setup(app):
# Add hook for building doxygen xml when needed
app.connect("builder-inited", generate_doxygen_xml)
- setup_path(app)
+
+
View
@@ -21,7 +21,7 @@ To run the examples locally, you will need to build them with ```make```.
What is Allreduce
-----------------
The main methods provided by rabit are Allreduce and Broadcast. Allreduce performs reduction across different computation nodes,
-and returns the result to every node. To understand the behavior of the function, consider the following example in [basic.cc](https://github.com/dmlc/rabit/blob/master/guide/basic.cc) (there is a python example right after this if you are more familiar with python).
+and returns the result to every node. To understand the behavior of the function, consider the following example in [basic.cc](../guide/basic.cc) (there is a python example right after this if you are more familiar with python).
```c++
#include <rabit.h>
using namespace rabit;
@@ -88,7 +88,7 @@ You can run the program using the following command
```
Broadcast is another method provided by rabit besides Allreduce. This function allows one node to broadcast its
-local data to all other nodes. The following code in [broadcast.cc](https://github.com/dmlc/rabit/blob/master/guide/broadcast.cc) broadcasts a string from
+local data to all other nodes. The following code in [broadcast.cc](../guide/broadcast.cc) broadcasts a string from
node 0 to all other nodes.
```c++
#include <rabit.h>
@@ -114,7 +114,7 @@ The following command starts the program with three worker processes.
```
Besides strings, rabit also allows to broadcast constant size array and vectors.
-The counterpart in python can be found in [broadcast.py](https://github.com/dmlc/rabit/blob/master/guide/broadcast.py). Here is a snippet so that you can get a better sense of how simple is to use the python library:
+The counterpart in python can be found in [broadcast.py](../guide/broadcast.py). Here is a snippet so that you can get a better sense of how simple is to use the python library:
```python
import rabit
@@ -201,8 +201,8 @@ into the data buffer, pass the data to Allreduce function, and get the reduced r
from failure, we can directly recover the result from other nodes(see also [Fault Tolerance](#fault-tolerance)) and
the data preparation procedure no longer necessary. Rabit Allreduce add an optional parameter preparation function
to support such scenario. User can pass in a function that corresponds to the data preparation procedure to Allreduce
-calls, and the data preparation function will only be called when necessary. We use [lazy_allreduce.cc](https://github.com/dmlc/rabit/blob/master/guide/lazy_allreduce.cc)
-as an example to demonstrate this feature. It is modified from [basic.cc](https://github.com/dmlc/rabit/blob/master/guide/basic.cc), and you can compare the two codes.
+calls, and the data preparation function will only be called when necessary. We use [lazy_allreduce.cc](../guide/lazy_allreduce.cc)
+as an example to demonstrate this feature. It is modified from [basic.cc](../guide/basic.cc), and you can compare the two codes.
```c++
#include <rabit.h>
using namespace rabit;
@@ -241,15 +241,15 @@ the effect when a process goes down. You can run the program using the following
The additional arguments ```mock=0,0,1,0``` will cause node 0 to kill itself before second call of Allreduce (see also [mock test](#link-against-mock-test-rabit-library)).
You will find that the prepare function's print is only executed once and node 0 will no longer execute the preparation function when it restarts from failure.
-You can also find python version of the example in [lazy_allreduce.py](https://github.com/dmlc/rabit/blob/master/guide/lazy_allreduce.py), and run it using the followin command
+You can also find python version of the example in [lazy_allreduce.py](../guide/lazy_allreduce.py), and run it using the followin command
```bash
../tracker/rabit_demo.py -n 2 lazy_allreduce.py mock=0,0,1,0
```
Since lazy preparation function may not be called during execution. User should be careful when using this feature. For example, a possible mistake
could be putting some memory allocation code in the lazy preparation function, and the computing memory was not allocated when lazy preparation function is not called.
-The example in [lazy_allreduce.cc](https://github.com/dmlc/rabit/blob/master/guide/lazy_allreduce.cc) provides a simple way to migrate normal prepration code([basic.cc](https://github.com/dmlc/rabit/blob/master/guide/basic.cc)) to lazy version: wrap the preparation
+The example in [lazy_allreduce.cc](../guide/lazy_allreduce.cc) provides a simple way to migrate normal prepration code([basic.cc](../guide/basic.cc)) to lazy version: wrap the preparation
code with a lambda function, and pass it to allreduce.
#### Checkpoint and LazyCheckpoint
@@ -267,7 +267,7 @@ Because the different nature of the two types of models, different strategy will
nodes (selected using a ring replication strategy). The checkpoint is only saved in the memory without touching the disk which makes rabit programs more efficient.
User is encouraged to use ```global_model``` only when is sufficient for better efficiency.
-To enable a model class to be checked pointed, user can implement a [serialization interface](https://github.com/dmlc/rabit/blob/master/include/rabit_serialization.h). The serialization interface already
+To enable a model class to be checked pointed, user can implement a [serialization interface](../include/rabit_serialization.h). The serialization interface already
provide serialization functions of STL vector and string. For python API, user can checkpoint any python object that can be pickled.
There is a special Checkpoint function called [LazyCheckpoint](http://homes.cs.washington.edu/~tqchen/rabit/doc/namespacerabit.html#a99f74c357afa5fba2c80cc0363e4e459),
@@ -288,10 +288,10 @@ improve the efficiency of the program.
Compile Programs with Rabit
---------------------------
Rabit is a portable library, to use it, you only need to include the rabit header file.
-* You will need to add the path to [../include](https://github.com/dmlc/rabit/blob/master/include) to the header search path of the compiler
+* You will need to add the path to [../include](../include) to the header search path of the compiler
- Solution 1: add ```-I/path/to/rabit/include``` to the compiler flag in gcc or clang
- Solution 2: add the path to the environment variable CPLUS_INCLUDE_PATH
-* You will need to add the path to [../lib](https://github.com/dmlc/rabit/blob/master/lib) to the library search path of the compiler
+* You will need to add the path to [../lib](../lib) to the library search path of the compiler
- Solution 1: add ```-L/path/to/rabit/lib``` to the linker flag
- Solution 2: add the path to environment variable LIBRARY_PATH AND LD_LIBRARY_PATH
* Link against lib/rabit.a
View
@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+"""Helper utilty function for customization."""
+import sys
+import os
+import subprocess
+
+if os.environ.get('READTHEDOCS', None) == 'True':
+ subprocess.call('cd ..; git clone https://github.com/tqchen/recommonmark recommonmark-customized;' +
+ 'ln -s recommonmark-customized/recommonmark recom', shell=True)
+
+sys.path.insert(0, os.path.abspath('..'))
+from recom import parser
+
+class MarkdownParser(parser.CommonMarkParser):
+ github_doc_root = None
+
+ @staticmethod
+ def remap_url(url):
+ if MarkdownParser.github_doc_root is None or url is None:
+ return url
+ arr = url.split('#', 1)
+ if arr[0].endswith('.md') and arr[0].find('://') == -1:
+ arr[0] = arr[0][:-3] + '.html'
+ return '#'.join(arr)
+ else:
+ return MarkdownParser.github_doc_root + url
+
+ def reference(self, block):
+ block.destination = remap_url(block.destination)
+ return super(MarkdownParser, self).reference(block)
+
+# inplace modify the function in recommonmark module to allow link remap
+old_ref = parser.reference
+
+def reference(block):
+ block.destination = MarkdownParser.remap_url(block.destination)
+ return old_ref(block)
+
+parser.reference = reference

0 comments on commit 9ff0301

Please sign in to comment.