[MXNET-641] fix R windows install docs #11805

ankkhedia · 2018-07-18T21:25:08Z

Description

This PR fixes the install instructions for R in Windows. It will fix
#8936 #8927 #9952

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

Changes

It fixed documentation for MXNet-R installation on Windows

Comments

The changes are complete , however the binaries are yet to be put in the bucket for GPU prebuild binaries instructions to work

ankkhedia · 2018-07-18T21:26:15Z

@aaronmarkham @hetong007 @anirudhacharya Could you please review the changes in README

anirudhacharya · 2018-07-18T21:37:28Z

docs/install/windows_setup.md

+2. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu_cuX.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version
+3. Create a folder called ```R-package/inst/libs/x64```. MXNet supports only 64-bit operating systems, so you need the x64 folder.
+4. Copy the following shared libraries (.dll files) into the ```R-package/inst/libs/x64``` folder:
+```


why were these removed -
-cublas64_80.dll
-cudart64_80.dll
-cudnn64_5.dll
-curand64_80.dll

These don't need to be there in the R package libraries,it is taken from cuda toolkit installation and not here

lupesko

Looks good. A few small changes requested.

lupesko · 2018-07-23T08:11:10Z

docs/install/windows_setup.md

-  options(repos = cran)
-  install.packages("mxnet")
+```sh
+git clone --recursive https://github.com/dmlc/mxnet


"https://github.com/dmlc/mxnet" is out of date. Please use "https://github.com/apache/incubator-mxnet"

lupesko · 2018-07-23T08:14:00Z

docs/install/windows_setup.md


-To install MXNet R package on a computer with a GPU processor, you need the following:
+* Install Microsoft Visual Studio 2017 (required by CUDA)


Maybe add link? https://visualstudio.microsoft.com/downloads/

Thanks! will add that

lupesko · 2018-07-23T08:14:38Z

docs/install/windows_setup.md


-* The MXNet package
+Note: A pre-requisite to above softwares is [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) which we assume is installed.


Why not list as a regular dependency and not a Note?

Usually it comes pre-installed with Windows system, however this needs to be an additional step in Windows Server images on ec2. So, added as note

Again it might be very helpful to identify a version number. I've had issues with this before and it took a lot of digging to figure out that I needed a driver upgrade.

lupesko · 2018-07-23T08:15:24Z

docs/install/windows_setup.md

+Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently, MXNet supports these versions of CUDA.
+#### Building MXNet from Source Code(GPU)
+After you have installed above softwares 
+1. Clone the MXNet github repo.

 ```sh
 git clone --recursive https://github.com/dmlc/mxnet


Same comment as above, please update the link to point at the new GitHub repo URL

will fix that

Pulling changes

aaronmarkham

Some clarifications would be good to add.

aaronmarkham · 2018-07-24T16:08:04Z

docs/install/windows_setup.md


-To install MXNet R package on a computer with a GPU processor, you need the following:
+* Install [Microsoft Visual Studio 2017](https://visualstudio.microsoft.com/downloads/) (required by CUDA)


Several other MXNet-related instructions ask for 2015 (MKL-DNN for example). I think we should have both here and identify any nuance in the setups

aaronmarkham · 2018-07-24T16:09:05Z

docs/install/windows_setup.md


-* The NVidia CUDA Toolkit
+* Download and install [CuDNN 7](https://developer.nvidia.com/cudnn) (to provide a Deep Neural Network library)


Can we specify a minor version, or at least say cuDNN > 7.0?

suggested to use latest version

aaronmarkham · 2018-07-24T16:10:26Z

docs/install/windows_setup.md


-* The MXNet package
+Note: A pre-requisite to above softwares is [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) which we assume is installed.


Again it might be very helpful to identify a version number. I've had issues with this before and it took a lot of digging to figure out that I needed a driver upgrade.

aaronmarkham · 2018-07-24T16:11:01Z

docs/install/windows_setup.md

-* CuDNN (to provide a Deep Neural Network library)
+#### Installing MXNet with the Prebuilt Binary Package(GPU)
+For Windows users, MXNet provides prebuilt binary packages.
+You can install the package directly in the R console after you have the above softwares installed


software (it's both singular and plural)

Also add a period at the end.

Thanks for pointing out. Taken care

aaronmarkham · 2018-07-24T16:11:55Z

docs/install/windows_setup.md

+  options(repos = cran)
+  install.packages("mxnet")
+```
+Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently, MXNet supports these versions of CUDA.


I believe there's some issue with 9.1 and it is not recommended.

How does one check their current version #? If you are installing everything for the first time, maybe the recommendation should be made for the user to install 9.2?

Suggested 9.2 for first time installation

aaronmarkham · 2018-07-24T16:13:10Z

docs/install/windows_setup.md

+```
+Change X to 80,90,91 or 92 based on your CUDA toolkit version. Currently, MXNet supports these versions of CUDA.
+#### Building MXNet from Source Code(GPU)
+After you have installed above softwares 


software, continue with the following steps to build MXNet:

aaronmarkham · 2018-07-24T16:15:33Z

docs/install/windows_setup.md

-cudart64_80.dll
-cudnn64_5.dll
-curand64_80.dll
+2. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu_cuX.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version


This seems odd. Aren't these are binaries for VS2015 (vc14), but the prerequisite was for VS2017? Is there a binary tagged with vc15 available?

added that both 2015 and 2017 are supported

aaronmarkham · 2018-07-24T16:17:01Z

docs/install/windows_setup.md

 ```
-9. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
-10. Now open the Windows CMD and change the directory to the `mxnet` folder. Then use the following commands
+6. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.


Make sure that the R executable is....

aaronmarkham · 2018-07-24T16:17:32Z

docs/install/windows_setup.md

-9. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
-10. Now open the Windows CMD and change the directory to the `mxnet` folder. Then use the following commands
+6. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
+7. Also make sure that Rtools in installed and added to your ```PATH``` in the environment variables.


Rtools executable (or folder path if there's more than one in there...)

aaronmarkham · 2018-07-24T16:18:28Z

docs/install/windows_setup.md

-10. Now open the Windows CMD and change the directory to the `mxnet` folder. Then use the following commands
+6. Make sure that R is added to your ```PATH``` in the environment variables. Running the ```where R``` command at the command prompt should return the location.
+7. Also make sure that Rtools in installed and added to your ```PATH``` in the environment variables.
+8. Temporary patch - im2rec currently results in crashes during the build. Remove the im2rec.h and im2rec.cc files in R-package/src/ from cloned repository and comment out the two im2rec lines in R-package/src/mxnet.cc.


You might want to show the two lines. These seems kind of vague and easy to mess up.

aaronmarkham

Some markdown rendering issues with the numbering and one sentence update.

aaronmarkham · 2018-07-24T21:03:34Z

docs/install/windows_setup.md

@@ -114,81 +114,166 @@ For CPU-only package:
  install.packages("mxnet")
 ```

-For GPU-enabled package:
+#### Building MXNet from Source Code(CPU)
+1. Clone the MXNet github repo.


View the markdown output. This numbering is wrapping instead of using new lines.

aaronmarkham · 2018-07-24T21:05:15Z

docs/install/windows_setup.md


-* Microsoft Visual Studio 2013
+However, few dependencies remains same for both options.  You will need the following:


a few dependencies remain for both options.

aaronmarkham · 2018-07-24T21:06:08Z

docs/install/windows_setup.md

-cudart64_80.dll
-cudnn64_5.dll
-curand64_80.dll
+2. Download prebuilt GPU-enabled MXNet libraries for Windows from https://github.com/yajiedesign/mxnet/releases. You will need `mxnet_x64_vc14_gpu_cuX.7z` and `prebuildbase_win10_x64_vc14.7z` where X stands for your CUDA toolkit version


The numbering is wrapping here too.

ankkhedia · 2018-07-24T21:44:16Z

@aaronmarkham I have addressed all of the comments. Could you take a look?

aaronmarkham

LGTM.

* adding param for list of tags to display on website * using new website display argument for artifact placement in version folder * adding display logic * remove restricted setting for testing * update usage instructions * reverted Jenkinsfile to use restricted nodes [MXAPPS-581] Fixes for broken Straight Dope tests. (apache#11923) * Update relative paths pointing to the data directory to point to the correct place in the testing temporary folder. * Enable the notebooks that were previously broken because of relative file paths not pointing to the correct place. * Move some notebooks we do not plan to test to the whitelist. These notebooks are not published in the Straight Dope book. * Clean-up: Convert print statements to info/warn/error logging statements. Add some logging statements for better status. Disable flaky test: test_spatial_transformer_with_type (apache#11930) apache#11839 Add linux and macos MKLDNN Building Instruction (apache#11049) * add linux and macos doc * update doc * Update MKL_README.md * Update MKL_README.md Add convolution code to verify mkldnn backend * add homebrew link * rename to MKLDNN_README * add mkl verify * trigger * trigger * set mac complier to gcc47 * add VS2017 support experimentally * improve quality * improve quality * modify mac build instruction since prepare_mkldnn.sh has been rm * trigger * add some improvement [MXNET-531] Add download util (apache#11866) * add changes to example * place the file to the util * add retry scheme * fix the retry logic * change the DownloadUtil to Util * Trigger the CI [MXNET-11241] Avoid use of troublesome cudnnFind() results when grad_req='add' (apache#11338) * Add tests that fail due to issue 11241 * Fix apache#11241 Conv1D throws CUDNN_STATUS_EXECUTION_FAILED * Force algo 1 when grad_req==add with large c. Expand tests. * Shorten test runtimes. Improving documentation and error messages for Async distributed training with Gluon (apache#11910) * Add description about update on kvstore * add async check for gluon * only raise error if user set update_on_kvstore * fix condition * add async nightly test * fix case when no kvstore * add example for trainer creation in doc [MXNET-641] fix R windows install docs (apache#11805) * fix R windows install docs * addressed PR comments * PR comments * PR comments * fixed line wrappings * fixed line wrappings a hot fix for mkldnn link (apache#11939) re-enabling randomized test_l2_normalization (apache#11900) [MXNET-651] MXNet Model Backwards Compatibility Checker (apache#11626) * Added MNIST-MLP-Module-API models to check model save and load_checkpoint methods * Added LENET with Conv2D operator training file * Added LENET with Conv2d operator inference file * Added LanguageModelling with RNN training file * Added LamguageModelling with RNN inference file * Added hybridized LENET Gluon Model training file * Added hybridized LENET gluon model inference file * Added license headers * Refactored the model and inference files and extracted out duplicate code in a common file * Added runtime function for executing the MBCC files * Added JenkinsFile for MBCC to be run as a nightly job * Added boto3 install for s3 uploads * Added README for MBCC * Added license header * Added more common functions from lm_rnn_gluon_train and inference files into common.py to clean up code * Added scripts for training models on older versions of MXNet * Added check for preventing inference script from crashing in case no trained models are found * Fixed indentation issue * Replaced Penn Tree Bank Dataset with Sherlock Holmes Dataset * Fixed indentation issue * Removed training in models and added smaller models. Now we are simply checking a forward pass in the model with dummy data. * Updated README * Fixed indentation error * Fixed indentation error * Removed code duplication in the training file * Added comments for runtime_functions script for training files * Merged S3 Buckets for storing data and models into one * Automated the process to fetch MXNet versions from git tags * Added defensive checks for the case where the data might not be found * Fixed issue where we were performing inference on state model files * Replaced print statements with logging ones * Removed boto install statements and move them into ubuntu_python docker * Separated training and uploading of models into separate files so that training runs in Docker and upload runs outside Docker * Fixed pylint warnings * Updated comments and README * Removed the venv for training process * Fixed indentation in the MBCC Jenkins file and also separated out training and inference into two separate stages * Fixed indendation * Fixed erroneous single quote * Added --user flag to check for Jenkins error * Removed unused methods * Added force flag in the pip command to install mxnet * Removed the force-re-install flag * Changed exit 1 to exit 0 * Added quotes around the shell command * added packlibs and unpack libs for MXNet builds * Changed PythonPath from relative to absolute * Created dedicated bucket with correct permission * Fix for python path in training * Changed bucket name to CI bucket * Added set -ex to the upload shell script * Now raising an exception if no models are found in the S3 bucket * Added regex to train models script * Added check for performing inference only on models trained on same major versions * Added set -ex flags to shell scripts * Added multi-version regex checks in training * Fixed typo in regex * Now we will train models for all the minor versions for a given major version by traversing the tags * Added check for validating current_version [MXNET-531] NeuralStyle Example for Scala (apache#11621) * add initial neuralstyle and test coverage * Add two more test and README * kill comments * patch on memory leaks fix * fix formatting issues * remove redundant files * disable the Gan example for now * add ignore method * add new download scheme to match the changes

* fix R windows install docs * addressed PR comments * PR comments * PR comments * fixed line wrappings * fixed line wrappings

fix R windows install docs

844124c

ankkhedia requested a review from szha as a code owner July 18, 2018 21:25

anirudhacharya reviewed Jul 18, 2018

View reviewed changes

lupesko suggested changes Jul 23, 2018

View reviewed changes

ankkhedia added 2 commits July 23, 2018 12:58

Merge remote-tracking branch 'upstream/master'

d8dbb98

Pulling changes

addressed PR comments

5f66a10

aaronmarkham suggested changes Jul 24, 2018

View reviewed changes

ankkhedia added 2 commits July 24, 2018 13:16

PR comments

956a4e4

PR comments

31f7d5e

aaronmarkham suggested changes Jul 24, 2018

View reviewed changes

ankkhedia added 2 commits July 24, 2018 14:39

fixed line wrappings

dad0712

fixed line wrappings

40a1098

aaronmarkham approved these changes Jul 24, 2018

View reviewed changes

hetong007 approved these changes Jul 30, 2018

View reviewed changes

hetong007 merged commit 815f42d into apache:master Jul 30, 2018

XinYao1994 pushed a commit to XinYao1994/incubator-mxnet that referenced this pull request Aug 29, 2018

[MXNET-641] fix R windows install docs (apache#11805)

348e92d

* fix R windows install docs * addressed PR comments * PR comments * PR comments * fixed line wrappings * fixed line wrappings


		To install MXNet R package on a computer with a GPU processor, you need the following:
		* Install Microsoft Visual Studio 2017 (required by CUDA)


		* The MXNet package
		Note: A pre-requisite to above softwares is [Nvidia-drivers](http://www.nvidia.com/Download/index.aspx?lang=en-us) which we assume is installed.


		To install MXNet R package on a computer with a GPU processor, you need the following:
		* Install [Microsoft Visual Studio 2017](https://visualstudio.microsoft.com/downloads/) (required by CUDA)


		* The NVidia CUDA Toolkit
		* Download and install [CuDNN 7](https://developer.nvidia.com/cudnn) (to provide a Deep Neural Network library)


		* Microsoft Visual Studio 2013
		However, few dependencies remains same for both options. You will need the following:

[MXNET-641] fix R windows install docs #11805

[MXNET-641] fix R windows install docs #11805

Conversation

ankkhedia commented Jul 18, 2018 • edited Loading

Description

Checklist

Essentials

Changes

Comments

ankkhedia commented Jul 18, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lupesko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronmarkham left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ankkhedia commented Jul 24, 2018

aaronmarkham left a comment

Choose a reason for hiding this comment

ankkhedia commented Jul 18, 2018 •

edited

Loading

ankkhedia commented Jul 18, 2018 •

edited

Loading