Windows #5676
Open
Commits
Show all changes
96 commits
Select commit
Hold shift + click to select a range
52f7402
Add support for windows build
pavlejosipovic f2f0a0d
make branch README for Windows port
shelhamer bed9df0
Update README.md
pavlejosipovic 14f17f9
support signal handler
happynear 55f3cfd
Tweak default build options
pavlejosipovic 90fcdc0
Merge master@{2016-03-07} into windows
sasagalic-MSFT cf7c0a4
Fix issues after the latest merge
sasagalic-MSFT 3372cef
Add support for matcaffe build in Windows
lunzueta a67623e
Merge bvlc/master into bvlc/windows
sasagalic-MSFT f043ecb
Fix readme file
mnidza adc4f0c
Update HDF5 and LMDB packages
pavlejosipovic ebcd91a
Fix several issues with spaces in paths
sasagalic-MSFT 2fb4711
Update HDF5 NuGet package
pavlejosipovic 34e8e6e
Remove optimization in Appveyor build
sasagalic-MSFT 9eb541b
Support windows build for rest of the tools
sasagalic-MSFT 4b8fa07
Merge master@{2016-04-04} into windows
sasagalic-MSFT ad52ef5
Merge master@{2016-05-09} into windows
sasagalic-MSFT cc46e08
Fix issues after the latest merge
sasagalic-MSFT a473c77
Add nuget sources
sasagalic-MSFT 249184c
Define preprocessing macro to properly report version on caffe comman…
IsaacYangSLA d0bd302
Merge pull request #4200 from IsaacYangSLA/dev/caffe_version
shelhamer 714d0ac
Remove Python.h include since Boost Python does it and also avoid add…
mlamarre 5db3074
Merge pull request #4418 from VLAM3D/windows
longjon 2794c97
Merge cmake changes into latest windows branch
willyd 630cbc1
Updated appveyor badge for windows branch
willyd dbd5101
Merge latest master into windows
willyd 74f3bf7
Merge remote-tracking branch 'origin/pytest-fix' into windows-merge-m…
willyd 1e8db6c
Merged master branch 2016/10/18
willyd e90123e
Make download prebuilt dependencies compatible with python 3.5
willyd 6f451f5
Added CI for VS 2015 and workaround for VS 2015 bug
willyd f19a8a9
Updated README
willyd c17dda5
Merge pull request #4939 from willyd/update-readme
willyd 5f29ad4
Clarified CUDNN install instructions
elhigu 87fbed1
Merge pull request #4955 from elhigu/patch-1
willyd 75149ee
Merge branch master into windows
willyd de3fb13
Merge pull request #4963 from willyd/windows-update
willyd 187c444
Merge pull request #4865 from willyd/python3-fix
willyd a77e8d8
Added build script and updated README.md accordingly.
willyd a9a7698
Merge pull request #5039 from willyd/build-script
willyd fcb70ff
Merge branch 'master' into windows
willyd a61b7b9
fix typo when including cmake files of opencv
twmht 58479e9
Merge pull request #5047 from twmht/windows
willyd ea72266
Merge pull request #5042 from willyd/windows
willyd 4ef2c3e
Added artifacts to appveyor builds
willyd 3d1f43d
Merge pull request #5049 from willyd/artifacts
willyd 3649015
Added missing pyyaml in appveyor build script
willyd a9ace02
Merge pull request #5161 from willyd/pyyaml
willyd 38862a5
Fixed appveyor build status badge and added prebuilt binaries downloa…
willyd 2da94c4
Merge pull request #5162 from willyd/cibadge
willyd e4f3702
Merge branch 'master' into windows
willyd 58563ab
Update CMake files to build Matcaffe on Windows
lunzueta 8bb7cbc
Merge pull request #5180 from willyd/matlab-support
willyd 86425b1
Merge branch 'master' into windows
willyd 82c087c
Added support for python 3.5
willyd e483c7b
Merge pull request #5088 from willyd/python35
willyd f4db952
Fixed AppVeyor artifacts download links
willyd 372e920
Added CI with CUDA on Windows and disabled some Debug builds on AppVeyor
willyd 655be9f
Merge pull request #4895 from willyd/cuda-ci
willyd a3f1cce
Fix the following error when running the script without defining "WIT…
forderud 382ae79
Merge pull request #5204 from forderud/cuda-define
willyd 8a49d45
Fix broken download links for CUDA enabled AppVeyor builds
willyd 46b3d46
Make it possible to change build configuration on Windows without hav…
forderud a7cc718
Merge pull request #5206 from forderud/win-build-cfg
willyd d14012e
Removed unnecessary 1 with SHOW_PROGRESS option and fixed error messa…
willyd ac59d48
Merge branch 'master' into windows
willyd 2106a62
Added nccl ExternalProject to build nccl on Windows
willyd 882bc3d
Merge pull request #5231 from willyd/windows-nccl
willyd 7c1daec
Added *.lib/*.dll/*.pyd/*.exe to .gitignore
willyd ebc54c0
Added powershell scripts to mimic the .sh script to download and crea…
willyd 8024d67
Added copy of required dependencies to examples executables
willyd 6900c4b
Merge pull request #5287 from willyd/windows-ignore
willyd cc0614d
Merge pull request #5288 from willyd/windows-mnist
willyd 527f97c
Updated prebuilt dependencies. Fixes #5348
willyd d8c6e87
Merge pull request #5355 from willyd/lmdb-fix
willyd a35cc49
Added train_lenet.ps1, fixed create_mnist.ps1 and update mnist readme…
willyd 2a35878
Merge branch 'master' into windows
willyd ca360a1
Fixed lint issue when .pb.h and .pb.cc files do not exist
willyd de0ee29
Fixed wrong VS 2013 Update 5 version string. Fixes #5430
willyd 5d3dced
Merge pull request #5441 from willyd/vs2013-up5-fix
willyd 621c5c7
Removed deprecated VS based build.
willyd 751ff4d
Merge branch 'master' into windows
willyd d800357
Dependencies URLs updated for newer version
AhmedOS 88ddc95
Merge pull request #5460 from AhmedOS/patch-1
willyd a166218
Merge branch 'master' into windows
willyd a48e81f
Fixed issue with missing pydot and graphviz under windows
willyd 90e8abc
Merge pull request #5501 from willyd/pydot-fix
willyd 6280643
Added GPU arch option.
PENGUINLIONG 37ff78a
Merge pull request #5553 from PENGUINLIONG/dev-penguinliong
willyd 1f75c51
Merge branch 'master' into windows
willyd af191e2
Merge branch 'master' into windows
willyd c46b910
Added missing dependency between libcaffe and nccl
willyd ea0d92d
Added support for resolving MATLAB prerequisites. Also fixes a bug wi…
willyd 5f92dbd
Merge pull request #5590 from willyd/matlab-prerequisites
willyd 77624b5
Merge pull request #5389 from willyd/mnist-fix
willyd 35f3fcf
Merge branch 'master' into windows
willyd 6bfc5ca
Merge branch 'master' into windows
willyd
Jump to file or symbol
Failed to load files and symbols.
| @@ -0,0 +1,63 @@ | ||
| +############################################################################### | ||
| +# Set default behavior to automatically normalize line endings. | ||
| +############################################################################### | ||
| +* text=auto | ||
| + | ||
| +############################################################################### | ||
| +# Set default behavior for command prompt diff. | ||
| +# | ||
| +# This is need for earlier builds of msysgit that does not have it on by | ||
| +# default for csharp files. | ||
| +# Note: This is only used by command line | ||
| +############################################################################### | ||
| +#*.cs diff=csharp | ||
| + | ||
| +############################################################################### | ||
| +# Set the merge driver for project and solution files | ||
| +# | ||
| +# Merging from the command prompt will add diff markers to the files if there | ||
| +# are conflicts (Merging from VS is not affected by the settings below, in VS | ||
| +# the diff markers are never inserted). Diff markers may cause the following | ||
| +# file extensions to fail to load in VS. An alternative would be to treat | ||
| +# these files as binary and thus will always conflict and require user | ||
| +# intervention with every merge. To do so, just uncomment the entries below | ||
| +############################################################################### | ||
| +#*.sln merge=binary | ||
| +#*.csproj merge=binary | ||
| +#*.vbproj merge=binary | ||
| +#*.vcxproj merge=binary | ||
| +#*.vcproj merge=binary | ||
| +#*.dbproj merge=binary | ||
| +#*.fsproj merge=binary | ||
| +#*.lsproj merge=binary | ||
| +#*.wixproj merge=binary | ||
| +#*.modelproj merge=binary | ||
| +#*.sqlproj merge=binary | ||
| +#*.wwaproj merge=binary | ||
| + | ||
| +############################################################################### | ||
| +# behavior for image files | ||
| +# | ||
| +# image files are treated as binary by default. | ||
| +############################################################################### | ||
| +#*.jpg binary | ||
| +#*.png binary | ||
| +#*.gif binary | ||
| + | ||
| +############################################################################### | ||
| +# diff behavior for common document formats | ||
| +# | ||
| +# Convert binary document formats to text before diffing them. This feature | ||
| +# is only available from the command line. Turn it on by uncommenting the | ||
| +# entries below. | ||
| +############################################################################### | ||
| +#*.doc diff=astextplain | ||
| +#*.DOC diff=astextplain | ||
| +#*.docx diff=astextplain | ||
| +#*.DOCX diff=astextplain | ||
| +#*.dot diff=astextplain | ||
| +#*.DOT diff=astextplain | ||
| +#*.pdf diff=astextplain | ||
| +#*.PDF diff=astextplain | ||
| +#*.rtf diff=astextplain | ||
| +#*.RTF diff=astextplain |
15
.gitignore
| @@ -1,37 +1,129 @@ | ||
| -# Caffe | ||
| +# Windows Caffe | ||
| -[](https://travis-ci.org/BVLC/caffe) | ||
| -[](LICENSE) | ||
| +**This is an experimental, communtity based branch led by Guillaume Dumont (@willyd). It is a work-in-progress.** | ||
| -Caffe is a deep learning framework made with expression, speed, and modularity in mind. | ||
| -It is developed by Berkeley AI Research ([BAIR](http://bair.berkeley.edu))/The Berkeley Vision and Learning Center (BVLC) and community contributors. | ||
| +This branch of Caffe ports the framework to Windows. | ||
| -Check out the [project site](http://caffe.berkeleyvision.org) for all the details like | ||
| +[](https://travis-ci.org/BVLC/caffe) Travis (Linux build) | ||
| -- [DIY Deep Learning for Vision with Caffe](https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.p) | ||
| -- [Tutorial Documentation](http://caffe.berkeleyvision.org/tutorial/) | ||
| -- [BAIR reference models](http://caffe.berkeleyvision.org/model_zoo.html) and the [community model zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo) | ||
| -- [Installation instructions](http://caffe.berkeleyvision.org/installation.html) | ||
| +[](https://ci.appveyor.com/project/BVLC/caffe/branch/windows) AppVeyor (Windows build) | ||
| -and step-by-step examples. | ||
| +## Prebuilt binaries | ||
| -[](https://gitter.im/BVLC/caffe?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) | ||
| +Prebuilt binaries can be downloaded from the latest CI build on appveyor for the following configurations: | ||
| -Please join the [caffe-users group](https://groups.google.com/forum/#!forum/caffe-users) or [gitter chat](https://gitter.im/BVLC/caffe) to ask questions and talk about methods and models. | ||
| -Framework development discussions and thorough bug reports are collected on [Issues](https://github.com/BVLC/caffe/issues). | ||
| +- Visual Studio 2015, CPU only, Python 3.5: [Caffe Release](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DRelease%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D3%2C%20WITH_CUDA%3D0), ~~[Caffe Debug](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DDebug%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D3%2C%20WITH_CUDA%3D0)~~ | ||
| -Happy brewing! | ||
| +- Visual Studio 2015, CUDA 8.0, Python 3.5: [Caffe Release](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D1%2C%20CMAKE_CONFIG%3DRelease%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D3%2C%20WITH_CUDA%3D1) | ||
| -## License and Citation | ||
| +- Visual Studio 2015, CPU only, Python 2.7: [Caffe Release](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DRelease%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D2%2C%20WITH_CUDA%3D0), [Caffe Debug](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DDebug%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D2%2C%20WITH_CUDA%3D0) | ||
| -Caffe is released under the [BSD 2-Clause license](https://github.com/BVLC/caffe/blob/master/LICENSE). | ||
| -The BAIR/BVLC reference models are released for unrestricted use. | ||
| +- Visual Studio 2015,CUDA 8.0, Python 2.7: [Caffe Release](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D14%2C%20WITH_NINJA%3D1%2C%20CMAKE_CONFIG%3DRelease%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D2%2C%20WITH_CUDA%3D1) | ||
| -Please cite Caffe in your publications if it helps your research: | ||
| +- Visual Studio 2013, CPU only, Python 2.7: [Caffe Release](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D12%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DRelease%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D2%2C%20WITH_CUDA%3D0), [Caffe Debug](https://ci.appveyor.com/api/projects/BVLC/caffe/artifacts/build/caffe.zip?branch=windows&job=Environment%3A%20MSVC_VERSION%3D12%2C%20WITH_NINJA%3D0%2C%20CMAKE_CONFIG%3DDebug%2C%20CMAKE_BUILD_SHARED_LIBS%3D0%2C%20PYTHON_VERSION%3D2%2C%20WITH_CUDA%3D0) | ||
| - @article{jia2014caffe, | ||
| - Author = {Jia, Yangqing and Shelhamer, Evan and Donahue, Jeff and Karayev, Sergey and Long, Jonathan and Girshick, Ross and Guadarrama, Sergio and Darrell, Trevor}, | ||
| - Journal = {arXiv preprint arXiv:1408.5093}, | ||
| - Title = {Caffe: Convolutional Architecture for Fast Feature Embedding}, | ||
| - Year = {2014} | ||
| - } | ||
| + | ||
| +## Windows Setup | ||
| + | ||
| +### Requirements | ||
| + | ||
| + - Visual Studio 2013 or 2015 | ||
| + - [CMake](https://cmake.org/) 3.4 or higher (Visual Studio and [Ninja](https://ninja-build.org/) generators are supported) | ||
| + | ||
| +### Optional Dependencies | ||
| + | ||
| + - Python for the pycaffe interface. Anaconda Python 2.7 or 3.5 x64 (or Miniconda) | ||
| + - Matlab for the matcaffe interface. | ||
| + - CUDA 7.5 or 8.0 (use CUDA 8 if using Visual Studio 2015) | ||
| + - cuDNN v5 | ||
| + | ||
| + We assume that `cmake.exe` and `python.exe` are on your `PATH`. | ||
| + | ||
| +### Configuring and Building Caffe | ||
| + | ||
| +The fastest method to get started with caffe on Windows is by executing the following commands in a `cmd` prompt (we use `C:\Projects` as a root folder for the remainder of the instructions): | ||
| +```cmd | ||
| +C:\Projects> git clone https://github.com/BVLC/caffe.git | ||
| +C:\Projects> cd caffe | ||
| +C:\Projects\caffe> git checkout windows | ||
| +:: Edit any of the options inside build_win.cmd to suit your needs | ||
| +C:\Projects\caffe> scripts\build_win.cmd | ||
| +``` | ||
| +The `build_win.cmd` script will download the dependencies, create the Visual Studio project files (or the ninja build files) and build the Release configuration. By default all the required DLLs will be copied (or hard linked when possible) next to the consuming binaries. If you wish to disable this option, you can by changing the command line option `-DCOPY_PREREQUISITES=0`. The prebuilt libraries also provide a `prependpath.bat` batch script that can temporarily modify your `PATH` envrionment variable to make the required DLLs available. | ||
| + | ||
| +Below is a more complete description of some of the steps involved in building caffe. | ||
| + | ||
| +### Install the caffe dependencies | ||
| + | ||
| +By default CMake will download and extract prebuilt dependencies for your compiler and python version. It will create a folder called `libraries` containing all the required dependencies inside your build folder. Alternatively you can build them yourself by following the instructions in the [caffe-builder](https://github.com/willyd/caffe-builder) [README](https://github.com/willyd/caffe-builder/blob/master/README.md). | ||
| + | ||
| +### Use cuDNN | ||
| + | ||
| +To use cuDNN the easiest way is to copy the content of the `cuda` folder into your CUDA toolkit installation directory. For example if you installed CUDA 8.0 and downloaded cudnn-8.0-windows10-x64-v5.1.zip you should copy the content of the `cuda` directory to `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0`. Alternatively, you can define the CUDNN_ROOT cache variable to point to where you unpacked the cuDNN files e.g. `C:/Projects/caffe/cudnn-8.0-windows10-x64-v5.1/cuda`. For example the command in [scripts/build_win.cmd](scripts/build_win.cmd) would become: | ||
| +``` | ||
| +cmake -G"!CMAKE_GENERATOR!" ^ | ||
| + -DBLAS=Open ^ | ||
| + -DCMAKE_BUILD_TYPE:STRING=%CMAKE_CONFIG% ^ | ||
| + -DBUILD_SHARED_LIBS:BOOL=%CMAKE_BUILD_SHARED_LIBS% ^ | ||
| + -DBUILD_python:BOOL=%BUILD_PYTHON% ^ | ||
| + -DBUILD_python_layer:BOOL=%BUILD_PYTHON_LAYER% ^ | ||
| + -DBUILD_matlab:BOOL=%BUILD_MATLAB% ^ | ||
| + -DCPU_ONLY:BOOL=%CPU_ONLY% ^ | ||
| + -DCUDNN_ROOT=C:/Projects/caffe/cudnn-8.0-windows10-x64-v5.1/cuda ^ | ||
| + -C "%cd%\libraries\caffe-builder-config.cmake" ^ | ||
| + "%~dp0\.." | ||
| +``` | ||
| + | ||
| +Alternatively, you can open `cmake-gui.exe` and set the variable from there and click `Generate`. | ||
| + | ||
| +### Building only for CPU | ||
| + | ||
| +If CUDA is not installed Caffe will default to a CPU_ONLY build. If you have CUDA installed but want a CPU only build you may use the CMake option `-DCPU_ONLY=1`. | ||
| + | ||
| +### Using the Python interface | ||
| + | ||
| +The recommended Python distribution is Anaconda or Miniconda. To successfully build the python interface you need to add the following conda channels: | ||
| +``` | ||
| +conda config --add channels conda-forge | ||
| +conda config --add channels willyd | ||
| +``` | ||
| +and install the following packages: | ||
| +``` | ||
| +conda install --yes cmake ninja numpy scipy protobuf==3.1.0 six scikit-image pyyaml pydotplus graphviz | ||
| +``` | ||
| +If Python is installed the default is to build the python interface and python layers. If you wish to disable the python layers or the python build use the CMake options `-DBUILD_python_layer=0` and `-DBUILD_python=0` respectively. In order to use the python interface you need to either add the `C:\Projects\caffe\python` folder to your python path of copy the `C:\Projects\caffe\python\caffe` folder to your `site_packages` folder. | ||
| + | ||
| +### Using the MATLAB interface | ||
| + | ||
| +Follow the above procedure and use `-DBUILD_matlab=ON`. Change your current directory in MATLAB to `C:\Projects\caffe\matlab` and run the following command to run the tests: | ||
| +``` | ||
| +>> caffe.run_tests() | ||
| +``` | ||
| +If all tests pass you can test if the classification_demo works as well. First, from `C:\Projects\caffe` run `python scripts\download_model_binary.py models\bvlc_reference_caffenet` to download the pre-trained caffemodel from the model zoo. Then change your MATLAB directory to `C:\Projects\caffe\matlab\demo` and run `classification_demo`. | ||
| + | ||
| +### Using the Ninja generator | ||
| + | ||
| +You can choose to use the Ninja generator instead of Visual Studio for faster builds. To do so, change the option `set WITH_NINJA=1` in the `build_win.cmd` script. To install Ninja you can download the executable from github or install it via conda: | ||
| +```cmd | ||
| +> conda config --add channels conda-forge | ||
| +> conda install ninja --yes | ||
| +``` | ||
| +When working with ninja you don't have the Visual Studio solutions as ninja is more akin to make. An alternative is to use [Visual Studio Code](https://code.visualstudio.com) with the CMake extensions and C++ extensions. | ||
| + | ||
| +### Building a shared library | ||
| + | ||
| +CMake can be used to build a shared library instead of the default static library. To do so follow the above procedure and use `-DBUILD_SHARED_LIBS=ON`. Please note however, that some tests (more specifically the solver related tests) will fail since both the test exectuable and caffe library do not share static objects contained in the protobuf library. | ||
| + | ||
| +### Troubleshooting | ||
| + | ||
| +Should you encounter any error please post the output of the above commands by redirecting the output to a file and open a topic on the [caffe-users list](https://groups.google.com/forum/#!forum/caffe-users) mailing list. | ||
| + | ||
| +## Known issues | ||
| + | ||
| +- The `GPUTimer` related test cases always fail on Windows. This seems to be a difference between UNIX and Windows. | ||
| +- Shared library (DLL) build will have failing tests. | ||
| +- Shared library build only works with the Ninja generator | ||
| + | ||
| +## Further Details | ||
| + | ||
| +Refer to the BVLC/caffe master branch README for all other details such as license, citation, and so on. |
Oops, something went wrong.