How can I make build_win.cmd point to python 3.5 rather than python 2.7?

[a52siddi]

I can access python 3.5 from the command line:

C:\Users\MYNAME\projects\caffe>python
Python 3.5.5 | packaged by conda-forge | (default, Jul 24 2018, 01:52:17) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

I followed all the steps in this link (https://github.com/BVLC/caffe/tree/windows) and set up build_win.cmd in the following way:

@echo off
@setlocal EnableDelayedExpansion

:: Default values
if DEFINED APPVEYOR (
    echo Setting Appveyor defaults
    if NOT DEFINED MSVC_VERSION set MSVC_VERSION=14
    if NOT DEFINED WITH_NINJA set WITH_NINJA=0
    if NOT DEFINED CPU_ONLY set CPU_ONLY=0
    if NOT DEFINED CUDA_ARCH_NAME set CUDA_ARCH_NAME=Auto
    if NOT DEFINED CMAKE_CONFIG set CMAKE_CONFIG=Release
    if NOT DEFINED USE_NCCL set USE_NCCL=0
    if NOT DEFINED CMAKE_BUILD_SHARED_LIBS set CMAKE_BUILD_SHARED_LIBS=0
    if NOT DEFINED PYTHON_VERSION set PYTHON_VERSION=3
    if NOT DEFINED BUILD_PYTHON set BUILD_PYTHON=1
    if NOT DEFINED BUILD_PYTHON_LAYER set BUILD_PYTHON_LAYER=1
    if NOT DEFINED BUILD_MATLAB set BUILD_MATLAB=0
    if NOT DEFINED PYTHON_EXE set PYTHON_EXE=python
    if NOT DEFINED RUN_TESTS set RUN_TESTS=1
    if NOT DEFINED RUN_LINT set RUN_LINT=1
    if NOT DEFINED RUN_INSTALL set RUN_INSTALL=1

    :: Set python 2.7 with conda as the default python
    if !PYTHON_VERSION! EQU 2 (
        set CONDA_ROOT=C:\Users\Abdullah Siddiqui\Anaconda3
    )
    :: Set python 3.5 with conda as the default python
    if !PYTHON_VERSION! EQU 3 (
        set CONDA_ROOT=C:\Users\Abdullah Siddiqui\Anaconda3
    )
    set PATH=!CONDA_ROOT!;!CONDA_ROOT!\Scripts;!CONDA_ROOT!\Library\bin;!PATH!

    :: Check that we have the right python version
    !PYTHON_EXE! --version
    :: Add the required channels
    conda config --add channels conda-forge
    conda config --add channels willyd
    :: Update conda
    conda update conda -y
    :: Download other required packages
    conda install --yes cmake ninja numpy scipy protobuf==3.1.0 six scikit-image pyyaml pydotplus graphviz

    if ERRORLEVEL 1  (
      echo ERROR: Conda update or install failed
      exit /b 1
    )

    :: Install cuda and disable tests if needed
    if !WITH_CUDA! == 1 (
        call %~dp0\appveyor\appveyor_install_cuda.cmd
        set CPU_ONLY=0
        set RUN_TESTS=0
        set USE_NCCL=1
    ) else (
        set CPU_ONLY=1
    )

    :: Disable the tests in debug config
    if "%CMAKE_CONFIG%" == "Debug" (
        echo Disabling tests on appveyor with config == %CMAKE_CONFIG%
        set RUN_TESTS=0
    )

    :: Disable linting with python 3 until we find why the script fails
    if !PYTHON_VERSION! EQU 3 (
        set RUN_LINT=0
    )

) else (
    :: Change the settings here to match your setup
    :: Change MSVC_VERSION to 12 to use VS 2013
    if NOT DEFINED MSVC_VERSION set MSVC_VERSION=14
    :: Change to 1 to use Ninja generator (builds much faster)
    if NOT DEFINED WITH_NINJA set WITH_NINJA=0
    :: Change to 1 to build caffe without CUDA support
    if NOT DEFINED CPU_ONLY set CPU_ONLY=0
    :: Change to generate CUDA code for one of the following GPU architectures
    :: [Fermi  Kepler  Maxwell  Pascal  All]
    if NOT DEFINED CUDA_ARCH_NAME set CUDA_ARCH_NAME=Auto
    :: Change to Debug to build Debug. This is only relevant for the Ninja generator the Visual Studio generator will generate both Debug and Release configs
    if NOT DEFINED CMAKE_CONFIG set CMAKE_CONFIG=Release
    :: Set to 1 to use NCCL
    if NOT DEFINED USE_NCCL set USE_NCCL=0
    :: Change to 1 to build a caffe.dll
    if NOT DEFINED CMAKE_BUILD_SHARED_LIBS set CMAKE_BUILD_SHARED_LIBS=0
    :: Change to 3 if using python 3.5 (only 2.7 and 3.5 are supported)
    if NOT DEFINED PYTHON_VERSION set PYTHON_VERSION=3
    :: Change these options for your needs.
    if NOT DEFINED BUILD_PYTHON set BUILD_PYTHON=1
    if NOT DEFINED BUILD_PYTHON_LAYER set BUILD_PYTHON_LAYER=1
    if NOT DEFINED BUILD_MATLAB set BUILD_MATLAB=0
    :: If python is on your path leave this alone
    if NOT DEFINED PYTHON_EXE set PYTHON_EXE=python
    :: Run the tests
    if NOT DEFINED RUN_TESTS set RUN_TESTS=0
    :: Run lint
    if NOT DEFINED RUN_LINT set RUN_LINT=0
    :: Build the install target
    if NOT DEFINED RUN_INSTALL set RUN_INSTALL=0
)

:: Set the appropriate CMake generator
:: Use the exclamation mark ! below to delay the
:: expansion of CMAKE_GENERATOR
if %WITH_NINJA% EQU 0 (
    if "%MSVC_VERSION%"=="14" (
        set CMAKE_GENERATOR=Visual Studio 14 2015 Win64
    )
    if "%MSVC_VERSION%"=="12" (
        set CMAKE_GENERATOR=Visual Studio 12 2013 Win64
    )
    if "!CMAKE_GENERATOR!"=="" (
        echo ERROR: Unsupported MSVC version
        exit /B 1
    )
) else (
    set CMAKE_GENERATOR=Ninja
)

echo INFO: ============================================================
echo INFO: Summary:
echo INFO: ============================================================
echo INFO: MSVC_VERSION               = !MSVC_VERSION!
echo INFO: WITH_NINJA                 = !WITH_NINJA!
echo INFO: CMAKE_GENERATOR            = "!CMAKE_GENERATOR!"
echo INFO: CPU_ONLY                   = !CPU_ONLY!
echo INFO: CUDA_ARCH_NAME             = !CUDA_ARCH_NAME!
echo INFO: CMAKE_CONFIG               = !CMAKE_CONFIG!
echo INFO: USE_NCCL                   = !USE_NCCL!
echo INFO: CMAKE_BUILD_SHARED_LIBS    = !CMAKE_BUILD_SHARED_LIBS!
echo INFO: PYTHON_VERSION             = !PYTHON_VERSION!
echo INFO: BUILD_PYTHON               = !BUILD_PYTHON!
echo INFO: BUILD_PYTHON_LAYER         = !BUILD_PYTHON_LAYER!
echo INFO: BUILD_MATLAB               = !BUILD_MATLAB!
echo INFO: PYTHON_EXE                 = "!PYTHON_EXE!"
echo INFO: RUN_TESTS                  = !RUN_TESTS!
echo INFO: RUN_LINT                   = !RUN_LINT!
echo INFO: RUN_INSTALL                = !RUN_INSTALL!
echo INFO: ============================================================

:: Build and exectute the tests
:: Do not run the tests with shared library
if !RUN_TESTS! EQU 1 (
    if %CMAKE_BUILD_SHARED_LIBS% EQU 1 (
        echo WARNING: Disabling tests with shared library build
        set RUN_TESTS=0
    )
)

if NOT EXIST build mkdir build
pushd build

:: Setup the environement for VS x64
set batch_file=!VS%MSVC_VERSION%0COMNTOOLS!..\..\VC\vcvarsall.bat
call "%batch_file%" amd64

:: Configure using cmake and using the caffe-builder dependencies
:: Add -DCUDNN_ROOT=C:/Projects/caffe/cudnn-8.0-windows10-x64-v5.1/cuda ^
:: below to use cuDNN
cmake -G"!CMAKE_GENERATOR!" ^
      -DBLAS=Open ^
      -DCMAKE_BUILD_TYPE:STRING=%CMAKE_CONFIG% ^
      -DBUILD_SHARED_LIBS:BOOL=%CMAKE_BUILD_SHARED_LIBS% ^
      -DBUILD_python:BOOL=%BUILD_PYTHON% ^
      -DBUILD_python_layer:BOOL=%BUILD_PYTHON_LAYER% ^
      -DBUILD_matlab:BOOL=%BUILD_MATLAB% ^
      -DCPU_ONLY:BOOL=%CPU_ONLY% ^
      -DCOPY_PREREQUISITES:BOOL=1 ^
      -DINSTALL_PREREQUISITES:BOOL=1 ^
      -DUSE_NCCL:BOOL=!USE_NCCL! ^
      -DCUDA_ARCH_NAME:STRING=%CUDA_ARCH_NAME% ^
      "%~dp0\.."

if ERRORLEVEL 1 (
  echo ERROR: Configure failed
  exit /b 1
)

:: Lint
if %RUN_LINT% EQU 1 (
    cmake --build . --target lint  --config %CMAKE_CONFIG%
)

if ERRORLEVEL 1 (
  echo ERROR: Lint failed
  exit /b 1
)

:: Build the library and tools
cmake --build . --config %CMAKE_CONFIG%

if ERRORLEVEL 1 (
  echo ERROR: Build failed
  exit /b 1
)

:: Build and exectute the tests
if !RUN_TESTS! EQU 1 (
    cmake --build . --target runtest --config %CMAKE_CONFIG%

    if ERRORLEVEL 1 (
        echo ERROR: Tests failed
        exit /b 1
    )

    if %BUILD_PYTHON% EQU 1 (
        if %BUILD_PYTHON_LAYER% EQU 1 (
            :: Run python tests only in Release build since
            :: the _caffe module is _caffe-d is debug
            if "%CMAKE_CONFIG%"=="Release" (
                :: Run the python tests
                cmake --build . --target pytest

                if ERRORLEVEL 1 (
                    echo ERROR: Python tests failed
                    exit /b 1
                )
            )
        )
    )
)

if %RUN_INSTALL% EQU 1 (
    cmake --build . --target install --config %CMAKE_CONFIG%
)

popd
@endlocal

It gives me the following output:

C:\Users\MYNAME\projects\caffe>scripts\build_win.cmd
The system cannot find the drive specified.
The system cannot find the drive specified.
INFO: ============================================================
INFO: Summary:
INFO: ============================================================
INFO: MSVC_VERSION               = 14
INFO: WITH_NINJA                 = 0
INFO: CMAKE_GENERATOR            = "Visual Studio 14 2015 Win64"
INFO: CPU_ONLY                   = 0
INFO: CUDA_ARCH_NAME             = Auto
INFO: CMAKE_CONFIG               = Release
INFO: USE_NCCL                   = 0
INFO: CMAKE_BUILD_SHARED_LIBS    = 0
INFO: PYTHON_VERSION             = 3
INFO: BUILD_PYTHON               = 1
INFO: BUILD_PYTHON_LAYER         = 1
INFO: BUILD_MATLAB               = 0
INFO: PYTHON_EXE                 = "python"
INFO: RUN_TESTS                  = 0
INFO: RUN_LINT                   = 0
INFO: RUN_INSTALL                = 0
INFO: ============================================================
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler using: Visual Studio 14 2015 Win64
-- Check for working C compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64
-- Check for working CXX compiler using: Visual Studio 14 2015 Win64 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: C:/cygwin64/bin/python2.7.exe (found suitable version "2.7.14", minimum required is "2.7")
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
--   chrono
--   date_time
--   atomic
-- Found GFlags: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found gflags  (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: gflags_shared)
-- Found Glog: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found glog    (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: glog)
-- Found Protobuf: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/bin/protoc.exe (found version "3.1.0")
-- Found PROTOBUF Compiler: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/bin/protoc.exe
-- Found LMDB: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found lmdb    (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: lmdb)
-- Found LevelDB: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found LevelDB (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: leveldb)
-- Found ZLIB: optimized;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlib.lib;debug;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlibd.lib (found version "1.2.8")
-- Found Snappy: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Found Snappy  (include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include, library: snappy_static;optimized;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlib.lib;debug;C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/caffezlibd.lib)
CMake Warning at cmake/Dependencies.cmake:97 (message):
  -- CUDA is not detected by cmake.  Building without it...
Call Stack (most recent call first):
  CMakeLists.txt:80 (include)


-- OpenCV found (C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries)
-- Found OpenBLAS libraries: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/lib/libopenblas.dll.a
-- Found OpenBLAS include: C:/Users/Abdullah Siddiqui/.caffe/dependencies/libraries_v140_x64_py27_1.1.0/libraries/include
-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named numpy
-- Could NOT find NumPy (missing:  NUMPY_INCLUDE_DIR NUMPY_VERSION) (Required is at least version "1.7.1")
-- Boost version: 1.61.0
-- Found the following Boost libraries:
--   python
-- Python interface is disabled or not all required dependencies found. Building without it...
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.10.1.windows.1")
--
-- ******************* Caffe Configuration Summary *******************
-- General:
--   Version           :   1.0.0
--   Git               :   1.0-111-g509dae0-dirty
--   System            :   Windows
--   C++ compiler      :   C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
--   Release CXX flags :   /MD /O2 /Ob2 /D NDEBUG /DWIN32 /D_WINDOWS /W3 /GR /EHsc
--   Debug CXX flags   :   /D_DEBUG /MDd /Zi /Ob0 /Od /RTC1 /DWIN32 /D_WINDOWS /W3 /GR /EHsc
--   Build type        :   Release
--
--   BUILD_SHARED_LIBS :   0
--   BUILD_python      :   1
--   BUILD_matlab      :   0
--   BUILD_docs        :
--   CPU_ONLY          :   0
--   USE_OPENCV        :   ON
--   USE_LEVELDB       :   ON
--   USE_LMDB          :   ON
--   USE_NCCL          :   0
--   ALLOW_LMDB_NOLOCK :   OFF
--
-- Dependencies:
--   BLAS              :   Yes (Open)
--   Boost             :   Yes (ver. 1.61)
--   glog              :   Yes
--   gflags            :   Yes
--   protobuf          :   Yes (ver. 3.1.0)
--   lmdb              :   Yes (ver. 0.9.70)
--   LevelDB           :   Yes (ver. 1.18)
--   Snappy            :   Yes (ver. 1.1.1)
--   OpenCV            :   Yes (ver. 3.1.0)
--   CUDA              :   No
--
-- Install:
--   Install path      :   C:/Users/Abdullah Siddiqui/projects/caffe/build/install
--
-- Configuring done
CMake Error at CMakeLists.txt:143 (add_dependencies):
  The dependency target "pycaffe" of target "pytest" does not exist.


-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_python_layer
    CUDA_ARCH_NAME


-- Build files have been written to: C:/Users/Abdullah Siddiqui/projects/caffe/build
ERROR: Configure failed

I am really confused about this line:

-- Found PythonInterp: C:/cygwin64/bin/python2.7.exe (found suitable version "2.7.14", minimum required is "2.7")

Why is it detecting python2.7 when the script should be seeing python 3.5? Please help me.

[a52siddi]

More details of my issue are in this link: https://stackoverflow.com/questions/52358784/why-do-i-keep-seeing-could-not-find-pythonlibs-missing-python-libraries-py

[willyd]

Have you tried to use -DPYTHON_EXECUTABLE=<your python 3.5 path> on the cmake configure line?

[a52siddi]

Do you mean like this:

C:\Users\MYNAME\projects\caffe\build>cmake -DPYTHON_EXECUTABLE="C:\Users\MYNAME\Anaconda3\python.exe" ..

?

[a52siddi]

When I do that I get errors about missing libraries (glog being one of them):

-- Could NOT find Glog (missing:  GLOG_LIBRARY)
CMake Error at cmake/ProtoBuf.cmake:6 (find_package):
  Could not find a configuration file for package "Protobuf" that is
  compatible with requested version "".

[a52siddi]

When I run the cmake command from the command prompt, I get errors about missing libraries and when I run the command scripts\build_win.cmd I find that it is able to detect the libraries but NOT the mainstream python libraries (PYTHONLIBS).

[a52siddi]

The script has started working properly after I included the line -DPYTHON_EXECUTABLE=<your python 3.5 path> in it.

[willyd]

Closing as resolved.

[bit-scientist]

@a52siddi what file is it that you included line -DPYTHON_EXECUTABLE=<your python 3.5 path>

[a52siddi]

@bit-scientist I included that line in the file build_win.cmd. It is found in the folder called scripts.

[Aishwaryabaskaran]

Hello @a52siddi . I got the same error while executing scripts\build_win.cmd. Tried to include -DPYTHON_EXECUTABLE=<your python 3.5 path>in the build file but it says, "syntax of the command is incorrect" when i tried to run scripts\build_win.cmd.

caffe> scripts/build_win.cmd
The syntax of the command is incorrect.

I tried to remove it and execute scripts\build_win.cmd and again stuck with the previous error

[a52siddi]

Hello @Aishwaryabaskaran. I cannot confidently say why you are seeing this error until I see how you are typing -DPYTHON_EXECUTABLE=<your python 3.5 path> in build_win.cmd. It would help if you post the script (with your change in it) on this thread.

I tried to remove it and execute scripts\build_win.cmd and again stuck with the previous error

This is probably happening because you are not calling this command from the correct folder. You either don't know the directory structure of your project folder or you don't fully understand how command line arguments for windows and linux work.

You should first know whether you have a windows system or a linux system. You can then learn about the command line argument styles suitable for your system as they are NOT the same for all types of systems. I advise you to learn about file paths (absolute and relative paths) from these web resources: (https://www.geeksforgeeks.org/absolute-relative-pathnames-unix/) and (https://en.wikipedia.org/wiki/Path_(computing)). To test your understanding of this concept, you can then create a folder with other sub-folders in it and try to access these folders (enter into them and exit them) from the command line.