evaluation tool compilation on win10x64 with vs2019

[ustclbh]

Hi there, I am currently willing to compile the C++ evaluation tools for culane following the instructions in INSTALL.md. There is a subtle difference of vs version between mine(vs2019 community) and the one used in the given command, "cmake .. -G "Visual Studio 15 2017 Win64" " which is vs2017.
I have tried some possible commands according to my setup like,
cmake .. -G "Visual Studio 16 2019 Win64" ,
cmake .. -G "Visual Studio 15 2017 Win64" ,
cmake .. -G "Visual Studio 16 2019" -A x64 which I found here ,
cmake .. -G "Visual Studio 16 2019 Win64" -A x64 ,
but neither works...

Here are the cmake error messages. Is there any chance I can fix this up without install a 2017vs by modifying the cmakelist or any other configuration? Thanks in advance!

I also try to directly open the folder with vs2019, but get a following error.

CMake Error at E:\workstation\lanedet\UltraFast\evaluation\culane\CMakeLists.txt:18 (string):
string sub-command REPLACE requires at least four arguments. culane_evaluator E:\workstation\lanedet\UltraFast\evaluation\culane\CMakeLists.txt 18

[ustclbh]

As you can see, I am using the command shell. I ensure that cmake and opencv c++ are installed and the cmake cache is cleared before I rerun any cmake command.
I have no experience of "cmake". Any clue or relevant tutorial is appreciated!

[cfzd]

@zchrissirhcz Could you do me a favor? I'm not familiar with cmake either.

[zchrissirhcz]

@cfzd @ustclbh
The CMakeLists.txt was created by me, wasn't tested on Windows.

@ustclbh The error your snapshot presents is about, "where are the source and header files?". To make it work, change CMakeLists.txt to the following:

cmake_minimum_required(VERSION 3.1)

project(culane_evaluator)

set(CMAKE_CXX_STANDARD 11)

add_definitions(
    -DCPU_ONLY
)

set(SRC_LST
    ${CMAKE_SOURCE_DIR}/src/counter.cpp
    ${CMAKE_SOURCE_DIR}/src/evaluate.cpp
    ${CMAKE_SOURCE_DIR}/src/lane_compare.cpp
    ${CMAKE_SOURCE_DIR}/src/spline.cpp
)

set(HDR_LST
    ${CMAKE_SOURCE_DIR}/include/counter.hpp
    ${CMAKE_SOURCE_DIR}/include/hungarianGraph.hpp
    ${CMAKE_SOURCE_DIR}/include/lane_compare.hpp
    ${CMAKE_SOURCE_DIR}/include/spline.hpp
)

add_executable(${PROJECT_NAME}
    ${SRC_LST}
    ${HDR_LST}
)

set(dep_libs "")

#--- OpenCV
# You may switch different version of OpenCV like this:
# set(OpenCV_DIR "/usr/local/opencv-4.3.0" CACHE PATH "")
find_package(OpenCV REQUIRED 
    COMPONENTS core highgui imgproc imgcodecs
)
if(NOT OpenCV_FOUND) # if not OpenCV 4.x/3.x, then imgcodecs are not found
    find_package(OpenCV REQUIRED COMPONENTS core highgui imgproc)
endif()

list(APPEND dep_libs 
    PUBLIC ${OpenCV_LIBS}
)

#--- OpenMP
find_package(OpenMP)
if(NOT TARGET OpenMP::OpenMP_CXX AND (OpenMP_CXX_FOUND OR OPENMP_FOUND))
    target_compile_options(${PROJECT_NAME} PRIVATE ${OpenMP_CXX_FLAGS})
endif()

if(OpenMP_CXX_FOUND OR OPENMP_FOUND)
    message(STATUS "Building with OpenMP")
    if(OpenMP_CXX_FOUND)
        list(APPEND dep_libs PUBLIC OpenMP::OpenMP_CXX)
    else()
        list(APPEND dep_libs PRIVATE "${OpenMP_CXX_FLAGS}")
    endif()
endif()

# --- target config with include dirs / libs
target_link_libraries(${PROJECT_NAME}
    ${dep_libs}
)

target_include_directories(${PROJECT_NAME}
    PUBLIC include
)

Note: a manually built of OpenCV is required for VS2019 since OpenCV official doesn't provide it.

[zchrissirhcz]

@ustclbh
Apart from modify CMakeLists.txt, we also have to transplant getopt.h/getopt.c.
I've just tested under VS2017 and VS2019, and made an Pull Request here #63

[zchrissirhcz]

@ustclbh
To build OpenCV with VS2019 quikly, you may just follow this steps:

git clone https://gitee.com/mirrors/opencv opencv-4.4.0
cd opencv-4.4.0
git checkout -b 4.4.0 4.4.0
md build
cd build

Then create and edit opencv-4.4.0/build/vs2019-x64.bat as:

@echo off

set BUILD_DIR=vs2019-x64
if not exist %BUILD_DIR% md %BUILD_DIR%
cd %BUILD_DIR%
cmake ../.. -G "Visual Studio 16 2019" -A x64 ^
	-DCMAKE_INSTALL_PREFIX=D:/lib/opencv/4.4.0 ^
	-DCV_DISABLE_OPTIMIZATION=ON ^
	-DWITH_CUDA=OFF ^
	-DWITH_VTK=OFF ^
	-DBUILD_LIST=core,imgcodecs,highgui,imgproc
	
cd ..

pause

Where D:/lib/opencv/4.4.0 is the installation directory of your opencv.
Then in cmd, run vs2019-x64.bat, and open OpenCV.sln, build both Debug and Release for INSTALL target.

Finally, when building culane evalation code with cmake, tell cmake where to find your opencv 4.4.0, such as:

cmake .. -G "Visual Studio 16 2019" -A x64  -D OpenCV_DIR=D:/lib/opencv/4.4.0

[ustclbh]

@zchrissirhcz Thanks a lot! After replacing the new MakeIist, I did meet with the missing "unistd.h" and "getopt" probelm and was struggling with it. Most google solutions do not work for me. I shall try your suggestion quite soon and hopefully take back positive news.

[ustclbh]

@zchrissirhcz Thanks again for your help! Even though I build the opencv myself, I am still stuck with the "undefined identifier getopt" error when run "cmake --build . --config Release", just like before. Everything before that goes smoothly. (including build the opencv, run the "opencv-4.4.0/build/vs2019-x64.bat", build debug and release opencv target).
Here is the error message.

As for the "missing unistd.h and getopt" problem, I find related implementation link web, which includes unistd.h, getopt.h, getopt.c. A closely related issue was discussed here.
Should I download these files and put them somewhere so that getopt func can be found during compilation?
As you mentioned, "we also have to transplant getopt.h/getopt.c." . Would you please give more details about it?

[zchrissirhcz]

Let me make it clear: you can go to see this link: #63

Or, just see my fork of this repo: https://github.com/zchrissirhcz/Ultra-Fast-Lane-Detection/tree/master/evaluation/culane/getopt

[cfzd]

@zchrissirhcz
Thanks a lot!
@ustclbh
The PR is already merged. You can pull this repo again to update the evaluation tool.

[ustclbh]

@zchrissirhcz @cfzd
Thank you both for your help! With the updated evaluation tool, compilation with vs2019 works normally now. Actually, there are some other subtle modification needed for final evaluation of Precision and Recall on Windows. In case others may encounter the same problem, here is what I did just for reference:

1. build the evaluation tools following the instructions above
2. move the "evaluation/culane/build-vs2019/release/culane_evaluator.exe" to "evaluation/culane/", then rename it as "evaluate.exe"
3. run test command:
   python test.py configs/your_culane_config.py --test_model model_path --test_work_dir work_dir_path

Probably you meet some errors when parsing args, like :
Traceback (most recent call last):
File "test.py", line 10, in
args, cfg = merge_config()
File "E:\workstation\lanedet\New Folder\UltraFast\utils\common.py", line 50, in merge_config
cfg = Config.fromfile(args.config)
File "E:\workstation\lanedet\New Folder\UltraFast\utils\config.py", line 160, in fromfile
cfg_dict, cfg_text = Config._file2dict(filename)
File "E:\workstation\lanedet\New Folder\UltraFast\utils\config.py", line 87, in _file2dict
osp.join(temp_config_dir, temp_config_name))
File "C:\Users\ustc\anaconda3\lib\shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:\Users\ustc\AppData\Local\Temp\tmpyctcz435\tmp6xszxji2.py'

4. To sovle this, simply copy the test.py to another file( copy_test.py ) and replacing cfg and args attributes with constant string.
5. modify the eval code snippet of call_culane_eval function in eval_wrapper.py as:
   subprocess.check_call('./evaluation/culane/evaluate.exe -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list0,w_lane,iou,im_w,im_h,frame,out0))
   subprocess.check_call('./evaluation/culane/evaluate.exe -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list1,w_lane,iou,im_w,im_h,frame,out1))
   ...
   subprocess.check_call('./evaluation/culane/evaluate.exe -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(data_dir,detect_dir,data_dir,list8,w_lane,iou,im_w,im_h,frame,out8))
   Then run your copy_test.py and reproduce the precision and recall results.

[zchrissirhcz]

@ustclbh Thanks for your sharing. I'm not with my GPU enabled computer now, will try use your shared code in later several hours. Once tested I'll make a new PR if possible.

[cfzd]

@zchrissirhcz
I think the mentioned permission denied problem is a cross-platform problem of the tempfile module.

The process in utils/config.py is to open a temp file first, then copy the content of configs/culane_config.py to the temp file. Finally, all the content in the temp file would be imported and the loading of configs/culane_config.py is finished.

In this process, the temp file is opened twice. The first time is when it is created, and the second time is when the content of configs/culane_config.py is copied to the temp file. Under Unix, a file can be opened twice, while it can't be done under Windows NT kernel. So the permission denied problem occurs. I think I can fix this in the next commit.

@ustclbh But I don't know why you change the os.system call to the subprocess.check_call? In my opinion, os.system is a cross-platform function. Could you give us the reasons?

[ustclbh]

@cfzd
It sounds like a reasonable explanation with my limited knowledge. Expect to see your solution for this probelm.
As for the reason why os.system was replaced with subprocess.check_call, first it would produce an error with origin code os.system('./evaluation/culane/evaluate ...') , saying that . is not an internal or external command , which I thought was caused by the way to run the evaluate.exe at that time.
So I tried with os.system('./evaluation/culane/evaluate.exe ...') and os.system('evaluation/culane/evaluate.exe ...'), which gives the following error.

Since I did not find a quick fix-up, I found the current alternative solution I am using.
I believe there should be a simple way to deal with this using os.system. But subprocess module is indeed preferred instead of os.system for some reasons that I can not tell, see more here and here

[zchrissirhcz]

@ustclbh @cfzd When we are on Windows, we should use backslash \ instead of slash / to call evaluate, both would be OK, just try this in python/ipython:

subprocess.check_call('.\evaluation\culane\evaluate ')
os.system('.\evaluation\culane\evaluate ')

I think we could use this modification:

import platform

eval_cmd = './evaluation/culane/evaluate'
if platform.system() == 'Windows':
    eval_cmd = eval_cmd.replace('/', os.sep)

os.system('%s -a %s -d %s -i %s -l %s -w %s -t %s -c %s -r %s -f %s -o %s'%(eval_cmd, data_dir,detect_dir,data_dir,list4,w_lane,iou,im_w,im_h,frame,out4))
...

[ustclbh]

Yeah, it works well using \. Not surprisingly to know such simple reason behind, always feeling like a fooled idiot.

[cfzd]

@zchrissirhcz @ustclbh
Great thanks! I made a new push that should support windows now.

I don't have a windows machine with GPU, so I just tested the permission denied problem.
@ustclbh Could you help me test the new os.system call part?

[ustclbh]

@cfzd
All stuff now works well following the new instruction we discussed in this issue. I am able to build the tools and reproduce the Fmeasure with pretrained model.

[cfzd]

@ustclbh
Thanks! : )