document building FastRGF with Clang and AppleClang

[StrikerRUS]

Follow-up #316.

[StrikerRUS]

Hmmm, seems that compiled with Clang and AppleClang FastRGF binaries cannot pass our simple test during rgf_python installation.

rgf/python-package/setup.py

Lines 116 to 153 in 49756ae

	def is_fastrgf_response(path):
	temp_x_loc = os.path.join(CURRENT_DIR, 'temp_fastrgf.train.data.x')
	temp_y_loc = os.path.join(CURRENT_DIR, 'temp_fastrgf.train.data.y')
	temp_model_loc = os.path.join(CURRENT_DIR, "temp_fastrgf.model")
	temp_pred_loc = os.path.join(CURRENT_DIR, "temp_fastrgf.predictions.txt")
	path_train = os.path.join(path, ("forest_train.exe" if system() in ('Windows', 'Microsoft')
	else "forest_train"))
	params_train = []
	params_train.append("forest.ntrees=%s" % 10)
	params_train.append("tst.target=%s" % "BINARY")
	params_train.append("trn.x-file=%s" % temp_x_loc)
	params_train.append("trn.y-file=%s" % temp_y_loc)
	params_train.append("model.save=%s" % temp_model_loc)
	cmd_train = [path_train]
	cmd_train.extend(params_train)
	path_pred = os.path.join(path, ("forest_predict.exe" if system() in ('Windows', 'Microsoft')
	else "forest_predict"))
	params_pred = []
	params_pred.append("model.load=%s" % temp_model_loc)
	params_pred.append("tst.x-file=%s" % temp_x_loc)
	params_pred.append("tst.output-prediction=%s" % temp_pred_loc)
	cmd_pred = [path_pred]
	cmd_pred.extend(params_pred)
	try:
	os.chmod(path_train, os.stat(path_train).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
	os.chmod(path_pred, os.stat(path_pred).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
	except Exception:
	pass
	with open(temp_x_loc, 'w') as X, open(temp_y_loc, 'w') as y:
	for _ in range(14):
	X.write('1 0 1 0\n0 1 0 1\n')
	y.write('1\n-1\n')
	success = silent_call(cmd_train)
	success &= silent_call(cmd_pred)
	if success:
	return True
	else:
	return False

Refer to the corresponding Travis jobs (with debug logs): https://travis-ci.org/RGF-team/rgf/builds/657496643

Although FastRGF compiled successfully

Clang:

  INFO:rgf_python:Starting to compile FastRGF executable files.
  cmake version 3.12.4
  CMake suite maintained and supported by Kitware (kitware.com/cmake).
INFO:rgf_python:Trying to build executable files with CMake.
  -- The C compiler identification is Clang 7.0.0
  -- The CXX compiler identification is Clang 7.0.0
  -- Check for working C compiler: /usr/local/clang-7.0.0/bin/clang
  -- Check for working C compiler: /usr/local/clang-7.0.0/bin/clang -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/local/clang-7.0.0/bin/clang++
  -- Check for working CXX compiler: /usr/local/clang-7.0.0/bin/clang++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  Use OpenMP for multithreading
  -- Found OpenMP_C: -fopenmp=libomp (found version "3.1")
  -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1")
  -- Found OpenMP: TRUE (found version "3.1")
  C++ compiler: /usr/local/clang-7.0.0/bin/clang++
  C++ options: -O3 -std=c++11 -fopenmp=libomp -pthread -ftree-vectorize -ffast-math
  C++ definitions: USE_OMP
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-qiln_j36/compile/FastRGF/build  829
  Scanning dependencies of target base
    [  5%] Building CXX object src/base/CMakeFiles/base.dir/utils.cpp.o
    [ 10%] Building CXX object src/base/CMakeFiles/base.dir/data.cpp.o
    [ 15%] Building CXX object src/base/CMakeFiles/base.dir/discretization.cpp.o
    [ 21%] Building CXX object src/base/CMakeFiles/base.dir/classifier.cpp.o
    [ 26%] Linking CXX static library libbase.a
    [ 26%] Built target base
    Scanning dependencies of target forest
    [ 31%] Building CXX object src/forest/CMakeFiles/forest.dir/dtree_trainer.cpp.o
    [ 36%] Building CXX object src/forest/CMakeFiles/forest.dir/dtree.cpp.o
    [ 42%] Building CXX object src/forest/CMakeFiles/forest.dir/forest.cpp.o
    [ 47%] Linking CXX static library libforest.a
    [ 47%] Built target forest
    Scanning dependencies of target forest_train
    [ 52%] Building CXX object src/exe/CMakeFiles/forest_train.dir/forest_train.cpp.o
    [ 57%] Linking CXX executable forest_train
    [ 57%] Built target forest_train
    Scanning dependencies of target forest_predict
    [ 63%] Building CXX object src/exe/CMakeFiles/forest_predict.dir/forest_predict.cpp.o
    [ 68%] Linking CXX executable forest_predict
    [ 68%] Built target forest_predict
    Scanning dependencies of target discretized_gendata
    [ 73%] Building CXX object src/exe/CMakeFiles/discretized_gendata.dir/discretized_gendata.cpp.o
    [ 78%] Linking CXX executable discretized_gendata
    [ 78%] Built target discretized_gendata
    Scanning dependencies of target auc
    [ 84%] Building CXX object src/exe/CMakeFiles/auc.dir/auc.cpp.o
    [ 89%] Linking CXX executable auc
    [ 89%] Built target auc
    Scanning dependencies of target discretized_trainer
    [ 94%] Building CXX object src/exe/CMakeFiles/discretized_trainer.dir/discretized_trainer.cpp.o
    [100%] Linking CXX executable discretized_trainer
    [100%] Built target discretized_trainer
  [ 26%] Built target base
    [ 47%] Built target forest
    [ 57%] Built target forest_train
    [ 68%] Built target forest_predict
    [ 78%] Built target discretized_gendata
    [ 89%] Built target auc
    [100%] Built target discretized_trainer
    Install the project...
    -- Install configuration: ""
    -- Installing: /tmp/pip-req-build-qiln_j36/compile/FastRGF/bin/forest_train
    -- Installing: /tmp/pip-req-build-qiln_j36/compile/FastRGF/bin/forest_predict
  ERROR:rgf_python:Compilation of FastRGF executable files failed. Please build from binaries on your own and specify path to the compiled files in the config file.

AppleClang:

  INFO:rgf_python:Starting to compile FastRGF executable files.
  cmake version 3.16.2
  CMake suite maintained and supported by Kitware (kitware.com/cmake).
INFO:rgf_python:Trying to build executable files with CMake.
  -- The C compiler identification is AppleClang 11.0.0.11000033
  -- The CXX compiler identification is AppleClang 11.0.0.11000033
  -- Check for working C compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
  -- Check for working C compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  -- Check for working CXX compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  Use OpenMP for multithreading
  -- Found OpenMP_C: -Xclang -fopenmp (found version "3.1")
  -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
  -- Found OpenMP: TRUE (found version "3.1")
  C++ compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  C++ options: -O3 -std=c++11 -Xclang -fopenmp -pthread -ftree-vectorize -ffast-math
  C++ definitions: USE_OMP
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /private/var/folders/17/5mc7816d3mndxjqgplq6057w0000gn/T/pip-req-build-o1fxjh9r/compile/FastRGF/build
  ERROR:rgf_python:Compilation of FastRGF executable files failed. Please build from binaries on your own and specify path to the compiled files in the config file.

I guess this is somehow connected to #92.

I also tried to decrease the lvl of compiler optimizations - no luck.

So I'm going to revert the changes related to removal of used compiler restrictions in rgf_python. I think for now we should stick to the current policy of allowing only g++-5 and later as only this compiler can pass CI tests.

cc @ankane

[fukatani]

Thank you for your investigation and PR.
Revirting clang and apple clang will be done with other PR?

[StrikerRUS]

@fukatani Do you mean to revert #316? I think we can leave the possibility to compile with AppleClang and Clang as it compiles successfully now

Although FastRGF compiled successfully
#319 (comment)

and we have the evidence of successful run: #316 (comment). So, I guess Clang-compiled binaries work well under the some conditions (maybe with large enough datasets).
It will be consistent with our previous policy to not prohibit compilation itself with g++-4, while those binaries cannot pass our tests.

[fukatani]

Ah, I see.
Thank you for your contribution!