Default Tests Failing on Ubuntu 16.04

[FaroukY]

I manually compiled the library and ran make test and I got the following results:

99% tests passed, 10 tests failed out of 785

Total Test time (real) = 115.46 sec

The following tests FAILED:
	373 - integration_meta_cpp-clustering-gaussian_mixture_models (Failed)
	374 - integration_meta_python-clustering-gaussian_mixture_models (Failed)
	490 - python_legacy-structure_dynprog (Failed)
	491 - python_legacy-structure_discrete_hmsvm_mosek (Failed)
	492 - python_legacy-structure_discrete_hmsvm_bmrm (Failed)
	498 - python_legacy-serialization_complex_example (Failed)
	525 - python_legacy-mathematics_logdet (Failed)
	526 - python_legacy-mathematics_linsolver_cg (Failed)
	591 - python_legacy-features_sparse (Failed)
	594 - python_legacy-features_io (Failed)
Errors while running CTest
Makefile:61: recipe for target 'test' failed
make: *** [test] Error 8

I haven't changed anything in the Library. Are these tests supposed to fail?

[vigsterkr]

@FaroukY could you paste here the output of this command:
ctest -R integration_meta_cpp-clustering-gaussian_mixture_models -VV

[FaroukY]

@vigsterkr Here:

UpdateCTestConfiguration  from :/home/ubuntu/Desktop/shogun/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ubuntu/Desktop/shogun/build/DartConfiguration.tcl
Test project /home/ubuntu/Desktop/shogun/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 373
    Start 373: integration_meta_cpp-clustering-gaussian_mixture_models

373: Test command: /home/ubuntu/Desktop/shogun/build/tests/meta/meta_example_integration_tester "clustering" "gaussian_mixture_models.dat" "cpp" "generated_results" "reference_results"
373: Test timeout computed to be: 9.99988e+06
373: [DEBUG] Comparing parameter DynamicObjectArray::array of type shogun::ArrayReference<shogun::CSGObject*, int>.
373: [DEBUG] Comparing parameter Serializable::value of type int.
373: [DEBUG] All parameters of Serializable equal.
373: [DEBUG] Comparing parameter VectorSerializable::value of type shogun::SGVector<double>.
373: [DEBUG] Own parameter VectorSerializable::value=Vector<double>(2): [-2.13507,3.14104] different from provided VectorSerializable::value=Vector<double>(2): [-2.41763,0.769119]
373: [DEBUG] Own parameter DynamicObjectArray::array= different from provided DynamicObjectArray::array=
1/1 Test #373: integration_meta_cpp-clustering-gaussian_mixture_models ...***Failed    0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.07 sec

The following tests FAILED:
	373 - integration_meta_cpp-clustering-gaussian_mixture_models (Failed)
Errors while running CTest

[vigsterkr]

oh this is rather interesting :)

could you maybe paste here the output of your cmake ?
just create another directory (instead of build) and run cmake the same way the way you did before and paste here the full output... i guess you are running this on x86 arch right ? :)

[vigsterkr]

mmm i can get the very same problem with integration_meta_cpp-clustering-gaussian_mixture_models on freebsd where there's not lapack/blas backend. i'm thinking that actually maybe we are currently having some very different results from eigen for the eigenvectors/values when it's not having a LAPACK backend.

still would be great if you could get us your cmake so i can confirm this, i'll look into the above mentioned heuristics

[FaroukY]

@vigsterkr I think you're right. I didn't have lapack/blas installed. But I thought it was an optional library no?

Here is the output of cmake:

cmake -DINTERFACE_PYTHON=ON -DENABLE_TESTING=ON -DBUILD_META_EXAMPLES=ON ..
-- Found CCache: /usr/bin/ccache  
-- Enabling ccache for C
-- Enabling ccache for CXX
-- Using system's malloc
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE) (Required is at least version "1.8.6")
-- dir='/home/ubuntu/Desktop/shogun/src'
-- dir='/home/ubuntu/Desktop/shogun/build/src'
-- dir='/home/ubuntu/Desktop/shogun/src/gpl'
-- Could NOT find Eigen3 (missing:  EIGEN_INCLUDE_DIR) (Required is at least version "3.1.2")
-- Could NOT find OPENCL (missing:  OPENCL_LIBRARY OPENCL_INCLUDE_DIR) 
-- Could NOT find ViennaCL (missing:  VIENNACL_INCLUDE_DIR VIENNACL_ENCODED_VERSION OpenCL_INCLUDE_DIRS OpenCL_LIBRARIES) (Required is at least version "1.5.0")
CMake Warning at src/shogun/CMakeLists.txt:257 (FIND_PACKAGE):
  Could not find a package configuration file provided by "TFLogger"
  (requested version 0.1.0) with any of the following names:

    TFLoggerConfig.cmake
    tflogger-config.cmake

  Add the installation prefix of "TFLogger" to CMAKE_PREFIX_PATH or set
  "TFLogger_DIR" to a directory containing one of the above files.  If
  "TFLogger" provides a separate development package or SDK, be sure it has
  been installed.


-- A library with BLAS API not found. Please specify library location.
-- LAPACK requires BLAS
-- Could NOT find GLPK (missing:  GLPK_LIBRARY GLPK_INCLUDE_DIR GLPK_PROPER_VERSION_FOUND) 
-- Could NOT find CPLEX (missing:  CPLEX_LIBRARY CPLEX_INCLUDE_DIR) 
-- Could NOT find MOSEK (missing:  MOSEK_DIR MOSEK_INCLUDE_DIR MOSEK_LIBRARY MOSEK_LIBRARIES) 
-- Could NOT find Protobuf (missing:  PROTOBUF_LIBRARY PROTOBUF_INCLUDE_DIR) 
-- Could NOT find JSON (missing:  JSON_INCLUDE_DIR JSON_LIBRARY) (Required is at least version "0.11")
-- Could NOT find LibXml2 (missing:  LIBXML2_LIBRARIES LIBXML2_INCLUDE_DIR) 
-- Could NOT find HDF5 (missing:  HDF5_LIBRARIES HDF5_INCLUDE_DIRS) 
-- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR) 
-- Could NOT find ZLIB (missing:  ZLIB_LIBRARY ZLIB_INCLUDE_DIR) 
-- Could NOT find BZip2 (missing:  BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) 
-- Could NOT find LibLZMA (missing:  LIBLZMA_INCLUDE_DIR LIBLZMA_LIBRARY LIBLZMA_HAS_AUTO_DECODER LIBLZMA_HAS_EASY_ENCODER LIBLZMA_HAS_LZMA_PRESET) 
-- Could NOT find SNAPPY (missing:  SNAPPY_LIBRARIES SNAPPY_INCLUDE_DIR) 
-- Lzo includes and libraries NOT found. 
-- Could NOT find NLOPT (missing:  NLOPT_LIBRARY NLOPT_INCLUDE_DIR) 
-- Could NOT find LPSOLVE (missing:  LPSOLVE_LIBRARIES LPSOLVE_INCLUDE_DIR) 
-- Could NOT find ColPack (missing:  COLPACK_LIBRARIES COLPACK_LIBRARY_DIR COLPACK_INCLUDE_DIR) 
-- Could NOT find ARPREC (missing:  ARPREC_LIBRARIES ARPREC_INCLUDE_DIR) 
-- Linker: GNU gold
-- Ctags flavour: Exuberant
-- Found Ctags: /usr/bin/ctags (found version "exuberant ctags 5.9~svn20110310") 
-- Could NOT find Jinja2 (missing:  JINJA2_IMPORT_SUCCESS) 
-- Ctags flavour: Exuberant
Pleaseinstalljinja2forautomaticgeneratedtests.
CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example multiclass/gmnp_svm.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example gaussian_process/regression.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example distance/minkowski.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example distance/chisquare.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example
  converter/multidimensional_scaling.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example binary/svm_oneclass.sg


CMake Warning at tests/meta/CMakeLists.txt:54 (MESSAGE):
  Skipping integration test for meta example binary/sgd_svm.sg


transfer_multitask_trace_logistic_regression.py
transfer_multitask_logistic_regression.py
transfer_multitask_leastsquares_regression.py
transfer_multitask_l12_logistic_regression.py
transfer_multitask_clustered_logistic_regression.py
tests_check_commwordkernel_memleak.py
structure_plif_hmsvm_mosek.py
structure_plif_hmsvm_bmrm.py
structure_multiclass_bmrm.py
structure_hierarchical_multilabel_classification.py
structure_graphcuts.py
structure_factor_graph_model.py
structure_dynprog.py
structure_discrete_hmsvm_mosek.py
structure_discrete_hmsvm_bmrm.py
stochasticgbmachine.py
so_multiclass.py
serialization_svmlight.py
serialization_string_kernels.py
serialization_matrix.py
serialization_complex_example.py
regression_svrlight.py
regression_randomforest.py
regression_chaidtree.py
regression_cartree.py
preprocessor_sortwordstring.py
preprocessor_sortulongstring.py
preprocessor_randomfouriergausspreproc.py
preprocessor_prunevarsubmean.py
preprocessor_pca.py
preprocessor_normone.py
preprocessor_logplusone.py
preprocessor_kernelpca.py
preprocessor_fisherlda.py
preprocessor_dimensionreductionpreprocessor.py
multiclass_randomforest.py
multiclass_id3classifiertree.py
multiclass_c45classifiertree.py
modelselection_random_search_liblinear.py
modelselection_grid_search_libsvr.py
modelselection_grid_search_liblinear.py
modelselection_grid_search_krr.py
modelselection_grid_search_kernel.py
mkl_multiclass.py
mkl_binclass.py
metric_lmnn.py
mathematics_sparseinversecovariance.py
mathematics_logdet.py
mathematics_linsolver_cg.py
library_time.py
library_fisher2x3.py
labels_io.py
kernel_weighted_degree_string.py
kernel_weighted_degree_position_string.py
kernel_weighted_comm_word_string.py
kernel_wavelet.py
kernel_wave.py
kernel_tstudent.py
kernel_top.py
kernel_ssk_string.py
kernel_spline.py
kernel_spherical.py
kernel_sparse_poly.py
kernel_sparse_linear.py
kernel_sparse_gaussian.py
kernel_simple_locality_improved_string.py
kernel_sigmoid.py
kernel_salzberg_word_string.py
kernel_rationalquadratic.py
kernel_power.py
kernel_poly_match_word_string.py
kernel_poly_match_string.py
kernel_poly.py
kernel_oligo_string.py
kernel_multiquadric.py
kernel_match_word_string.py
kernel_log.py
kernel_locality_improved_string.py
kernel_local_alignment_string.py
kernel_linear_word.py
kernel_linear_string.py
kernel_linear_byte.py
kernel_linear.py
kernel_io.py
kernel_inversemultiquadric.py
kernel_histogram_word_string.py
kernel_gaussian_shift.py
kernel_gaussian.py
kernel_fixed_degree_string.py
kernel_fisher.py
kernel_exponential.py
kernel_distance.py
kernel_director_linear.py
kernel_diag.py
kernel_custom.py
kernel_const.py
kernel_comm_word_string.py
kernel_comm_ulong_string.py
kernel_combined_custom_poly.py
kernel_combined.py
kernel_circular.py
kernel_chi2.py
kernel_cauchy.py
kernel_auc.py
kernel_anova.py
features_string_word.py
features_string_ulong.py
features_string_sliding_window.py
features_string_hashed_wd.py
features_string_file_char.py
features_string_file.py
features_string_char_compressed.py
features_string_char.py
features_sparse.py
features_snp.py
features_read_svmlight_format.py
features_io.py
features_hasheddocdot.py
features_director_dot.py
features_dense_zero_copy.py
features_dense_real.py
features_dense_protocols.py
features_dense_longint.py
features_dense_io.py
features_dense_byte.py
features_dense.py
features_binned_dot.py
evaluation_thresholds.py
evaluation_rocevaluation.py
evaluation_prcevaluation.py
evaluation_multiclassovrevaluation.py
evaluation_multiclassaccuracy.py
evaluation_meansquaredlogerror.py
evaluation_meansquarederror.py
evaluation_director_contingencytableevaluation.py
evaluation_cross_validation_regression.py
evaluation_cross_validation_mkl_weight_storage.py
evaluation_contingencytableevaluation.py
evaluation_clustering_simple.py
evaluation_clustering.py
distribution_ppwm.py
distribution_linearhmm.py
distribution_hmm.py
distribution_histogram.py
distance_tanimoto.py
distance_sparseeuclidean.py
distance_normsquared.py
distance_minkowski.py
distance_manhattenword.py
distance_jensen.py
distance_hammingword.py
distance_geodesic.py
distance_director_euclidean.py
distance_chisquare.py
distance_chebyshew.py
distance_canberraword.py
distance_canberra.py
converter_tdistributedstochasticneighborembedding.py
converter_stochasticproximityembedding.py
converter_multidimensionalscaling.py
converter_localtangentspacealignment.py
converter_locallylinearembedding.py
converter_localitypreservingprojections.py
converter_linearlocaltangentspacealignment.py
converter_laplacianeigenmaps.py
converter_kernellocallylinearembedding.py
converter_isomap.py
converter_hessianlocallylinearembedding.py
converter_hasheddoc.py
converter_factoranalysis.py
converter_diffusionmaps.py
classifier_svmsgd.py
classifier_svmocas.py
classifier_svmlight_linear_term.py
classifier_svmlight_batch_linadd.py
classifier_svmlight.py
classifier_ssk.py
classifier_multilabeloutputliblinear.py
classifier_multiclassocas.py
classifier_multiclassmachine.py
classifier_multiclassliblinear.py
classifier_multiclass_ecoc.py
classifier_mpdsvm.py
classifier_libsvmoneclass.py
classifier_lda.py
classifier_larank.py
classifier_gpbtsvm.py
classifier_gmnpsvm.py
classifier_featureblock_logistic_regression.py
classifier_domainadaptationsvm.py
classifier_custom_kernel.py
-- Ctags flavour: Exuberant
-- Failed to locate sphinx-build executable (missing:  SPHINX_EXECUTABLE) 
-- Failed to locate pandoc executable (missing:  PANDOC_EXECUTABLE) 
-- ===================================================================================================================
-- Summary of Configuration Variables
-- 
-- The following OPTIONAL packages have been found:

 * CCache
 * GDB
 * OpenMP
 * CxaDemangle
 * Threads
 * Ctags

-- The following REQUIRED packages have been found:

 * SWIG (required version >= 3.0.12)
 * PythonLibs (required version == 2.7.12)
 * NumPy (required version >= 1.7)
 * PythonInterp
 * PLY

-- The following OPTIONAL packages have not been found:

 * Doxygen (required version >= 1.8.6)
 * Eigen3 (required version >= 3.1.2)
 * ViennaCL (required version >= 1.5.0)
 * rxcpp
 * TFLogger (required version >= 0.1.0)
 * BLAS
 * GLPK
 * CPLEX
 * ARPACK
 * Mosek
 * Protobuf
 * JSON (required version >= 0.11)
 * LibXml2
 * HDF5
 * CURL
 * ZLIB
 * BZip2
 * LibLZMA
 * SNAPPY
 * LZO
 * NLopt
 * LpSolve
 * ColPack
 * ARPREC
 * Jinja2
 * Sphinx
 * Pandoc

-- ===================================================================================================================
-- Integrations
--   OpenCV Integration is OFF 	 enable with -DOpenCV=ON
-- ===================================================================================================================
-- Interfaces
--   Python is ON
--   Octave is OFF 		 enable with -DINTERFACE_OCTAVE=ON
--   Java is OFF 		 enable with -DINTERFACE_JAVA=ON
--   Perl is OFF 		 enable with -DINTERFACE_PERL=ON
--   Ruby is OFF 		 enable with -DINTERFACE_RUBY=ON
--   C# is OFF 			 enable with -DINTERFACE_CSHARP=ON
--   R is OFF 			 enable with -DINTERFACE_R=ON
--   Lua is OFF 		 enable with -DINTERFACE_LUA=ON
--   Scala is OFF 		 enable with -DINTERFACE_SCALA=ON
-- ===================================================================================================================
-- To compile shogun type
--   make
-- 
-- To install shogun to /usr/local type
--   make install
-- 
-- or to install to a custom directory
--   make install DESTDIR=/my/special/path
--   (or rerun cmake with -DCMAKE_INSTALL_PREFIX=/my/special/path) to just change the prefix
-- ===================================================================================================================
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/Desktop/shogun/build

[vigsterkr]

yes yes it's options and it seems this is a bug.

[FaroukY]

@vigsterkr Okay so I install lapack/blas and did make then make test and now they're down to 2 failed test cases:

99% tests passed, 2 tests failed out of 801

Total Test time (real) = 121.37 sec

The following tests FAILED:
	514 - python_legacy-serialization_complex_example (Failed)
	610 - python_legacy-features_io (Failed)
Errors while running CTest
Makefile:61: recipe for target 'test' failed
make: *** [test] Error 8

Do you recommend just installing all the Optional Libraries for now as a work around?

[vigsterkr]

@FaroukY @vinx13 @shubham808 these errors are actually blockers for the new release. the only way to reproduce this is actually to disable the LAPACK backend to be used by eigen (ENABLE_EIGEN_LAPACK) and compile shogun with it... it'd be great if you could somehow figure out why this happens and help in fixing these tests

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue is now being closed due to a lack of activity. Feel free to reopen it.