Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recent conda changes breaking Travis #6030

Closed
bashtage opened this issue Sep 27, 2017 · 82 comments
Closed

Recent conda changes breaking Travis #6030

bashtage opened this issue Sep 27, 2017 · 82 comments

Comments

@bashtage
Copy link

@bashtage bashtage commented Sep 27, 2017

Not 100% sure that this is a conda issue, but it looks like changes in .27 are breaking Travis builds that
use extension modules.

In particular, the compiler is identified as

compiler: x86_64-conda_cos6-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -fPIC

Not sure why conda would be in the compiler.

And example of a failed build can be see https://travis-ci.org/statsmodels/statsmodels/builds/280537305

@msarahan
Copy link
Member

@msarahan msarahan commented Sep 27, 2017

This is because python bakes in compiler flags at build time. We have used new compilers in the 4.3.27 miniconda. These are available as gcc_linux-64, but it's better to use conda-build 3 and the new compiler jinja2 function.

Here's a recipe for statsmodels that has been modified to use conda-build 3 features: https://github.com/anacondarecipes/statsmodels-feedstock

@bashtage
Copy link
Author

@bashtage bashtage commented Sep 27, 2017

Is the solution to install gcc_linux-64 so that the normal python setup.py develop will correctly build?

@msarahan
Copy link
Member

@msarahan msarahan commented Sep 27, 2017

gcc_linux-64 will provide x86_64-conda_cos6-linux-gnu-gcc - this is a "prefixed" compiler - the stuff preceding gcc is known as the triplet, in gcc terms. Installing the package might work, but it also might not, depending on whether you activate the build environment. If you do, it should work, if you don't, it probably won't be on PATH, and won't work.

It's a bug in our python packages that things are not falling back to your system gcc. We're going to change the symlinks for miniconda-latest to point to the older release until we have this fixed. Sorry for the headache.

@bashtage
Copy link
Author

@bashtage bashtage commented Sep 27, 2017

Thanks.

I did a conda install gcc_linux-64 and conda .27 recommended upgrading a Python 3.5 to a Python 3.6 install, which seems pretty heavy handed.

@kalefranz
Copy link
Contributor

@kalefranz kalefranz commented Sep 27, 2017

I did a conda install gcc_linux-64 and conda .27 recommended upgrading a Python 3.5 to a Python 3.6 install, which seems pretty heavy handed.

This is probably a result of #5923

@lesteve
Copy link

@lesteve lesteve commented Sep 27, 2017

Similar problem for scikit-learn for example https://travis-ci.org/scikit-learn/scikit-learn/jobs/280535821. Not sure why but it seems to be happening only for our Python 2.7 build using conda. I am hoping to work-around the problem by using an older miniconda installer and not do conda update conda.

@kalefranz
Copy link
Contributor

@kalefranz kalefranz commented Sep 27, 2017

@msarahan
Copy link
Member

@msarahan msarahan commented Sep 27, 2017

As Kale said, we changed the "latest" minicondas to point at the 4.3.21 version from June. We have a fix, and are building packages. We'll upload new minicondas after testing. Sorry for the trouble.

Updating conda triggers this because updating conda pulls in packages from the new "main" channel, which is part of defaults. That channel has the new python builds that are causing this issue. Updating our python packages will be the fix.

@lesteve
Copy link

@lesteve lesteve commented Sep 27, 2017

Great stuff, thanks!

@hainm
Copy link

@hainm hainm commented Sep 28, 2017

gcc: error: unrecognized command line option ‘-fstack-protector-strong’
gcc: error: unrecognized command line option ‘-fno-plt’

I got this today :(

@twiecki
Copy link
Contributor

@twiecki twiecki commented Sep 28, 2017

I'm getting when travis is trying to build subprocess32.

  x86_64-conda_cos6-linux-gnu-gcc -pthread -fno-strict-aliasing -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -DNDEBUG -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/travis/miniconda2/envs/testenv/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o

  unable to execute 'x86_64-conda_cos6-linux-gnu-gcc': No such file or directory

  error: command 'x86_64-conda_cos6-linux-gnu-gcc' failed with exit status 1
@sandys
Copy link

@sandys sandys commented Sep 28, 2017

hi guys,
is there a workaround for this ? all our builds are failing . I installed gcc_linux-64, but i keep getting other errors (cc1plus not found, etc)

@mingwandroid
Copy link
Contributor

@mingwandroid mingwandroid commented Sep 28, 2017

@lesteve
Copy link

@lesteve lesteve commented Sep 28, 2017

The simple work-around I am using for scikit-learn (and for myself) is: do NOT do conda update in your root conda environment. Otherwise this picks up a conda that has the problem (4.3.27-py36h2866c0b_0 for me on Python 3.6).

Doing this you stay with conda 4.3.25 and you are not affected by this issue.

@twiecki
Copy link
Contributor

@twiecki twiecki commented Sep 28, 2017

@sandys
Copy link

@sandys sandys commented Sep 28, 2017

i may have a partial fix.
i changed my meta.yaml to the following and it kinda worked. it is still failing on test.


build:
    number: {{ build }}
    script: python setup.py install --single-version-externally-managed --record record.txt

requirements:
  host:
    - python
    - setuptools
    - cython 0.26*
    - numpy 1.12*
  build:
    - python
    - opencv 3.1*
    - numpy 1.12*
    - {{ compiler('cxx') }}
    - cython 0.26*
  run:
    - python
    - opencv 3.1*
    - numpy 1.12*
    - cython 0.26*
@bashtage
Copy link
Author

@bashtage bashtage commented Sep 28, 2017

@lesteve 's fix is working in statsmodels.

@bashtage
Copy link
Author

@bashtage bashtage commented Sep 28, 2017

@twiecki You need to pin conda<=4.3.25

From your build log during an installation:

 conda-4.3.27               |   py27hff99c7a_0         508 KB
@sandys
Copy link

@sandys sandys commented Sep 28, 2017

@twiecki
Copy link
Contributor

@twiecki twiecki commented Sep 28, 2017

@bashtage I thought it was replaced by @mingwandroid:

The lastest minicondas have been temporarily replaced with the previous one while we name a fix here.

@bashtage
Copy link
Author

@bashtage bashtage commented Sep 28, 2017

AFAIU The miniconda install was changed so that the default conda is < 4.3.27, the problematic one. Your environment creation is triggering an automatic update of conda to latest, which is 4.3.27. The conda hasn't been unpublished -- only the miniconda(2/3)-latest installer has been retargetted.

@armaneshaghi
Copy link

@armaneshaghi armaneshaghi commented Dec 6, 2017

I have the same issue with the latest Anaconda (downloaded today) when installing new packages in R.

@mingwandroid
Copy link
Contributor

@mingwandroid mingwandroid commented Dec 6, 2017

@armaneshaghi what issue? You need to install our compilers. The package is called gxx_linux-64.

@armaneshaghi
Copy link

@armaneshaghi armaneshaghi commented Dec 6, 2017

@mingwandroid thanks for the response, conda install gxx_linux-64 solved my issue. I just wonder why this is no longer automatic for naive users (like me)?

@mingwandroid
Copy link
Contributor

@mingwandroid mingwandroid commented Dec 6, 2017

Historically we didn't provide any compilers, then we provided old ones but our R packages didn't use the those old ones instead relying on system ones (when you use install.packages()). That brings it's own problems (I don't know how capable your system ones are and anything you build with them will likely not work on other systems and loading R packages built with them could prevent binaries we built from loading or working).

Forcing our compilers on everyone who installs R on Linux might be the best plan though, but actually upstream R is moving towards providing prebuilt binaries (at least for Windows and macOS).

The other complication is that you are better off not using install.packages() for most things since we already provide prebuilt binary conda packages for a lot of the most popular R packages, so conda install r-something is better.

I am considering to patch install.packages() so that it tries installing a conda package first, and failing that, installs compilers if necessary before running the normal code-path.

Overall it's very difficult to know what is best here!

jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this issue Dec 18, 2017
alexlib added a commit to alexlib/dockerfiles that referenced this issue Jan 9, 2018
read something on this one, not sure it's the right solution, going to test ,conda/conda#6030
@corytu
Copy link

@corytu corytu commented Feb 25, 2018

@mingwandroid some research on the Internet led me here. I am facing difficulties when installing R packages just like @nick-youngblut did. I see your suggestion for using conda skeleton cran approach instead of using R function install.packages, and I have done conda install gxx_linux-64. However, I keep failing at the conda build step.

Specifically, I want to install an R package called "RPostgreSQL", and here is what I have done:

conda install gxx_linux-64
conda skeleton cran r-rpostgresql
conda build r-rpostgresql
# I also tried:
# conda build --R 3.4.2 r-rpostgresql
# conda build --R=3.4.2 r-rpostgresql

The error includes:

make: /home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/bin/x86_64-conda_cos6-linux-gnu-cc: Command not found

, and

make: *** [/home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/R/etc/Makeconf:160: RS-DBI.o] Error 127
ERROR: compilation failed for package ‘RPostgreSQL’

, and finally

subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/work/conda_build.sh']' returned non-zero exit status 1.

You could find some other information in my question on Stack Overflow: Installing packages in IRkernel on Jupyter Notebook.

How should I do to solve this?

@casperdcl casperdcl mentioned this issue Feb 26, 2019
2 of 3 tasks complete
casperdcl added a commit to SyneRBI/SIRF-SuperBuild that referenced this issue Feb 27, 2019
casperdcl added a commit to SyneRBI/SIRF-SuperBuild that referenced this issue Feb 28, 2019
@zhou13
Copy link

@zhou13 zhou13 commented Jul 22, 2019

I am on Archlinux and miniconda always gives me file not recognized: file format not recognized when gcc or ld is called. Removing compiler_compat/ld solves the issue.

@astrojuanlu
Copy link
Contributor

@astrojuanlu astrojuanlu commented Aug 1, 2019

@shreyasbapat and I just independently reproduced what @zhou13 mentioned on Arch Linux. I guess this is being tracked in ContinuumIO/anaconda-issues#11152 now.

@mingwandroid
Copy link
Contributor

@mingwandroid mingwandroid commented Aug 2, 2019

@Juanlu001
Would it be possible to try the package mentioned in my last comment on issue 11152 please?

@astrojuanlu
Copy link
Contributor

@astrojuanlu astrojuanlu commented Aug 2, 2019

Thanks @mingwandroid, doing conda install -c rdonnelly python=3.7.4 fixes the issue 👍

@chux0519
Copy link

@chux0519 chux0519 commented Sep 6, 2019

@zhou13 thanks, it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet