Recent conda changes breaking Travis

[bashtage]

Not 100% sure that this is a conda issue, but it looks like changes in .27 are breaking Travis builds that
use extension modules.

In particular, the compiler is identified as

compiler: x86_64-conda_cos6-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -fPIC

Not sure why conda would be in the compiler.

And example of a failed build can be see https://travis-ci.org/statsmodels/statsmodels/builds/280537305

[msarahan]

This is because python bakes in compiler flags at build time. We have used new compilers in the 4.3.27 miniconda. These are available as gcc_linux-64, but it's better to use conda-build 3 and the new compiler jinja2 function.

Here's a recipe for statsmodels that has been modified to use conda-build 3 features: https://github.com/anacondarecipes/statsmodels-feedstock

[bashtage]

Is the solution to install gcc_linux-64 so that the normal python setup.py develop will correctly build?

[msarahan]

gcc_linux-64 will provide x86_64-conda_cos6-linux-gnu-gcc - this is a "prefixed" compiler - the stuff preceding gcc is known as the triplet, in gcc terms. Installing the package might work, but it also might not, depending on whether you activate the build environment. If you do, it should work, if you don't, it probably won't be on PATH, and won't work.

It's a bug in our python packages that things are not falling back to your system gcc. We're going to change the symlinks for miniconda-latest to point to the older release until we have this fixed. Sorry for the headache.

[bashtage]

Thanks.

I did a conda install gcc_linux-64 and conda .27 recommended upgrading a Python 3.5 to a Python 3.6 install, which seems pretty heavy handed.

[kalefranz]

I did a conda install gcc_linux-64 and conda .27 recommended upgrading a Python 3.5 to a Python 3.6 install, which seems pretty heavy handed.

This is probably a result of #5923

[lesteve]

Similar problem for scikit-learn for example https://travis-ci.org/scikit-learn/scikit-learn/jobs/280535821. Not sure why but it seems to be happening only for our Python 2.7 build using conda. I am hoping to work-around the problem by using an older miniconda installer and not do conda update conda.

[kalefranz]

The '-latest' miniconda installers have been repainted back to the previous installers with the python version that won't break you.

…

Sent from my iPhone

On Sep 27, 2017, at 4:45 PM, Loïc Estève ***@***.***> wrote: Similar problem for scikit-learn for example https://travis-ci.org/scikit-learn/scikit-learn/jobs/280535821. Not sure why but it seems to be happening only for our Python 2.7 build using conda. I am hoping to work-around the problem by using an older miniconda installer and not do conda update conda. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[msarahan]

As Kale said, we changed the "latest" minicondas to point at the 4.3.21 version from June. We have a fix, and are building packages. We'll upload new minicondas after testing. Sorry for the trouble.

Updating conda triggers this because updating conda pulls in packages from the new "main" channel, which is part of defaults. That channel has the new python builds that are causing this issue. Updating our python packages will be the fix.

[lesteve]

Great stuff, thanks!

[hainm]

gcc: error: unrecognized command line option ‘-fstack-protector-strong’
gcc: error: unrecognized command line option ‘-fno-plt’

I got this today :(

[twiecki]

I'm getting when travis is trying to build subprocess32.

  x86_64-conda_cos6-linux-gnu-gcc -pthread -fno-strict-aliasing -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -pipe -DNDEBUG -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/travis/miniconda2/envs/testenv/include/python2.7 -c _posixsubprocess.c -o build/temp.linux-x86_64-2.7/_posixsubprocess.o

  unable to execute 'x86_64-conda_cos6-linux-gnu-gcc': No such file or directory

  error: command 'x86_64-conda_cos6-linux-gnu-gcc' failed with exit status 1

[sandys]

hi guys,
is there a workaround for this ? all our builds are failing . I installed gcc_linux-64, but i keep getting other errors (cc1plus not found, etc)

[mingwandroid]

The lastest minicondas have been temporarily replaced with the previous one while we name a fix here. If you want to continue with the new stuff and the new compilers (which I would recommend to be honest) then also install gxx_linux-64 to fix your cc1plus problem.

…

On Sep 28, 2017 1:03 PM, "Sandeep Srinivasa" ***@***.***> wrote: hi guys, is there a workaround for this ? all our builds are failing . I installed gcc_linux-64, but i keep getting other errors (cc1plus not found, etc) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#6030 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_pdPQElDgBQKM7NC3MFjoqMfW3SddQks5sm28GgaJpZM4PmOLu> .

[lesteve]

The simple work-around I am using for scikit-learn (and for myself) is: do NOT do conda update in your root conda environment. Otherwise this picks up a conda that has the problem (4.3.27-py36h2866c0b_0 for me on Python 3.6).

Doing this you stay with conda 4.3.25 and you are not affected by this issue.

[twiecki]

@mingwandroid Still getting the same error. https://travis-ci.org/pymc-devs/pymc3/jobs/280543334

[sandys]

i may have a partial fix.
i changed my meta.yaml to the following and it kinda worked. it is still failing on test.

build:
    number: {{ build }}
    script: python setup.py install --single-version-externally-managed --record record.txt

requirements:
  host:
    - python
    - setuptools
    - cython 0.26*
    - numpy 1.12*
  build:
    - python
    - opencv 3.1*
    - numpy 1.12*
    - {{ compiler('cxx') }}
    - cython 0.26*
  run:
    - python
    - opencv 3.1*
    - numpy 1.12*
    - cython 0.26*

[bashtage]

@lesteve 's fix is working in statsmodels.

[bashtage]

@twiecki You need to pin conda<=4.3.25

From your build log during an installation:

 conda-4.3.27               |   py27hff99c7a_0         508 KB

[sandys]

Installing conda-build force upgrades conda

…

On 28-Sep-2017 16:48, "Kevin Sheppard" ***@***.***> wrote: @twiecki <https://github.com/twiecki> You need to pin conda<=4.3.35 From your build log during an installation: conda-4.3.27 | py27hff99c7a_0 508 KB — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#6030 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEsU-cpc1WJq1Zf_PYsn32pDGMJ9xRDks5sm4BpgaJpZM4PmOLu> .

[twiecki]

@bashtage I thought it was replaced by @mingwandroid:

The lastest minicondas have been temporarily replaced with the previous one while we name a fix here.

[bashtage]

AFAIU The miniconda install was changed so that the default conda is < 4.3.27, the problematic one. Your environment creation is triggering an automatic update of conda to latest, which is 4.3.27. The conda hasn't been unpublished -- only the miniconda(2/3)-latest installer has been retargetted.

[armaneshaghi]

@mingwandroid thanks for the response, conda install gxx_linux-64 solved my issue. I just wonder why this is no longer automatic for naive users (like me)?

[mingwandroid]

Historically we didn't provide any compilers, then we provided old ones but our R packages didn't use the those old ones instead relying on system ones (when you use install.packages()). That brings it's own problems (I don't know how capable your system ones are and anything you build with them will likely not work on other systems and loading R packages built with them could prevent binaries we built from loading or working).

Forcing our compilers on everyone who installs R on Linux might be the best plan though, but actually upstream R is moving towards providing prebuilt binaries (at least for Windows and macOS).

The other complication is that you are better off not using install.packages() for most things since we already provide prebuilt binary conda packages for a lot of the most popular R packages, so conda install r-something is better.

I am considering to patch install.packages() so that it tries installing a conda package first, and failing that, installs compilers if necessary before running the normal code-path.

Overall it's very difficult to know what is best here!

[corytu]

@mingwandroid some research on the Internet led me here. I am facing difficulties when installing R packages just like @nick-youngblut did. I see your suggestion for using conda skeleton cran approach instead of using R function install.packages, and I have done conda install gxx_linux-64. However, I keep failing at the conda build step.

Specifically, I want to install an R package called "RPostgreSQL", and here is what I have done:

conda install gxx_linux-64
conda skeleton cran r-rpostgresql
conda build r-rpostgresql
# I also tried:
# conda build --R 3.4.2 r-rpostgresql
# conda build --R=3.4.2 r-rpostgresql

The error includes:

make: /home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/bin/x86_64-conda_cos6-linux-gnu-cc: Command not found

, and

make: *** [/home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib/R/etc/Makeconf:160: RS-DBI.o] Error 127
ERROR: compilation failed for package ‘RPostgreSQL’

, and finally

subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/home/ytu/anaconda3/conda-bld/r-rpostgresql_1519544237983/work/conda_build.sh']' returned non-zero exit status 1.

You could find some other information in my question on Stack Overflow: Installing packages in IRkernel on Jupyter Notebook.

How should I do to solve this?

[zhou13]

I am on Archlinux and miniconda always gives me file not recognized: file format not recognized when gcc or ld is called. Removing compiler_compat/ld solves the issue.

[astrojuanlu]

@shreyasbapat and I just independently reproduced what @zhou13 mentioned on Arch Linux. I guess this is being tracked in ContinuumIO/anaconda-issues#11152 now.

[mingwandroid]

@Juanlu001
Would it be possible to try the package mentioned in my last comment on issue 11152 please?

[astrojuanlu]

Thanks @mingwandroid, doing conda install -c rdonnelly python=3.7.4 fixes the issue 👍

[chux0519]

@zhou13 thanks, it works

[github-actions]

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.