Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove pyenv from "conan new" CI templates for OSX builds #2385

Closed
3 tasks done
solvingj opened this issue Jan 25, 2018 · 3 comments
Closed
3 tasks done

Remove pyenv from "conan new" CI templates for OSX builds #2385

solvingj opened this issue Jan 25, 2018 · 3 comments
Milestone

Comments

@solvingj
Copy link
Contributor

solvingj commented Jan 25, 2018

After auditing and trying to streamline our CI process for OSX, we discovered that the TravisCI templates that have been generated by the conan new command with various --ci_ options have included installing/updating/using pyenv for OSX. Code such as this:

    brew outdated pyenv || brew upgrade pyenv
    brew install pyenv-virtualenv
    ...
    pyenv install 2.7.10
    pyenv virtualenv 2.7.10 conan
    pyenv rehash
    pyenv activate conan

After bringing it up with the Conan team, it was determined that this code could be removed from CI templates moving forward, as it was a workaround for a legacy issue which no longer exists.

In our testing of the boost_build package on OSX, removing this code dramatically reduced the package build time (approximately 40%).

Before (5:03)
https://circleci.com/gh/bincrafters/conan-boost_build/7
After (3:08)
https://circleci.com/gh/bincrafters/conan-boost_build/10

2 out of the 5 minutes of the job were spent modifying the environment for pyenv.

In the grand scheme of things for Conan packaging of OSS libraries for OSX, this discovery seems to be substantial. A 2-minute build-time tax on all MacOS builds is huge. The impact is compounded by the fact that the TravisCI MacOS queue for OSS is shared across all OSS projects. This queue has been notorious for being constantly oversubscribed, resulting in extremely long wait times for builds. The situation has continued to git worse over the past year, and the Travis CI team is currently struggling meet the demand.

It's expected that this PYENV code is likely widespread throughout all the Conan OSS community recipes, affecting virtually all packages which are built for OSX with Travis CI, and thus substantially affecting the shared Travis queue for OSX. For this reason, we recommend broadcasting a summary of the situation to the user community ASAP, and advising them to remove the related code from any existing CI scripts.

  • I've read the CONTRIBUTING guide.
  • I've specified the Conan version, operating system version and any tool that can be relevant.
  • I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.
@solvingj
Copy link
Contributor Author

Note that our current templates for Travis are now reduced to the following. Whether or not we should just embed this code in the .travis.yml file directly is an open and valid question at this time.

install.sh

#!/bin/bash

set -e
set -x
if [[ "$(uname -s)" == 'Darwin' ]]; then
    brew update || brew update
    brew install cmake || true
fi
pip install conan --upgrade
pip install conan_package_tools bincrafters_package_tools
conan user

run.sh

#!/bin/bash

set -e
set -x

python build.py

@lasote
Copy link
Contributor

lasote commented Feb 1, 2018

Hi @solvingj can you show me some build running with that proposed configuration? is not working ok for me. thanks.

@solvingj
Copy link
Contributor Author

solvingj commented Feb 3, 2018

I believe we've concluded that we cannot remove the pyenv from Travis macos afterall. it looks like travis's standard image doesn't have the same stuff as circleci, so this only works for cicrcle ci at the moment. apologies for causing the confusion.

@solvingj solvingj closed this as completed Feb 4, 2018
@ghost ghost removed the to do label Feb 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants