Versioning fails when 'CI' is 'true'

[bennyrowland]

I have written a package and would like to use quaternion as a dependency. On my local machine I installed it fine using pip and got everything working the way I wanted to and the tests passing. Then I pushed the new code to GitHub and started a TravisCI build, which failed with errors something like this:

Collecting numpy-quaternion (from suspect==0.3.0a0)
  Downloading numpy-quaternion-2017.03.16.21.51.57.dev242408302.tar.gz (42kB)
    100% |████████████████████████████████| 51kB 11.6MB/s 
    Complete output from command python setup.py egg_info:
    fatal: Not a git repository (or any of the parent directories): .git
    /tmp/pip-build-n4nfx3tp/numpy-quaternion/auto_version/__init__.py:87: UserWarning:
    The 'calculate_version' function failed to get the git version.Maybe your version of python (<2.7?) is too old.  Here's the exception:
    Command 'git show -s --format="%ci %h" HEAD' returned non-zero exit status 128
      warn(warning)
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-n4nfx3tp/numpy-quaternion/setup.py", line 35, in <module>
        version=calculate_version(validate, error_on_invalid),
      File "/tmp/pip-build-n4nfx3tp/numpy-quaternion/auto_version/__init__.py", line 91, in calculate_version
        raise e
      File "/tmp/pip-build-n4nfx3tp/numpy-quaternion/auto_version/__init__.py", line 57, in calculate_version
        git_revision = subprocess.check_output("""git show -s --format="%ci %h" HEAD""", shell=use_shell).decode('ascii').rstrip()
      File "/opt/python/3.5.2/lib/python3.5/subprocess.py", line 626, in check_output
        **kwargs).stdout
      File "/opt/python/3.5.2/lib/python3.5/subprocess.py", line 708, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command 'git show -s --format="%ci %h" HEAD' returned non-zero exit status 128
    Raising exception because environment variable 'CI' is "true"
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-n4nfx3tp/numpy-quaternion/
The command "pip install ." failed and exited with 1 during .

I am not sure but I think this might be because the auto_version assumes that if it is being installed on CI then it is being tested itself and so must be a git repo, whereas it is actually only a dependency without a repo of its own.

[moble]

In your .travis.yml file, I'm guessing you have a section with something like

install:
  - pip install numpy numpy-quaternion

Try changing that to

install:
  - pip install numpy
  - CI=false pip install numpy-quaternion

That sort of thing would work in bash by just changing the value of CI for that one command. Hopefully travis doesn't do so much "magic" that this is broken.

Unfortunately, auto_version is a little too dumb and used throughout too many of my projects to provide a more elegant solution. I'm afraid I don't really use pip myself, and I won't have a lot of time for this stuff any time soon (we're having twins!), so I'm sorry if this isn't the most robust or helpful solution, but I am open to suggestions that don't break my own process.

[bennyrowland]

Thanks for a very speedy response, at least. Actually in my .travis.yml file I have a pip install . which installs my own module and its dependencies from setup.py. I could no doubt break out the dependency separately as you suggest but as you note it is not super elegant, as it removes the automatic installation that pip otherwise provides.

From my point of view the obvious question is whether it is necessary to error out on CI if the git command fails. I guess that you put that in to make sure that it was working early on, but I don't see why that is a useful feature now. Not suggesting that it needs to be changed though, I can work around it as you suggest. I will close the issue for now.

[moble]

This is a legitimate problem, so I'll leave it open in case other people come looking for a way to solve it.

From my point of view the obvious question is whether it is necessary to error out on CI if the git command fails.

My answer is yes. The problem is that the CI process doesn't just test, but also uploads pip and conda packages, and those packages get their versions from this git information. Some times git does indeed fail, even now when things are generally working — usually due to subtle errors or changes in travis-ci. These particular version numbers fulfill a few requirements, including solving conda dependencies, and ensuring that data produced from my code (in which I include the version numbers of any packages I use) can be easily traced back to its origins down to the commit level. So I definitely don't want the packages going out if they can't get their version numbers. There's probably a better way to set the versions using git hooks, but it won't be easy for me to change at this point.

[moble]

This should be fixed since b732124, which simplified the version number so that it no longer uses auto_version (even though the submodule is still in the code...).

[jvgomez]

Should this work now then?
pip install --no-cache-dir numpy numpy-quaternion because this still fails for me

[Achllle]

This is still an issue @moble

[moble]

@Achllle No, this is not still an issue; the issue you appear to be talking about is jvgomez's question, which is a different issue. This issue was about the CI stuff. See #99.

[Achllle]

Alright, sorry that I commented on this issue. Seems like it's the same as #99 in that case.