Appveyor overhaul

[janschulz]

Mostly:

• use a slightly simpler appveyor script ...
• enables conda package builds again by using a workaround for the libpng
• syncs the conda-recipe with conda-forge, including tightening the versioned dependencies to the one in conda-forge (we have to because we otherwise would break when someone installed libpng from defaults :-/)
• use a better install line in the conda-recipe build scripts (which is the recommended default for setuptool based ones
• revert the wheel workaround on windows -> newer wheels work without this...
• check that the wheels really have no external dll dependencies
• install more optional dependencies so that more tests are run (cc: #6523)

[jenshnielsen]

Looks good, Any chance we can drop the patched bdist_whell now?

[janschulz]

Yes, that can also be gone now. Will add it here in this PR

[janschulz]

Meh:

[matplotlib-1.5.0+2326.gf349629-cp27-cp27m-win_amd64.whl]

λ dumpbin.exe /DEPENDENTS /NOLOGO _png.pyd

Dump of file _png.pyd

File Type: DLL

  Image has the following dependencies:

    zlib.dll
    MSVCP90.dll
    python27.dll
    MSVCR90.dll
    KERNEL32.dll

-> depends on zlib.dll, so probably doesn't work at the user if zlib isn't in PATH for some other reason :-(

[janschulz]

Ok, it builds and the tests pass but the coverage is going down :-(

The problem with non-static dependencies is fixed as far as I can see and the tests now include a test for that.

-> I would love to get a review, I don't have any more changes staged for this.

[jenshnielsen]

Don't worry about the coverage. It's comming from the travis builds and have been going up and down for that file all the time in various builds

[janschulz]

Ok, after installing inkscape and imagemagick the tests error out:

python tests.py
......KK.K.K.KK....SS...SSK.......SS...SSK.SS.SS..SS.....SSKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK.SS.SS.SS.SS...SS....SS.SS.SS.SS.SS.SS.SS.SS...SS.K.SS.SS.SS.SS.SS.....SS.SS.SS..SS.SS.SS.SS.SS.SS.SS.SS.SS.......SS........SS.SS.SS.SS......SS...SS.SS.SS.SS.SS.SS..SS.SS.SS...SS...........SS.SS.SS.SS.SS.SS.SS.SS.SS.SS.....SS...SS..SS.SS.SS.SS.............SS.SSc:\projects\matplotlib\lib\matplotlib\stackplot.py:95: RuntimeWarning: divide by zero encountered in true_divide
  inv_total = np.where(total > 0, 1./total, 0)
.SS.SS.SS.SS.SS.SS.SS....SS....................................................SSS........................SS....SSetProcessDpiAwareness failed: "COM error 0x80070005  (Unknown error 0x0ffffffff80070005)"

The test runtime of almost exactly 1h seem to suggest that the tests are stuck somewhere and run into a timeout :-( -> I'm removing the choko installs again...

[janschulz]

Ok, wasn't the choco lines, next try pillow... Please don't let it be miktex...

[cgohlke]

Re possible MiKTeX timeouts: check that MiKTeX is configured to "install missing packages on-the-fly", not "ask me first" (the default; opens a modal dialog).

[janschulz]

@cgohlke: Thank you, the miktex "ask" setting seems to have been the problem!

• old: OK (KNOWNFAIL=75, SKIP=1214)
• new without miktex: (KNOWNFAIL=82, SKIP=1214)
• new with miktex: OK (KNOWNFAIL=83, SKIP=1213)

-> It seems it still misses most of the tests due to missing tools :-( probably the image conversation util to convert svg/... to png

[jenshnielsen]

The images are converted using inkscape but I guess choko installing inkscape does not put it on path so it can be found by nose

[janschulz]

Current status: A failure on py3.4: https://ci.appveyor.com/project/mdboom/matplotlib/build/1.0.1712

======================================================================
FAIL: matplotlib.tests.test_backend_ps.test_savefig_to_stringio_eps_afm
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 152, in wrapped_callable
    func(*args, **kwargs)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 89, in test_savefig_to_stringio_eps_afm
    _test_savefig_to_stringio(format='eps', use_log=True)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 54, in _test_savefig_to_stringio
    assert values[0] == values[1]
AssertionError

----------------------------------------------------------------------
Ran 6123 tests in 587.735s

Next stop: trying to put inkscape into PATH thanks to the suggestion from @jenshnielsen

[jenshnielsen]

@janschulz I think that one is a random fluke. We are seeing it on Travis too

[janschulz]

Still something missing:

Ran 6123 tests in 565.587s 

OK (KNOWNFAIL=84, SKIP=1213)

I think I know why: the miktex packages only installs bat files in PATH which redirect to the exe in a subdir for most of the files (only a few get a wrapper written in GO -> it's 2MB and we have a lot of binaries in the miktex package, so only the most mportant binaries got that treatment) and I think subprocess can't call bat files without using a shell which it doesn't in the checks: https://github.com/matplotlib/matplotlib/blob/98a2e64338664fd8b000007b3843caae81837439/lib/matplotlib/__init__.py#L355

Oh, and another problem: inkscape comes in two flavors: inkscape.exe and inkscape.com. com seems to be the commandline app, exe the GUI:

λ C:\portabel\miniconda\pkgs\inkscape-0.91-0\Library\inkscape\inkscape.exe -V

λ C:\portabel\miniconda\pkgs\inkscape-0.91-0\Library\inkscape\inkscape.com -V
Inkscape 0.91 r13725 (Jan 30 2015)

-> unfortunately, .com must be explicitly asked for, a inkscape without shell=True only finds the exe one

[janschulz]

proper exe wrappers are comming: conda-forge/miktex-feedstock#8

[janschulz]

Current status:

A lot less skipped tests: (KNOWNFAIL=83, SKIP=609, failures=1). Tests now take 30min each instead of ~12 min -> we do a lot more image comparisons and also the conversion between image types :-(

But also a failure on py27, py33

https://ci.appveyor.com/project/mdboom/matplotlib/build/1.0.1721/job/4jt29336mqp6jx6g/artifacts

====================================================================== 
FAIL: matplotlib.tests.test_patches.test_wedge_range.test 
---------------------------------------------------------------------- 
Traceback (most recent call last):
  File "C:\Miniconda\envs\test-environment\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 55, in failer
    result = f(*args, **kwargs)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 267, in do_test
    '(RMS %(rms).3f)'%err)
ImageComparisonFailure: images not close: C:\projects\matplotlib\result_images\test_patches\wedge_range_pdf.png vs. C:\projects\matplotlib\result_images\test_patches\wedge_range-expected_pdf.png (RMS 1.617) 

Looking at the images (pdf versions, see the artifact in one of the failing appveyor runs), it needs a slightly higher tolerance?

[jenshnielsen]

I looked at the failed image. The diff seems insignificant so I am 👍 on changing the tolerance slightly

[janschulz]

ok, upped the tolerance on that test...

What is with the testing timeS: 4_30min=2h instead of 40min as before... I could install inkscape only on one of the tests, so that we get back to 1x30+3_15min... Or is 2h ok?

[jenshnielsen]

I think installing inkscape on only one of the nodes is probably the best solution at the moment.

[janschulz]

Any idea what's happening here (Travis -> linux):

 ======================================================================
FAIL: matplotlib.tests.test_backend_pdf.test_grayscale_alpha.test
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/matplotlib/matplotlib/venv/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/decorators.py", line 55, in failer
    result = f(*args, **kwargs)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/decorators.py", line 257, in do_test
    self._tol, in_decorator=True)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/compare.py", line 337, in compare_images
    rms = calculate_rms(expectedImage, actualImage)
  File "/home/travis/build/matplotlib/matplotlib/lib/matplotlib/testing/compare.py", line 248, in calculate_rms
    "actual size {1}".format(expectedImage.shape, actualImage.shape))
matplotlib.testing.exceptions.ImageComparisonFailure: image sizes do not match expected size: (432, 576, 3) actual size (346, 461, 3)

[jenshnielsen]

It seems to happen randomly on both Travis and appveyor. It's a fluke not related to any on this but we should probably try to get to the bottom of this.

[janschulz]

It seems the choco install if inkscape is cached, at least it's now available in in all test runs :-/

[janschulz]

set LIBRARY_LIB=%CONDA_DEFAULT_ENV%\Library\lib
mkdir lib || cmd /c "exit /b 0"
A subdirectory or file lib already exists.
copy %LIBRARY_LIB%\zlibstatic.lib lib\z.lib
The system cannot find the path specified.
Command exited with code 1

WTF? I installed the save zlib in a local env and it worked...?

[janschulz]

Ok, the problem was that conda changed the CONDA_DEFAULT_ENV variable from containing the full path to the env to the env name only. This meant that paths which were based on that env variable were of course failing... The new way seems to be CONDA_PREFIX...

[janschulz]

I opened conda/conda-build#1061 for the new "cannot find cl.exe" problem.

Using conda and conda-build isn't the easiest thing to do right now :-(

[QuLogic]

Isn't 32-bit more likely to have problems?

[janschulz]

not sure. That was the one I had to do fixes for in this iteration. The last time it was a different version. I think that's probably just random?

[QuLogic]

I think maybe @cgohlke or @matthew-brett said 32-bit required the most finagling to build packages.

[janschulz]

note that these controls only the image comparison tests for image formats other than png, png image tests (and all the rest of the tests) are run on all platforms.

[janschulz]

changed to py27/32bit

[janschulz]

There is a new conda version out, lets see if this fixes things (close and reopen)

[tacaswell]

@janschulz you seem to be fighting windows, multi-process, setuptools, and conda* simultaneously right now via appevyor.

If we ever meet in person I owe you 🍻 .

[janschulz]

Now it's this in only one of the runs:

C:\Miniconda-x64\conda-bld\work>call "C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcvarsall.bat" amd64 
Setting environment for using Microsoft Visual Studio 2008 x64 tools.
The input line is too long.
The syntax of the command is incorrect.
Command failed: C:\windows\system32\cmd.exe /c bld.bat
Command exited with code 1

[Kojoley]

May be this link will be helpful https://support.microsoft.com/en-us/kb/830473

The input line is too long.
The syntax of the command is incorrect.

[tacaswell]

Restarted the build to see if this is a real problem or something being flaky

[janschulz]

I'm starting to think that we should just remove the conda build again as long as it seems not reliable... :-(

[janschulz]

I've disabled the conda-build and removed the debugging output (PR for both is in #6682), so hopefully this succeeds now and can be merged...

[WeatherGod]

The tests are now all successful. Who should be the one to give the final review and check?

[QuLogic]

Don't think this needs to be changed?

[janschulz]

yep, will change

[tacaswell]

Do we version pin something to make sure we are using a fixed version of the wheels build code?

[janschulz]

@tacaswell No, they are basically build against the same code which is in conda-forge at this time.

[tacaswell]

I am a bit concerned about removing the bug workaround in setup.py which may bite users on windows that want to build wheels which have not updated.

I am happy with saying that people must have newer versions of setuptools (I assume that is where it comes from?), but should probably put in a check for that.

I would also be happy to be told I am confused about this and it is fine as-is.

[janschulz]

I saw basically two bugreports about the wheel issue: mine and one from cgohlke who also tried compiling mpl. So IMO this is pretty rare: noone in the right mind tries to compile mpl from source to a wheel, everyone just gets it from cgohlke :-)

Wheel is from it's own package (at least under conda) but gets probably installed per default (at least conda installs it automatically in each new python env)?

[tacaswell]

Fair enough.

[janschulz]

Yay :-) Thanks!

[tacaswell]

Thank you for fighting the good fight with appveyor!

[Kojoley]

Build times have increased from ~10 min (build <= 1.0.1922) to ~30 min (build >= 1.0.1924) on all four cases (not just one 833f903) after merging this.

[janschulz]

before:

Ran 6124 tests in 423.671s
OK (KNOWNFAIL=74, SKIP=1214)

after:

Ran 6125 tests in 1028.213s
OK (KNOWNFAIL=82, SKIP=609)

And 64bit and the lone 32 bit (which was instructed to run all image comp tests) have the same runtime and the same number of tests.

Should I remove the additional requirements (latex, ffmeg) on all but one build, so only one gets down to 609 skips vs 1200 in the old days?

[Kojoley]

On my machine inkscape is terribly slow, I hope some day librsvg would replace it.

[tacaswell]

librsvg has other issues like not implementing the svg spec correctly (https://bugzilla.gnome.org/show_bug.cgi?id=748565).

[jenshnielsen]

I think @mdboom mentioned that they had fixed that issue but probably not in a release yet