Appveyor overhaul #6520

jankatins · 2016-06-01T13:05:55Z

Mostly:

use a slightly simpler appveyor script ...
enables conda package builds again by using a workaround for the libpng
syncs the conda-recipe with conda-forge, including tightening the versioned dependencies to the one in conda-forge (we have to because we otherwise would break when someone installed libpng from defaults :-/)
use a better install line in the conda-recipe build scripts (which is the recommended default for setuptool based ones
revert the wheel workaround on windows -> newer wheels work without this...
check that the wheels really have no external dll dependencies
install more optional dependencies so that more tests are run (cc: matplotlib-2.0.0b1 test errors on Windows #6523)

jenshnielsen · 2016-06-01T13:09:08Z

Looks good, Any chance we can drop the patched bdist_whell now?

jankatins · 2016-06-01T13:17:51Z

Yes, that can also be gone now. Will add it here in this PR

jankatins · 2016-06-01T18:17:27Z

Meh:

[matplotlib-1.5.0+2326.gf349629-cp27-cp27m-win_amd64.whl]

λ dumpbin.exe /DEPENDENTS /NOLOGO _png.pyd

Dump of file _png.pyd

File Type: DLL

  Image has the following dependencies:

    zlib.dll
    MSVCP90.dll
    python27.dll
    MSVCR90.dll
    KERNEL32.dll

-> depends on zlib.dll, so probably doesn't work at the user if zlib isn't in PATH for some other reason :-(

jankatins · 2016-06-03T01:58:55Z

Ok, it builds and the tests pass but the coverage is going down :-(

The problem with non-static dependencies is fixed as far as I can see and the tests now include a test for that.

-> I would love to get a review, I don't have any more changes staged for this.

jenshnielsen · 2016-06-03T08:48:11Z

Don't worry about the coverage. It's comming from the travis builds and have been going up and down for that file all the time in various builds

jankatins · 2016-06-03T19:48:04Z

Ok, after installing inkscape and imagemagick the tests error out:

python tests.py
......KK.K.K.KK....SS...SSK.......SS...SSK.SS.SS..SS.....SSKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK.SS.SS.SS.SS...SS....SS.SS.SS.SS.SS.SS.SS.SS...SS.K.SS.SS.SS.SS.SS.....SS.SS.SS..SS.SS.SS.SS.SS.SS.SS.SS.SS.......SS........SS.SS.SS.SS......SS...SS.SS.SS.SS.SS.SS..SS.SS.SS...SS...........SS.SS.SS.SS.SS.SS.SS.SS.SS.SS.....SS...SS..SS.SS.SS.SS.............SS.SSc:\projects\matplotlib\lib\matplotlib\stackplot.py:95: RuntimeWarning: divide by zero encountered in true_divide
  inv_total = np.where(total > 0, 1./total, 0)
.SS.SS.SS.SS.SS.SS.SS....SS....................................................SSS........................SS....SSetProcessDpiAwareness failed: "COM error 0x80070005  (Unknown error 0x0ffffffff80070005)"

The test runtime of almost exactly 1h seem to suggest that the tests are stuck somewhere and run into a timeout :-( -> I'm removing the choko installs again...

jankatins · 2016-06-04T10:17:08Z

Ok, wasn't the choco lines, next try pillow... Please don't let it be miktex...

cgohlke · 2016-06-04T18:21:22Z

Re possible MiKTeX timeouts: check that MiKTeX is configured to "install missing packages on-the-fly", not "ask me first" (the default; opens a modal dialog).

jankatins · 2016-06-06T10:24:23Z

@cgohlke: Thank you, the miktex "ask" setting seems to have been the problem!

old: OK (KNOWNFAIL=75, SKIP=1214)
new without miktex: (KNOWNFAIL=82, SKIP=1214)
new with miktex: OK (KNOWNFAIL=83, SKIP=1213)

-> It seems it still misses most of the tests due to missing tools :-( probably the image conversation util to convert svg/... to png

jenshnielsen · 2016-06-06T11:22:15Z

The images are converted using inkscape but I guess choko installing inkscape does not put it on path so it can be found by nose

jankatins · 2016-06-06T12:45:06Z

Current status: A failure on py3.4: https://ci.appveyor.com/project/mdboom/matplotlib/build/1.0.1712

======================================================================
FAIL: matplotlib.tests.test_backend_ps.test_savefig_to_stringio_eps_afm
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Miniconda3-x64\envs\test-environment\lib\site-packages\nose\case.py", line 198, in runTest
    self.test(*self.arg)
  File "c:\projects\matplotlib\lib\matplotlib\testing\decorators.py", line 152, in wrapped_callable
    func(*args, **kwargs)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 89, in test_savefig_to_stringio_eps_afm
    _test_savefig_to_stringio(format='eps', use_log=True)
  File "c:\projects\matplotlib\lib\matplotlib\tests\test_backend_ps.py", line 54, in _test_savefig_to_stringio
    assert values[0] == values[1]
AssertionError

----------------------------------------------------------------------
Ran 6123 tests in 587.735s

Next stop: trying to put inkscape into PATH thanks to the suggestion from @jenshnielsen

jenshnielsen · 2016-06-06T13:02:59Z

@JanSchulz I think that one is a random fluke. We are seeing it on Travis too

This is mainly because the additional tests (which are otherwise skipped) increase the runtime of one test run (we have 4) from 13min to 30mins and thereby increasing one complete Appveyor run from 40min to 2 hours. With this change, we only have 30+3*13min. The py27 test was chosen because up it made the most problems during this Appveyor/Windows fixing round...

jankatins · 2016-07-03T15:21:39Z

I've disabled the conda-build and removed the debugging output (PR for both is in #6682), so hopefully this succeeds now and can be merged...

WeatherGod · 2016-07-05T15:15:03Z

The tests are now all successful. Who should be the one to give the final review and check?

QuLogic · 2016-07-05T19:23:45Z

ci/conda_recipe/condaversion.patch

@@ -11,4 +11,4 @@ index 8af8b6d..4e4f9d2 100644
 +    f.write(__version__)

 # These are the packages in the order we want to display them.  This
- # list may contain strings to create section headers for the display.
+ # list may contain strings to create section headers for the display.


Don't think this needs to be changed?

yep, will change

Again, don't use `load_setuptools()` but the __conda_version__.txt file to get the final version of the package from the git commit id.

That seems to be the worst platform, so should trigger the most issues :-)

tacaswell · 2016-07-06T12:54:49Z

Do we version pin something to make sure we are using a fixed version of the wheels build code?

jankatins · 2016-07-06T13:27:25Z

@tacaswell No, they are basically build against the same code which is in conda-forge at this time.

tacaswell · 2016-07-06T13:49:49Z

I am a bit concerned about removing the bug workaround in setup.py which may bite users on windows that want to build wheels which have not updated.

I am happy with saying that people must have newer versions of setuptools (I assume that is where it comes from?), but should probably put in a check for that.

I would also be happy to be told I am confused about this and it is fine as-is.

jankatins · 2016-07-06T23:58:04Z

I saw basically two bugreports about the wheel issue: mine and one from cgohlke who also tried compiling mpl. So IMO this is pretty rare: noone in the right mind tries to compile mpl from source to a wheel, everyone just gets it from cgohlke :-)

Wheel is from it's own package (at least under conda) but gets probably installed per default (at least conda installs it automatically in each new python env)?

tacaswell · 2016-07-07T00:02:16Z

Fair enough.

jankatins · 2016-07-07T00:32:46Z

Yay :-) Thanks!

tacaswell · 2016-07-07T00:42:25Z

Thank you for fighting the good fight with appveyor!

Kojoley · 2016-07-10T13:41:54Z

Build times have increased from ~10 min (build <= 1.0.1922) to ~30 min (build >= 1.0.1924) on all four cases (not just one 833f903) after merging this.

jankatins · 2016-07-10T17:21:10Z

before:

Ran 6124 tests in 423.671s
OK (KNOWNFAIL=74, SKIP=1214)

after:

Ran 6125 tests in 1028.213s
OK (KNOWNFAIL=82, SKIP=609)

And 64bit and the lone 32 bit (which was instructed to run all image comp tests) have the same runtime and the same number of tests.

Should I remove the additional requirements (latex, ffmeg) on all but one build, so only one gets down to 609 skips vs 1200 in the old days?

Kojoley · 2016-07-10T20:46:01Z

On my machine inkscape is terribly slow, I hope some day librsvg would replace it.

tacaswell · 2016-07-11T23:48:18Z

librsvg has other issues like not implementing the svg spec correctly (https://bugzilla.gnome.org/show_bug.cgi?id=748565).

jenshnielsen · 2016-07-12T08:21:48Z

I think @mdboom mentioned that they had fixed that issue but probably not in a release yet

mdboom added the status: needs review label Jun 1, 2016

jankatins force-pushed the appveyor_overhaul branch from e124896 to 8b0198c Compare June 1, 2016 13:25

jankatins force-pushed the appveyor_overhaul branch 12 times, most recently from 361164b to 22be5ae Compare June 2, 2016 21:05

jankatins mentioned this pull request Jun 2, 2016

MNT: Update pinned versions. conda-forge/matplotlib-feedstock#29

Merged

jankatins force-pushed the appveyor_overhaul branch from ea676d3 to 22c7f7b Compare June 2, 2016 22:57

jankatins mentioned this pull request Jun 3, 2016

matplotlib-2.0.0b1 test errors on Windows #6523

Closed

jankatins force-pushed the appveyor_overhaul branch from 129a9e8 to a4b2e1a Compare June 3, 2016 14:06

jankatins force-pushed the appveyor_overhaul branch 2 times, most recently from e25b820 to e7c9b56 Compare July 3, 2016 15:19

jankatins mentioned this pull request Jul 3, 2016

DO NOT MERGE: conda-build failure on appveyor #6682

Closed

QuLogic reviewed Jul 5, 2016
View reviewed changes

jankatins added 3 commits July 5, 2016 21:31

conda-recipe: Readd condaversion.patch

34d64e5

Again, don't use `load_setuptools()` but the __conda_version__.txt file to get the final version of the package from the git commit id.

Fix local windows build script due to conda changes

e0bccef

Appveyor: disable building conda packages as they fail the build

32fb993

jankatins force-pushed the appveyor_overhaul branch from e7c9b56 to 32fb993 Compare July 5, 2016 19:32

Appveyor: run pdf/svg image tests in py27/32bit

61bdf00

That seems to be the worst platform, so should trigger the most issues :-)

tacaswell merged commit b38f558 into matplotlib:master Jul 7, 2016

tacaswell removed the status: needs review label Jul 7, 2016

tacaswell mentioned this pull request Jul 7, 2016

appveyor is failing on pre-install #6657

Closed

QuLogic added OS: Microsoft topic: testing labels Jul 7, 2016

jankatins mentioned this pull request Aug 9, 2016

Add conda-wrappers recipe conda-forge/staged-recipes#1205

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Appveyor overhaul #6520

Appveyor overhaul #6520

jankatins commented Jun 1, 2016 •

edited

jenshnielsen commented Jun 1, 2016

jankatins commented Jun 1, 2016

jankatins commented Jun 1, 2016 •

edited

jankatins commented Jun 3, 2016 •

edited

jenshnielsen commented Jun 3, 2016

jankatins commented Jun 3, 2016

jankatins commented Jun 4, 2016

cgohlke commented Jun 4, 2016

jankatins commented Jun 6, 2016 •

edited

jenshnielsen commented Jun 6, 2016

jankatins commented Jun 6, 2016

jenshnielsen commented Jun 6, 2016

jankatins commented Jul 3, 2016 •

edited

WeatherGod commented Jul 5, 2016

QuLogic Jul 5, 2016

jankatins Jul 5, 2016

tacaswell commented Jul 6, 2016

jankatins commented Jul 6, 2016

tacaswell commented Jul 6, 2016

jankatins commented Jul 6, 2016 •

edited

tacaswell commented Jul 7, 2016

jankatins commented Jul 7, 2016

tacaswell commented Jul 7, 2016

Kojoley commented Jul 10, 2016 •

edited

jankatins commented Jul 10, 2016 •

edited

Kojoley commented Jul 10, 2016

tacaswell commented Jul 11, 2016

jenshnielsen commented Jul 12, 2016

Appveyor overhaul #6520

Appveyor overhaul #6520

Conversation

jankatins commented Jun 1, 2016 • edited

jenshnielsen commented Jun 1, 2016

jankatins commented Jun 1, 2016

jankatins commented Jun 1, 2016 • edited

jankatins commented Jun 3, 2016 • edited

jenshnielsen commented Jun 3, 2016

jankatins commented Jun 3, 2016

jankatins commented Jun 4, 2016

cgohlke commented Jun 4, 2016

jankatins commented Jun 6, 2016 • edited

jenshnielsen commented Jun 6, 2016

jankatins commented Jun 6, 2016

jenshnielsen commented Jun 6, 2016

jankatins commented Jul 3, 2016 • edited

WeatherGod commented Jul 5, 2016

QuLogic Jul 5, 2016

Choose a reason for hiding this comment

jankatins Jul 5, 2016

Choose a reason for hiding this comment

tacaswell commented Jul 6, 2016

jankatins commented Jul 6, 2016

tacaswell commented Jul 6, 2016

jankatins commented Jul 6, 2016 • edited

tacaswell commented Jul 7, 2016

jankatins commented Jul 7, 2016

tacaswell commented Jul 7, 2016

Kojoley commented Jul 10, 2016 • edited

jankatins commented Jul 10, 2016 • edited

Kojoley commented Jul 10, 2016

tacaswell commented Jul 11, 2016

jenshnielsen commented Jul 12, 2016

jankatins commented Jun 1, 2016 •

edited

jankatins commented Jun 1, 2016 •

edited

jankatins commented Jun 3, 2016 •

edited

jankatins commented Jun 6, 2016 •

edited

jankatins commented Jul 3, 2016 •

edited

jankatins commented Jul 6, 2016 •

edited

Kojoley commented Jul 10, 2016 •

edited

jankatins commented Jul 10, 2016 •

edited