azure-pipelines: test packages on more configuration

[alaviss]

Attempts to test packages on more configurations to see if we can catch
out any issues related to changes related to those configurations.

Not sure if this is a good idea so only a draft for now.

[timotheecour]

• very much in favor and any failures in this PR are a justification that this PR is needed ;-)
• the only real drawback is it'll make it a bit slower to run CI given limited fixed number of parallel jobs (10 IIRC) ; but that's a problem worth throwing money at; https://azure.microsoft.com/en-us/pricing/details/devops/azure-devops-services/ shows 10 free parallels; and beyond that $40 per parallel job with unlimited minutes IIUC

[timotheecour]

looks like only 2 packages would break:

• https://github.com/trustable-code/NiGui /cc @trustable-code

2020-02-07T03:25:24.1787190Z �[32mPASS: �[36mhttps://github.com/icyphox/nicy C                           �[34m ( 2.33165503 secs)�[0m
2020-02-07T03:25:24.1787660Z �[1m�[31mFAIL: �[36mhttps://github.com/trustable-code/NiGui C�[0m
2020-02-07T03:25:24.1788120Z �[1m�[36mTest "https://github.com/trustable-code/NiGui" in category "nimble-packages"�[0m
2020-02-07T03:25:24.1788540Z �[1m�[31mFailure: reBuildFailed�[0m
2020-02-07T03:25:24.1788620Z package test failed

and https://github.com/status-im/nim-chronos/ (see status-im/nim-chronos#74)

maybe simply disable these conditionally in important_packages.nim, eg:

when not defined(osx):
  pkg "chronos", "", true

and then these could be fixed later

[simonkrauter]

What error appears with NiGui? I the comment above I just see "reBuildFailed".

[timotheecour]

What error appears with NiGui? I the comment above I just see "reBuildFailed".

relevant error:
2020-02-07T03:25:24.1795350Z could not load: libgtk-3.0.dylib

full logs:
https://dev.azure.com/nim-lang/255dfe86-e590-40bb-a8a2-3c0295ebdeb1/_apis/build/builds/2412/logs/78

2020-02-07T03:25:24.1787190Z �[32mPASS: �[36mhttps://github.com/icyphox/nicy C                           �[34m ( 2.33165503 secs)�[0m
2020-02-07T03:25:24.1787660Z �[1m�[31mFAIL: �[36mhttps://github.com/trustable-code/NiGui C�[0m
2020-02-07T03:25:24.1788120Z �[1m�[36mTest "https://github.com/trustable-code/NiGui" in category "nimble-packages"�[0m
2020-02-07T03:25:24.1788540Z �[1m�[31mFailure: reBuildFailed�[0m
2020-02-07T03:25:24.1788620Z package test failed
2020-02-07T03:25:24.1789330Z $ nim c -o:niguii -r src/nigui.nim

...

2020-02-07T03:25:24.1793050Z CC: nigui.nim
2020-02-07T03:25:24.1794230Z Hint: clang   -o /Users/runner/runners/2.164.7/work/1/s/pkgstemp/nigui/niguii  /Users/runner/.cache/nim/nigui_d/stdlib_assertions.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_io.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_system.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_math.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_unicode.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_strutils.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_posix.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_times.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_os.nim.c.o /Users/runner/.cache/nim/nigui_d/@mnigui@sprivate@sgtk3@sgtk3.nim.c.o /Users/runner/.cache/nim/nigui_d/stdlib_tables.nim.c.o /Users/runner/.cache/nim/nigui_d/@mnigui.nim.c.o  -lm   -ldl [Link]
2020-02-07T03:25:24.1794780Z Hint: 52736 LOC; 1.722 sec; 72.656MiB peakmem; Debug build; proj: /Users/runner/runners/2.164.7/work/1/s/pkgstemp/nigui/src/nigui.nim; out: /Users/runner/runners/2.164.7/work/1/s/pkgstemp/nigui/niguii [SuccessX]
2020-02-07T03:25:24.1794900Z Hint: /Users/runner/runners/2.164.7/work/1/s/pkgstemp/nigui/niguii  [Exec]
2020-02-07T03:25:24.1795350Z could not load: libgtk-3.0.dylib
2020-02-07T03:25:24.1795780Z compile with -d:nimDebugDlOpen for more information
2020-02-07T03:25:24.1796240Z Error: execution of an external program failed: '/Users/runner/runners/2.164.7/work/1/s/pkgstemp/nigui/niguii '
2020-02-07T03:25:24.1796330Z 
2020-02-07T03:33:49.2690630Z �[32mPASS: �[36mhttps://github.com/cheatfate/nimcrypto C                    �[34m (17.53565907 secs)�[0m

[simonkrauter]

How can we fix this? I'm not familiar with macOS. In simonkrauter/NiGui#63 it was proposed to compile with -d:gtk_quartz.

[timotheecour]

How can we fix this? I'm not familiar with macOS. In simonkrauter/NiGui#63 it was proposed to compile with -d:gtk_quartz.

• option 1: add when not defined(osx): pkg ... in this PR to unblock
• option 2: experiment with something like:

when defined(osx):
  pkg "nigui", "brew install gtk+3; nim c -o:niguii -r src/nigui.nim"
else: pkg "nigui", "nim c -o:niguii -r src/nigui.nim"

where I used brew install gtk+3; instead of brew install gtk+3 && in case package is already installed, to avoid failing there; there are more robust options too.

it works at least on my local OSX machine

• option 3 (somehow I don't like it as much since it looses the connection with nigui): add brew install gtk+3 inside azure-pipelines.yml for OSX

[EDIT]
this could be helped by nim-lang/RFCs#196 to improve CI speed

[Araq]

Ping @alaviss

[alaviss]

This PR is rather redundant now, we now have package testing for macOS and Linux upstream.

[timotheecour]

@alaviss but testing on windows should definitely be done; to avoid going in circles (see #13921 which moved from windows to linux), I suggest doing what I proposed here: #13954 (comment)

re-enable also windows in a way that doesn't slow down CI on average eg; multiple ways, eg: [...]