Merge bitcoin#17416: Appveyor improvement - text file for vcpkg packa…

…ge list

29eb039 Moves vcpkg list to a text file and updates the appveyor job and readme to use it. (Aaron Clauson)

Pull request description:

  bitcoin#17364 attempted to save a couple of minutes by skipping the `vcpkg` steps if the vcpkg install directory was already cached.

  The discussion in bitcoin#15382 highlights the approach used in bitcoin#17364 does not accommodate adding a new package.

  ~~This PR improves the approach to individually check whether  each vcpg package is installed rather than checking for the existence of the vcpkg install directory.~~

  This PR moves the list of required vcpkg packages into a separate file and uses changes to that file to invalidate the appveyor cache. Whenever the cache is invalidated the vcpkg sources will be updated, the vcpkg binary built and the required packages installed from the latest port files.

ACKs for top commit:
  MarcoFalke:
    ACK 29eb039

Tree-SHA512: 0c2a170f4e4b47ca0f9cef14f1e3892001b441a6d84f50bf5fd8a26bc4cdbd9358dfce7ef180d37150262e849650e9857d6b2bcd686964b963c3de6cd708a2f3

.appveyor.yml

@@ -7,19 +7,38 @@ clone_depth: 5
 environment:
   APPVEYOR_SAVE_CACHE_ON_ERROR: true
   CLCACHE_SERVER: 1
-  PACKAGES: berkeleydb boost-filesystem boost-signals2 boost-test libevent openssl rapidcheck zeromq
   PATH: 'C:\Python37-x64;C:\Python37-x64\Scripts;%PATH%'
   PYTHONUTF8: 1
 cache:
-- C:\tools\vcpkg\installed -> appveyor.yml
-- C:\Users\appveyor\clcache -> appveyor.yml, build_msvc\**, **\Makefile.am, **\*.vcxproj.in
+- C:\tools\vcpkg\installed -> build_msvc\vcpkg-packages.txt
+- C:\Users\appveyor\clcache -> .appveyor.yml, build_msvc\**, **\Makefile.am, **\*.vcxproj.in
+- C:\Qt5.9.8_x64_static_vs2019
 install:
 - cmd: pip install --quiet git+https://github.com/frerich/clcache.git@v4.2.0
 # Disable zmq test for now since python zmq library on Windows would cause Access violation sometimes.
 # - cmd: pip install zmq
-- cmd: vcpkg remove --outdated --recurse
-- cmd: vcpkg install --triplet %PLATFORM%-windows-static %PACKAGES% > NUL
-- cmd: del /s /q C:\Tools\vcpkg\installed\%PLATFORM%-windows-static\debug # Remove unused debug library
+# Powershell block below is to install the c++ dependencies via vcpkg. The pseudo code is:
+# 1. Check whether the vcpkg install directory exists (note that updating the vcpkg-packages.txt file
+#    will cause the appveyor cache rules to invalidate the directory)
+# 2. If the directory is missing:
+#    a. Update the vcpkg source (including port files) and build the vcpkg binary,
+#    b. Install the missing packages.
+- ps: |
+      $env:PACKAGES = Get-Content -Path build_msvc\vcpkg-packages.txt
+      Write-Host "vcpkg list: $env:PACKAGES"
+      if(!(Test-Path -Path ($env:VCPKG_INSTALL_PATH))) {
+          cd c:\tools\vcpkg
+          $env:GIT_REDIRECT_STDERR = '2>&1' # git is writing non-errors to STDERR when doing git pull. Send to STDOUT instead.
+          git pull origin master
+          .\bootstrap-vcpkg.bat
+          Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
+          .\vcpkg install --triplet $env:PLATFORM-windows-static $env:PACKAGES.split() > $null
+          cd "$env:APPVEYOR_BUILD_FOLDER"
+      }
+      else {
+        Write-Host "required vcpkg packages already installed."
+      }
+      c:\tools\vcpkg\vcpkg integrate install
 before_build:
 - ps:  clcache -M 536870912
 - cmd: python build_msvc\msvc-autogen.py

build_msvc/README.md

@@ -33,24 +33,32 @@ Building
 ---------------------
 The instructions below use vcpkg to install the dependencies.
 
-- Clone and vcpkg from the [github repository](https://github.com/Microsoft/vcpkg) and install as per the instructions in the main README.md.
-- Install the required packages (replace x64 with x86 as required):
-- Install the required dependencies with vcpkg:
+- Install [`vcpkg`](https://github.com/Microsoft/vcpkg).
+- Install the required packages (replace x64 with x86 as required). The list of required packages can be found in the `build_msvc\vcpkg-packages.txt` file. The PowerShell command below will work if run from the repository root directory and `vcpkg` is in the path. Alternatively the contents of the packages text file can be pasted in place of the `Get-Content` cmdlet.
 
 ```
-    PS >.\vcpkg install boost:x64-windows-static `
-    libevent:x64-windows-static `
-    openssl:x64-windows-static `
-    zeromq:x64-windows-static `
-    berkeleydb:x64-windows-static `
-    secp256k1:x64-windows-static `
-    leveldb:x64-windows-static
+PS >.\vcpkg install --triplet x64-windows-static $(Get-Content -Path build_msvc\vcpkg-packages.txt).split()
+```
+
+- Use Python to generate `*.vcxproj` from Makefile
+
+```
+PS >py -3 msvc-autogen.py
+```
+
+- An optional step is to adjust the settings in the build_msvc directory and the common.init.vcxproj file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set.
+
+- Build with Visual Studio 2017 or msbuild.
+
+```
+msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /p:PlatformToolset=v141 /t:build
+>>>>>>> a6f6333ba2 (Merge #17416: Appveyor improvement - text file for vcpkg package list)
 ```
 
 - Use Python to generate *.vcxproj from Makefile
 
 ```
-    PS >python msvc-autogen.py
+msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
 ```
 
-- Build in Visual Studio.
+- Build in Visual Studio.

build_msvc/vcpkg-packages.txt

@@ -0,0 +1 @@
+berkeleydb boost-filesystem boost-multi-index boost-signals2 boost-test boost-thread libevent openssl rapidcheck zeromq double-conversion