Bump the vcpkg builtin-baseline and remove most custom ports. (#4553)

[SC-37848](https://app.shortcut.com/tiledb-inc/story/37848/bump-the-vcpkg-builtin-baseline-and-remove-most-remaining-custom-ports) This PR bumps the vcpkg `builtin-baseline` which causes all of our dependencies to be updated to the most recent versions available on vcpkg. This removes the need for our custom ports and most were removed. The README of the ports folder was also updated to highlight that custom ports should be temporary. Apart from the remaining port overlays in the `ports` directory, the precise package versions we are using can be found in https://github.com/microsoft/vcpkg/blob/72010900b7cee36cea77aebb97695095c9358eaf/versions/baseline.json. --- TYPE: BUILD DESC: Update vcpkg version baseline to microsoft/vcpkg@7201090.
TileDB-Inc · Jan 25, 2024 · 2085d8c · 2085d8c
1 parent ca11ed2
commit 2085d8c
Show file tree

Hide file tree

Showing 117 changed files with 105 additions and 6,344 deletions.
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
@@ -56,6 +56,9 @@ jobs:
       - name: Checkout TileDB
         # v4 uses node 20 which is incompatible with the libc version of the manylinux image
         uses: actions/checkout@v3
+      - name: 'Homebrew setup'
+        run: brew install automake pkg-config
+        if: ${{ startsWith(matrix.os, 'macos-') == true }}  
       - name: Export GitHub Actions cache variables
         uses: actions/github-script@v6
         with:

diff --git a/.github/workflows/unit-test-runs.yml b/.github/workflows/unit-test-runs.yml
@@ -24,7 +24,7 @@ jobs:
       - uses: actions/checkout@v3
 
       - name: 'Homebrew setup'
-        run: brew install pkg-config
+        run: brew install automake pkg-config
         if: ${{ startsWith(matrix.os, 'macos-') == true }}
 
       # Configure required environment variables for vcpkg to use

diff --git a/cmake/Options/TileDBToolchain.cmake b/cmake/Options/TileDBToolchain.cmake
@@ -45,7 +45,7 @@ if (NOT DEFINED CMAKE_TOOLCHAIN_FILE)
         # Inspired from https://github.com/Azure/azure-sdk-for-cpp/blob/azure-core_1.10.3/cmake-modules/AzureVcpkg.cmake
         message("TILEDB_DISABLE_AUTO_VCPKG is not defined. Fetch a local copy of vcpkg.")
         # To help with resolving conflicts, when you update the commit, also update its date.
-        set(VCPKG_COMMIT_STRING ac2a14f35fcd57d7a38f09af75dd5258e96dd6ac) # 2023-11-16
+        set(VCPKG_COMMIT_STRING 72010900b7cee36cea77aebb97695095c9358eaf) # 2023-12-05
         message("Vcpkg commit string used: ${VCPKG_COMMIT_STRING}")
         include(FetchContent)
         FetchContent_Declare(

diff --git a/ports/README.md b/ports/README.md
@@ -1,22 +1,36 @@
-Port Overlays
-===
+# Port Overlays
 
-This directory contains the custom port overlays we're using for building depdencies. We're using overlays because some of our dependencies are quite old and getting working combinations from modern vcpkg was proving untenable. Overtime as our dependency versions are modernized, most if not all of the overlay ports will likely be removed.
+This directory contains the custom vcpkg port overlays we're using for building dependencies. 
 
-Updating a Port
----
+## Adding a Port
 
-The easiest way to update a port is to find the version of the port in the `microsoft/vcpkg` repository and extract it using a tool. So, for instance if we wanted to update our curl dependency from 7.80.0 to 7.84.0, we would look in the `path/to/microsoft/vcpkg/versions/c-/curl.json` file and find the tree-ish listed for 7.84.0. One thing to pay attention to here is that there can be multiple port versions for a given dependency version. So we want to pick the higest port version for the dependency at the version we are upgrading. In our hypothetical curl case that I may have just done, this gives us a treeish value of:
+> [!IMPORTANT]
+> Port overlays should be used as a temporary measure to fix issues with upstream ports or to add new ports that are not yet available to vcpkg. Once the upstream ports are fixed or added, the overlay ports should be removed.
 
-   `588fa4742c417db9d7c0f89e652b618296388d1e`
+If modifying an existing port, you have to first determine the version of the port in the `microsoft/vcpkg` repository and extract it using a tool. If the port does not have a version pin in the [`vcpkg.json`](../vcpkg.json) manifest (in either a `version>=` field or an entry in the `overrides` section), browse the `microsoft/vcpkg` repository in the commit specified in the `builtin-baseline` field and copy the port directory from there.
+
+If the port does have a version pin, and for instance we wanted to modify the curl port, which is in version 8.4.0, we would look in the [`versions/c-/curl.json` file](https://github.com/microsoft/vcpkg/blob/master/versions/c-/curl.json) and find the treeish listed for 8.4.0. One thing to pay attention to here is that there can be multiple port versions for a given dependency version. So we want to pick the highest port version for the dependency at the version we are upgrading. In our hypothetical curl case that I may have just done, this gives us a treeish value of:
+
+   `6125c796d6e2913a89a2996d7082375ce16b02dd`
 
 Once we have the tree-ish, we just need to be able to extract all of the files and store them in our overlay ports directory. The easiest approach for this is to use something like [this script](https://gist.github.com/mhl/498447/b245d48f2a22301415a30ca8a68241f96e0b3861) to do just that. If you put that script on your path (and remember to `chmod +x path/to/extract-tree-from-git.py`) you can follow these simple steps for updating the port:
 
 ```bash
 $ rm ports/curl/*
 $ cd path/to/microsoft/vcpkg
-$ extract-tree-from-git.py 588fa4742c417db9d7c0f89e652b618296388d1e path/to/tiledb/ports/curl/
+$ extract-tree-from-git.py 6125c796d6e2913a89a2996d7082375ce16b02dd path/to/tiledb/ports/curl/
 $ cd path/to/tiledb
 $ git add ports
 $ git commit
 ```
+
+After copying the port, add an entry to the table below. You should also contribute your changes to vcpkg and/or the upstream package repository.
+
+## List of port overlays
+
+|Port|Reason|
+|----|------|
+|`libmagic`|Updating to the upstream port deferred due to failures.|
+|`openssl`|Pinning to OpenSSL 1.1 until we can move to 3.0 in January 2024.|
+|`pcre2`|To be removed alongside libmagic.|
+|`azure-storage-common-cpp`|Patching to disable default features on libxml2 (https://github.com/Azure/azure-sdk-for-cpp/pull/5221).|
diff --git a/ports/aws-c-auth/portfile.cmake b/ports/aws-c-auth/portfile.cmake
diff --git a/ports/aws-c-auth/vcpkg.json b/ports/aws-c-auth/vcpkg.json
diff --git a/ports/aws-c-cal/portfile.cmake b/ports/aws-c-cal/portfile.cmake
diff --git a/ports/aws-c-cal/remove-libcrypto-messages.patch b/ports/aws-c-cal/remove-libcrypto-messages.patch
diff --git a/ports/aws-c-cal/vcpkg.json b/ports/aws-c-cal/vcpkg.json
diff --git a/ports/aws-c-common/disable-internal-crt-option.patch b/ports/aws-c-common/disable-internal-crt-option.patch
diff --git a/ports/aws-c-common/portfile.cmake b/ports/aws-c-common/portfile.cmake
diff --git a/ports/aws-c-common/vcpkg.json b/ports/aws-c-common/vcpkg.json
diff --git a/ports/aws-c-compression/portfile.cmake b/ports/aws-c-compression/portfile.cmake
diff --git a/ports/aws-c-compression/vcpkg.json b/ports/aws-c-compression/vcpkg.json
diff --git a/ports/aws-c-event-stream/portfile.cmake b/ports/aws-c-event-stream/portfile.cmake
diff --git a/ports/aws-c-event-stream/vcpkg.json b/ports/aws-c-event-stream/vcpkg.json