T361228 - Release WMDE.20

.github/workflows/_build_test.yml

@@ -117,15 +117,15 @@ jobs:
       matrix:
         # TODO: can we get this from the environment to not have it hardcoded in the pipeline code?
         version:
-          # Latest 1.38
-          - WMDE12
-          - WMDE12_BUNDLE
           # Latest 1.39
           - WMDE15
           - WMDE15_BUNDLE
           # Latest 1.40
           - WMDE16
           - WMDE16_BUNDLE
+          # Previous 1.41
+          - WMDE17
+          - WMDE17_BUNDLE
 
     steps:
       - uses: actions/checkout@v4

CHANGES.md

@@ -2,6 +2,16 @@
 
 This file is only intended to serve the developers of this repository and of the releases. This provides an overview of the releases that have been made using this release pipeline.
 
+## April 2024: Security releases for 1.41.1
+
+- [MediaWiki release announcement: 1.41.1](https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/V3WXEPXV2DU6WTVEKK4XHW4QXD5OFKD7/)
+- [MediaWiki full release notes](https://www.mediawiki.org/wiki/Release_notes/1.41#MediaWiki_1.41.1)
+- [Wikibase Suite Release task T361228](https://phabricator.wikimedia.org/T361228)
+
+| Suite Version | Version Info | Date available | Release task |
+| --- | --- | --- | --- |
+| wmde.20 ([spec](https://github.com/wmde/wikibase-release-pipeline/blob/wmde.20/variables.env)) | MediaWiki (1.41.1, PHP 8.2.15, debian/apache 2.4.57-2), WDQS (0.3.137) | 08 April 2024 | [T361228](https://phabricator.wikimedia.org/T361228) |
+
 ## March 2024: Mayor releases for 1.41.0
 
 - [MediaWiki release announcement: 1.41.0](https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/OMDFHJ2SKKJH775RW4UTC754OY4TP7UU/)
@@ -10,7 +20,7 @@ This file is only intended to serve the developers of this repository and of the
 
 | Suite Version | Version Info | Date available | Release task |
 | --- | --- | --- | --- |
-| wmde.17 ([spec](https://github.com/wmde/wikibase-release-pipeline/blob/wmde.17/variables.env)) | MediaWiki (1.41.0, PHP 8.2.15, debian/apache 2.4.57-2), WDQS (0.3.137) | XX March 2024 | [T354644](https://phabricator.wikimedia.org/T354644) |
+| wmde.17 ([spec](https://github.com/wmde/wikibase-release-pipeline/blob/wmde.17/variables.env)) | MediaWiki (1.41.0, PHP 8.2.15, debian/apache 2.4.57-2), WDQS (0.3.137) | 02 April 2024 | [T354644](https://phabricator.wikimedia.org/T354644) |
 
 ## February 2024: Security releases for 1.40.2
 
@@ -22,6 +32,14 @@ This file is only intended to serve the developers of this repository and of the
 | --- | --- | --- | --- | --- |
 | wmde.16 ([spec](https://github.com/wmde/wikibase-release-pipeline/blob/wmde.16/variables.env)) | MediaWiki (1.40.2, PHP 8.1.27, debian/apache 2.4.57-2), WDQS (0.3.137) | 21 February 2024 | [7976683909](https://github.com/wmde/wikibase-release-pipeline/actions/runs/7976683909) | [T354645](https://phabricator.wikimedia.org/T354645) |
 
+## February 2024: Security releases for 1.39.6
+
+- [WMDE 15 spec](https://github.com/wmde/wikibase-release-pipeline/blob/wmde.15/variables.env)
+- [MediaWiki security and maintenance release announcement: 1.35.14 / 1.39.6 / 1.40.2](https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/TDBUBCCOQJUT4SCHJNPHKQNPBUUETY52/)
+- [MediaWiki full release notes](https://www.mediawiki.org/wiki/Release_notes/1.39)
+- [Run number 7887239057](https://github.com/wmde/wikibase-release-pipeline/actions/runs/7887239057/)
+- [Release task T354646](https://phabricator.wikimedia.org/T354646)
+
 ## November 2023: First 1.40 release
 
 - [MediaWiki release announcement 1.40](https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/H46H5ZYZG2PYUQ5STK7NWKF7GXYW7H6B/)

build/WDQS/Dockerfile

@@ -71,7 +71,7 @@ WORKDIR /wdqs
 
 COPY --chown=blazegraph:blazegraph wait-for-it.sh entrypoint.sh runBlazegraph.sh runUpdate.sh /
 COPY --chown=blazegraph:blazegraph mwservices.json /templates/mwservices.json
-COPY --chown=blazegraph:blazegraph RWStore.properties whitelist.txt logback.xml /wdqs/
+COPY --chown=blazegraph:blazegraph RWStore.properties allowlist.txt logback.xml /wdqs/
 
 # TODO this shouldn't be needed, but CI currently doesnt check for the +x bit, which is why this line is here
 RUN chmod +x /wdqs/runUpdate.sh

build/WDQS/README.md

@@ -41,7 +41,7 @@ Note on `BLAZEGRAPH_EXTRA_OPTS`: These are options that are directly passed to t
 
 | File | Description |
 | --- | --- |
-| `/wdqs/whitelist.txt` | SPARQL endpoints that are allowed for federation |
+| `/wdqs/allowlist.txt` | SPARQL endpoints that are allowed for federation |
 | `/wdqs/RWStore.properties` | Properties for the service |
 | `/templates/mwservices.json` | Template for MediaWiki services (substituted to `/wdqs/mwservices.json` at runtime) |
 

build/Wikibase/Dockerfile

@@ -17,10 +17,6 @@ RUN git clone https://gerrit.wikimedia.org/r/mediawiki/extensions/Wikibase /tmp/
 FROM ${MEDIAWIKI_IMAGE_URL} as collector
 COPY --from=git-checkout /tmp/wikibase /var/www/html/extensions/Wikibase
 
-COPY ./fix-do-not-add-stdport-in-detectServer.patch /tmp/fix-do-not-add-stdport-in-detectServer.patch
-WORKDIR /var/www/html
-RUN patch -Np1 < /tmp/fix-do-not-add-stdport-in-detectServer.patch
-
 # ###########################################################################
 # hadolint ignore=DL3006
 FROM ${COMPOSER_IMAGE_URL} as composer

docs/topics/release-overview.md

@@ -2,6 +2,4 @@
 
 Wikibase and related software gets release through this repository. The process for releasing involves defining the release in this repository, testing all the components of the release together and finally publishing them.
 
-The diagram below describes the general steps involved in making a release.
-
-![Release workflow overview](../diagrams/output/release-workflow.svg "Release workflow overview")
+Find the checklist to go through for a release in the respective phabricator tasks. E.g. https://phabricator.wikimedia.org/T354644 for WMDE.17.

test/specs/repo/queryservice.ts

@@ -275,4 +275,58 @@ describe( 'QueryService', function () {
 				.$( 'span' )
 		).toHaveText( '1' );
 	} );
+
+	it( 'Should show results from a page in allowlist.txt', async function () {
+		// We don't currently have a way for the example suite to pass tests with breaking changes
+		// Please see T361575 for more info
+		if ( testEnv.settings.name === 'example' ) {
+			this.skip();
+		}
+
+		await QueryServiceUIPage.open( `
+			PREFIX wikidata_wd: <http://www.wikidata.org/entity/>
+			PREFIX wikidata_wdt: <http://www.wikidata.org/prop/direct/>
+
+			SELECT * WHERE {
+				service <https://query.wikidata.org/sparql> {
+					?wd wikidata_wdt:P31 wikidata_wd:Q976981 .
+					?wd wikidata_wdt:P31 ?type .
+					optional { ?wd rdfs:label ?label. filter(lang(?label) = "en") }
+					optional { ?type rdfs:label ?typelabel. filter(lang(?typelabel) = "en") }
+				}
+			}
+
+			LIMIT 5
+		` );
+
+		await QueryServiceUIPage.submit();
+
+		await expect(
+			QueryServiceUIPage.resultTable.$( 'tbody' ).$$( 'tr' )
+		).resolves.toHaveLength( 5 );
+	} );
+
+	it( 'Should show error from a page not in allowlist.txt', async function () {
+		// Returns results if https://wikibase.world/query/sparql added to allowlist.txt
+		await QueryServiceUIPage.open( `
+			PREFIX wdt: <https://wikibase.world/prop/direct/>
+			PREFIX wd: <https://wikibase.world/entity/>
+
+			SELECT * WHERE {
+				service <https://wikibase.world/query/sparql> {
+					?item wdt:P3 wd:Q10 .
+					?item wdt:P1 ?url .
+					?item wdt:P13 wd:Q54 .
+				}
+			}
+
+			LIMIT 5
+		` );
+
+		await QueryServiceUIPage.submit();
+
+		await expect( $( 'div#query-error' ) ).toHaveText(
+			/Service URI https:\/\/wikibase\.world\/query\/sparql is not allowed/
+		);
+	} );
 } );

test/suites/upgrade/versions.ts

@@ -26,6 +26,9 @@ export const versions = {
 	WMDE17: 'wikibase/wikibase:1.41.0-wmde.17',
 	WMDE17_BUNDLE: 'wikibase/wikibase-bundle:1.41.0-wmde.17',
 
+	WMDE20: 'wikibase/wikibase:1.41.1-wmde.20',
+	WMDE20_BUNDLE: 'wikibase/wikibase-bundle:1.41.1-wmde.20',
+
 	LOCAL_BUILD: 'wikibase/wikibase',
 	LOCAL_BUILD_BUNDLE: 'wikibase/wikibase-bundle'
 };

variables.env

@@ -7,7 +7,7 @@
 # Append -prerelease while working on releases.
 # Remove -prerelease when ready to merge release to the release branch.
 # Call it main on the main branch.
-WMDE_RELEASE_VERSION=wmde.17
+WMDE_RELEASE_VERSION=wmde.20
 
 
 # ##############################################################################
@@ -16,8 +16,8 @@ WMDE_RELEASE_VERSION=wmde.17
 # Update only patch versions for security releases.
 # Choose latest version for major releases.
 # https://hub.docker.com/_/mediawiki
-MEDIAWIKI_VERSION=1.41.0
-MEDIAWIKI_IMAGE_URL=mediawiki:1.41.0
+MEDIAWIKI_VERSION=1.41.1
+MEDIAWIKI_IMAGE_URL=mediawiki:1.41.1
 
 
 # ##############################################################################
@@ -112,31 +112,31 @@ DEBIAN_IMAGE_URL=debian:bookworm-slim
 # Shouldn't require much of a review.
 #
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/refs/heads/REL1_41
-WIKIBASE_COMMIT=c42002be38fa2756b8e475b38b1e6ab86d4f53ce
+WIKIBASE_COMMIT=60cd0cff1df38876a5050b83bc3800689933d2bd
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Babel/+/refs/heads/REL1_41
-BABEL_COMMIT=e110eaa7293796ae3cf149177551997cc2009095
+BABEL_COMMIT=bbc95078051d30716240fa8fcc99936b1cb6eed1
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/cldr/+/refs/heads/REL1_41
 CLDR_COMMIT=cad7768c4bd092318067fe15d674bf17ff965e2a
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/CirrusSearch/+/refs/heads/REL1_41
-CIRRUSSEARCH_COMMIT=f304f9a27ee1d99d9582fc222596e6c0e49d4ef9
+CIRRUSSEARCH_COMMIT=347c4f9c650114b726cd3c37f825ee481074ae76
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/ConfirmEdit/+/refs/heads/REL1_41
-CONFIRMEDIT_COMMIT=9fb5930d4485a72001016ad62e63e457addb91f2
+CONFIRMEDIT_COMMIT=9ff45ad2ac995bf7a7b7125556aba2742910623b
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Elastica/+/refs/heads/REL1_41
 ELASTICA_COMMIT=24f94b80b51555e5e5b30b0633d215ee001650d4
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/EntitySchema/+/refs/heads/REL1_41
 ENTITYSCHEMA_COMMIT=85ea615968509afab9214c572ac7aedc7dda4424
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Nuke/+/refs/heads/REL1_41
-NUKE_COMMIT=8e969d99c09faac3ec64b35ce5553e7ec8c87b28
+NUKE_COMMIT=74498ea565ad2cfa893fd852b377fa42cd4b5e2d
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/OAuth/+/refs/heads/REL1_41
-OAUTH_COMMIT=8cdf22304b8382ef0015da9ef3210f90eb7687ce
+OAUTH_COMMIT=c43d994be0076c0ede603049745a099897eead4b
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Scribunto/+/refs/heads/REL1_41
 SCRIBUNTO_COMMIT=825e1203b060850804885041dcd5d744f81a5df0
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/SyntaxHighlight_GeSHi/+/refs/heads/REL1_41
 SYNTAXHIGHLIGHT_GESHI_COMMIT=40908dd4204915eb20f1e43161c0489c7571f37d
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/UniversalLanguageSelector/+/refs/heads/REL1_41
-UNIVERSALLANGUAGESELECTOR_COMMIT=47158fbee4ce6870133c29f1ed6affdb2eb31acc
+UNIVERSALLANGUAGESELECTOR_COMMIT=aba190c78226b5f9ea9bcc73fee09c23feef9620
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/VisualEditor/+/refs/heads/REL1_41
-VISUALEDITOR_COMMIT=753916e615161ee417bd5c6a37a19cbcffd49fb0
+VISUALEDITOR_COMMIT=b18dbbcdf296a9461d1cd7676739a4cbaf3dfc37
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikibaseCirrusSearch/+/refs/heads/REL1_41
 WIKIBASECIRRUSSEARCH_COMMIT=7a4fdf2342eaecee9217e430d702c03c66dab55c
 # https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/WikibaseManifest/+/refs/heads/REL1_41