Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Install rust during Stage 0 of docker build #15239

Merged
merged 2 commits into from Mar 13, 2023
Merged

Install rust during Stage 0 of docker build #15239

merged 2 commits into from Mar 13, 2023

Conversation

DMRobertson
Copy link
Contributor

Thanks to @atomdmac for spotting the fix.
Fixes #15179.

David Robertson added 2 commits March 9, 2023 18:46
@DMRobertson
Copy link
Contributor Author

I don't have a PPC or arm machine handy to test this with; I suggest we merge this if CI is happy on the grounds of "not a regression". (I can't remember what parts of our CI build which docker images---but we must do so for complement runs, at least?)

@DMRobertson DMRobertson marked this pull request as ready for review March 9, 2023 18:49
@DMRobertson DMRobertson requested a review from a team as a code owner March 9, 2023 18:49
@DMRobertson
Copy link
Contributor Author

I suppose we could also make this behaviour opt-in via a build arg, if we're worried about the cost of another rust installation?

@atomdmac
Copy link

atomdmac commented Mar 11, 2023 •
edited

I tried building this on PopOS using docker buildx build --platform linux/arm/v7 and got an error. Here's the relevant portion of the log:

Log 
#0 176.8   × Building wheel for cryptography (pyproject.toml) did not run successfully.
#0 176.8   │ exit code: 1
#0 176.8   ╰─> [189 lines of output]
#0 176.8       running bdist_wheel
#0 176.8       running build
#0 176.8       running build_py
#0 176.8       creating build
#0 176.8       creating build/lib.linux-armv7l-cpython-311
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/utils.py -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/__about__.py -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/fernet.py -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/exceptions.py -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/base.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/general_name.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/oid.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/certificate_transparency.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/name.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/extensions.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/ocsp.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       copying src/cryptography/x509/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/x509
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat
#0 176.8       copying src/cryptography/hazmat/_oid.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat
#0 176.8       copying src/cryptography/hazmat/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings
#0 176.8       copying src/cryptography/hazmat/bindings/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/hmac.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/constant_time.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/padding.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/cmac.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/_cipheralgorithm.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/keywrap.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/poly1305.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/_serialization.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/_asymmetric.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       copying src/cryptography/hazmat/primitives/hashes.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends
#0 176.8       copying src/cryptography/hazmat/backends/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/openssl
#0 176.8       copying src/cryptography/hazmat/bindings/openssl/_conditional.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/openssl
#0 176.8       copying src/cryptography/hazmat/bindings/openssl/binding.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/openssl
#0 176.8       copying src/cryptography/hazmat/bindings/openssl/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/openssl
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/x448.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/ed448.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/utils.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/ed25519.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/rsa.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/padding.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/dsa.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/dh.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/x25519.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/ec.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       copying src/cryptography/hazmat/primitives/asymmetric/types.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/asymmetric
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/hkdf.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/kbkdf.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/x963kdf.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/concatkdf.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/pbkdf2.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       copying src/cryptography/hazmat/primitives/kdf/scrypt.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/kdf
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       copying src/cryptography/hazmat/primitives/serialization/base.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       copying src/cryptography/hazmat/primitives/serialization/pkcs7.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       copying src/cryptography/hazmat/primitives/serialization/ssh.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       copying src/cryptography/hazmat/primitives/serialization/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       copying src/cryptography/hazmat/primitives/serialization/pkcs12.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/serialization
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/twofactor
#0 176.8       copying src/cryptography/hazmat/primitives/twofactor/totp.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/twofactor
#0 176.8       copying src/cryptography/hazmat/primitives/twofactor/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/twofactor
#0 176.8       copying src/cryptography/hazmat/primitives/twofactor/hotp.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/twofactor
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       copying src/cryptography/hazmat/primitives/ciphers/base.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       copying src/cryptography/hazmat/primitives/ciphers/algorithms.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       copying src/cryptography/hazmat/primitives/ciphers/aead.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       copying src/cryptography/hazmat/primitives/ciphers/modes.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       copying src/cryptography/hazmat/primitives/ciphers/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/primitives/ciphers
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/x448.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/ed448.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/utils.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/backend.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/ed25519.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/rsa.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/hmac.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/ciphers.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/cmac.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/dsa.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/dh.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/aead.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/x25519.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/ec.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/decode_asn1.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/poly1305.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/__init__.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       copying src/cryptography/hazmat/backends/openssl/hashes.py -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/backends/openssl
#0 176.8       running egg_info
#0 176.8       writing src/cryptography.egg-info/PKG-INFO
#0 176.8       writing dependency_links to src/cryptography.egg-info/dependency_links.txt
#0 176.8       writing requirements to src/cryptography.egg-info/requires.txt
#0 176.8       writing top-level names to src/cryptography.egg-info/top_level.txt
#0 176.8       reading manifest file 'src/cryptography.egg-info/SOURCES.txt'
#0 176.8       reading manifest template 'MANIFEST.in'
#0 176.8       no previously-included directories found matching 'docs/_build'
#0 176.8       warning: no previously-included files found matching 'vectors'
#0 176.8       warning: no previously-included files matching '*' found under directory 'vectors'
#0 176.8       warning: no previously-included files matching '*' found under directory '.github'
#0 176.8       warning: no previously-included files found matching 'release.py'
#0 176.8       warning: no previously-included files found matching '.readthedocs.yml'
#0 176.8       warning: no previously-included files found matching 'ci-constraints-requirements.txt'
#0 176.8       warning: no previously-included files found matching 'tox.ini'
#0 176.8       warning: no previously-included files found matching 'mypy.ini'
#0 176.8       adding license file 'LICENSE'
#0 176.8       adding license file 'LICENSE.APACHE'
#0 176.8       adding license file 'LICENSE.BSD'
#0 176.8       adding license file 'LICENSE.PSF'
#0 176.8       writing manifest file 'src/cryptography.egg-info/SOURCES.txt'
#0 176.8       /tmp/pip-build-env-ctgt_2fm/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'cryptography.hazmat.bindings._rust' as data is deprecated, please list it in `packages`.
#0 176.8           !!
#0 176.8
#0 176.8
#0 176.8           ############################
#0 176.8           # Package would be ignored #
#0 176.8           ############################
#0 176.8           Python recognizes 'cryptography.hazmat.bindings._rust' as an importable package,
#0 176.8           but it is not listed in the `packages` configuration of setuptools.
#0 176.8
#0 176.8           'cryptography.hazmat.bindings._rust' has been automatically added to the distribution only
#0 176.8           because it may contain data files, but this behavior is likely to change
#0 176.8           in future versions of setuptools (and therefore is considered deprecated).
#0 176.8
#0 176.8           Please make sure that 'cryptography.hazmat.bindings._rust' is included as a package by using
#0 176.8           the `packages` configuration field or the proper discovery methods
#0 176.8           (for example by using `find_namespace_packages(...)`/`find_namespace:`
#0 176.8           instead of `find_packages(...)`/`find:`).
#0 176.8
#0 176.8           You can read more about "package discovery" and "data files" on setuptools
#0 176.8           documentation page.
#0 176.8
#0 176.8
#0 176.8       !!
#0 176.8
#0 176.8         check.warn(importable)
#0 176.8       copying src/cryptography/py.typed -> build/lib.linux-armv7l-cpython-311/cryptography
#0 176.8       copying src/cryptography/hazmat/bindings/_openssl.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings
#0 176.8       creating build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       copying src/cryptography/hazmat/bindings/_rust/__init__.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       copying src/cryptography/hazmat/bindings/_rust/asn1.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       copying src/cryptography/hazmat/bindings/_rust/ocsp.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       copying src/cryptography/hazmat/bindings/_rust/pkcs7.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       copying src/cryptography/hazmat/bindings/_rust/x509.pyi -> build/lib.linux-armv7l-cpython-311/cryptography/hazmat/bindings/_rust
#0 176.8       running build_ext
#0 176.8       running build_rust
#0 176.8       cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --release -v --features pyo3/abi3-py36 pyo3/extension-module -- --crate-type cdylib
#0 176.8           Updating crates.io index
#0 176.8       warning: spurious network error (2 tries remaining): could not read directory '/cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#0 176.8       warning: spurious network error (1 tries remaining): could not read directory '/cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#0 176.8       error: Unable to update registry `crates-io`
#0 176.8
#0 176.8       Caused by:
#0 176.8         failed to fetch `https://github.com/rust-lang/crates.io-index`
#0 176.8
#0 176.8       Caused by:
#0 176.8         could not read directory '/cargo/registry/index/github.com-1285ae84e5963aae/.git/refs': Value too large for defined data type; class=Os (2)
#0 176.8
#0 176.8           =============================DEBUG ASSISTANCE=============================
#0 176.8           If you are seeing a compilation error please try the following steps to
#0 176.8           successfully install cryptography:
#0 176.8           1) Upgrade to the latest pip and try again. This will fix errors for most
#0 176.8              users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
#0 176.8           2) Read https://cryptography.io/en/latest/installation/ for specific
#0 176.8              instructions for your platform.
#0 176.8           3) Check our frequently asked questions for more information:
#0 176.8              https://cryptography.io/en/latest/faq/
#0 176.8           4) Ensure you have a recent Rust toolchain installed:
#0 176.8              https://cryptography.io/en/latest/installation/#rust
#0 176.8
#0 176.8           Python: 3.11.2
#0 176.8           platform: Linux-6.1.11-76060111-generic-armv7l-with-glibc2.31
#0 176.8           pip: n/a
#0 176.8           setuptools: 67.6.0
#0 176.8           setuptools_rust: 1.5.2
#0 176.8           rustc: n/a
#0 176.8           =============================DEBUG ASSISTANCE=============================
#0 176.8
#0 176.8       error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path src/rust/Cargo.toml --release -v --features 'pyo3/abi3-py36 pyo3/extension-module' -- --crate-type cdylib` failed with code 101
#0 176.8       [end of output]
#0 176.8
#0 176.8   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 176.8   ERROR: Failed building wheel for cryptography
#0 176.8 Successfully built dulwich msgpack rapidfuzz
#0 176.8 Failed to build cryptography
#0 176.8 ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects
#0 177.4
#0 177.4 [notice] A new release of pip available: 22.3.1 -> 23.0.1
#0 177.4 [notice] To update, run: pip install --upgrade pip
------
ERROR: failed to solve: executor failed running [/bin/sh -c pip install --user "poetry==1.3.2"]: exit code: 1

@MatMaul
Copy link
Contributor

MatMaul commented Mar 13, 2023

This is another trouble specific to building armv7 on a 64bit architecture apparently:
docker/buildx#395

@DMRobertson DMRobertson merged commit d4eba44 into develop Mar 13, 2023
@DMRobertson DMRobertson deleted the dmr/docker-without-cryptography-wheels branch March 13, 2023 12:12
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Apr 9, 2023
@Midar
Update chat/matrix-synapse to 1.18.0
f524488 
Synapse 1.80.0 (2023-03-28)
===========================

No significant changes since 1.80.0rc2.


Synapse 1.80.0rc2 (2023-03-22)
==============================

Bugfixes
--------

- Fix a bug in which the [`POST /_matrix/client/v3/rooms/{roomId}/report/{eventId}`](https://spec.matrix.org/v1.6/client-server-api/#post_matrixclientv3roomsroomidreporteventid) endpoint would return the wrong error if the user did not have permission to view the event. This aligns Synapse's implementation with [MSC2249](matrix-org/matrix-spec-proposals#2249). ([\#15298](matrix-org/synapse#15298), [\#15300](matrix-org/synapse#15300))
- Fix a bug introduced in Synapse 1.75.0rc1 where the [SQLite port_db script](https://matrix-org.github.io/synapse/latest/postgres.html#porting-from-sqlite)
  would fail to open the SQLite database. ([\#15301](matrix-org/synapse#15301))


Synapse 1.80.0rc1 (2023-03-21)
==============================

Features
--------

- Stabilise support for [MSC3966](matrix-org/matrix-spec-proposals#3966): `event_property_contains` push condition. ([\#15187](matrix-org/synapse#15187))
- Implement [MSC2659](matrix-org/matrix-spec-proposals#2659): application service ping endpoint. Contributed by Tulir @ Beeper. ([\#15249](matrix-org/synapse#15249))
- Allow loading `/register/available` endpoint on workers. ([\#15268](matrix-org/synapse#15268))
- Improve performance of creating and authenticating events. ([\#15195](matrix-org/synapse#15195))
- Add topic and name events to group of events that are batch persisted when creating a room. ([\#15229](matrix-org/synapse#15229))


Bugfixes
--------

- Fix a long-standing bug in which the user directory would assume any remote membership state events represent a profile change. ([\#14755](matrix-org/synapse#14755), [\#14756](matrix-org/synapse#14756))
- Implement [MSC3873](matrix-org/matrix-spec-proposals#3873) to fix a long-standing bug where properties with dots were handled ambiguously in push rules. ([\#15190](matrix-org/synapse#15190))
- Faster joins: Fix a bug introduced in Synapse 1.66 where spurious "Failed to find memberships ..." errors would be logged. ([\#15232](matrix-org/synapse#15232))
- Fix a long-standing error when sending message into deleted room. ([\#15235](matrix-org/synapse#15235))


Updates to the Docker image
---------------------------

- Ensure the Dockerfile builds on platforms that don't have a `cryptography` wheel. ([\#15239](matrix-org/synapse#15239))
- Mirror images to the GitHub Container Registry (`ghcr.io/matrix-org/synapse`). ([\#15281](matrix-org/synapse#15281), [\#15282](matrix-org/synapse#15282))


Improved Documentation
----------------------

- Add a missing endpoint to the workers documentation. ([\#15223](matrix-org/synapse#15223))


Internal Changes
----------------

- Add additional functionality to declaring worker types when starting Complement in worker mode. ([\#14921](matrix-org/synapse#14921))
- Add `Synapse-Trace-Id` to `access-control-expose-headers` header. ([\#14974](matrix-org/synapse#14974))
- Make the `HttpTransactionCache` use the `Requester` in addition of the just the `Request` to build the transaction key. ([\#15200](matrix-org/synapse#15200))
- Improve log lines when purging rooms. ([\#15222](matrix-org/synapse#15222))
- Improve type hints. ([\#15230](matrix-org/synapse#15230), [\#15231](matrix-org/synapse#15231), [\#15238](matrix-org/synapse#15238))
- Move various module API callback registration methods to a dedicated class. ([\#15237](matrix-org/synapse#15237))
- Configure GitHub Actions for merge queues. ([\#15244](matrix-org/synapse#15244))
- Add schema comments about the `destinations` and `destination_rooms` tables. ([\#15247](matrix-org/synapse#15247))
- Skip processing of auto-join room behaviour if there are no auto-join rooms configured. ([\#15262](matrix-org/synapse#15262))
- Remove unused store method `_set_destination_retry_timings_emulated`. ([\#15266](matrix-org/synapse#15266))
- Reorganize URL preview code. ([\#15269](matrix-org/synapse#15269))
- Clean-up direct TCP replication code. ([\#15272](matrix-org/synapse#15272), [\#15274](matrix-org/synapse#15274))
- Make `configure_workers_and_start` script used in Complement tests compatible with older versions of Python. ([\#15275](matrix-org/synapse#15275))
- Add a `/versions` flag for [MSC3952](matrix-org/matrix-spec-proposals#3952). ([\#15293](matrix-org/synapse#15293))
- Bump hiredis from 2.2.1 to 2.2.2. ([\#15252](matrix-org/synapse#15252))
- Bump serde from 1.0.152 to 1.0.155. ([\#15253](matrix-org/synapse#15253))
- Bump pysaml2 from 7.2.1 to 7.3.1. ([\#15254](matrix-org/synapse#15254))
- Bump msgpack from 1.0.4 to 1.0.5. ([\#15255](matrix-org/synapse#15255))
- Bump gitpython from 3.1.30 to 3.1.31. ([\#15256](matrix-org/synapse#15256))
- Bump cryptography from 39.0.1 to 39.0.2. ([\#15257](matrix-org/synapse#15257))
- Bump pydantic from 1.10.4 to 1.10.6. ([\#15286](matrix-org/synapse#15286))
- Bump serde from 1.0.155 to 1.0.157. ([\#15287](matrix-org/synapse#15287))
- Bump anyhow from 1.0.69 to 1.0.70. ([\#15288](matrix-org/synapse#15288))
- Bump txredisapi from 1.4.7 to 1.4.9. ([\#15289](matrix-org/synapse#15289))
- Bump pygithub from 1.57 to 1.58.1. ([\#15290](matrix-org/synapse#15290))
- Bump types-requests from 2.28.11.12 to 2.28.11.15. ([\#15291](matrix-org/synapse#15291))



Synapse 1.79.0 (2023-03-14)
===========================

No significant changes since 1.79.0rc2.


Synapse 1.79.0rc2 (2023-03-13)
==============================

Bugfixes
--------

- Fix a bug introduced in Synapse 1.79.0rc1 where attempting to register a `on_remove_user_third_party_identifier` module API callback would be a no-op. ([\#15227](matrix-org/synapse#15227))
- Fix a rare bug introduced in Synapse 1.73 where events could remain unsent to other homeservers after a faster-join to a room. ([\#15248](matrix-org/synapse#15248))


Internal Changes
----------------

- Refactor `filter_events_for_server`. ([\#15240](matrix-org/synapse#15240))


Synapse 1.79.0rc1 (2023-03-07)
==============================

Features
--------

- Add two new Third Party Rules module API callbacks: [`on_add_user_third_party_identifier`](https://matrix-org.github.io/synapse/v1.79/modules/third_party_rules_callbacks.html#on_add_user_third_party_identifier) and [`on_remove_user_third_party_identifier`](https://matrix-org.github.io/synapse/v1.79/modules/third_party_rules_callbacks.html#on_remove_user_third_party_identifier). ([\#15044](matrix-org/synapse#15044))
- Experimental support for [MSC3967](matrix-org/matrix-spec-proposals#3967) to not require UIA for setting up cross-signing on first use. ([\#15077](matrix-org/synapse#15077))
- Add media information to the command line [user data export tool](https://matrix-org.github.io/synapse/v1.79/usage/administration/admin_faq.html#how-can-i-export-user-data). ([\#15107](matrix-org/synapse#15107))
- Add an [admin API](https://matrix-org.github.io/synapse/latest/usage/administration/admin_api/index.html) to delete a [specific event report](https://spec.matrix.org/v1.6/client-server-api/#reporting-content). ([\#15116](matrix-org/synapse#15116))
- Add support for knocking to workers. ([\#15133](matrix-org/synapse#15133))
- Allow use of the `/filter` Client-Server APIs on workers. ([\#15134](matrix-org/synapse#15134))
- Update support for [MSC2677](matrix-org/matrix-spec-proposals#2677): remove support for server-side aggregation of reactions. ([\#15172](matrix-org/synapse#15172))
- Stabilise support for [MSC3758](matrix-org/matrix-spec-proposals#3758): `event_property_is` push condition. ([\#15185](matrix-org/synapse#15185))


Bugfixes
--------

- Fix a bug introduced in Synapse 1.75 that caused experimental support for deleting account data to raise an internal server error while using an account data writer worker. ([\#14869](matrix-org/synapse#14869))
- Fix a long-standing bug where Synapse handled an unspecced field on push rules. ([\#15088](matrix-org/synapse#15088))
- Fix a long-standing bug where a URL preview would break if the discovered oEmbed failed to download. ([\#15092](matrix-org/synapse#15092))
- Fix a long-standing bug where an initial sync would not respond to changes to the list of ignored users if there was an initial sync cached. ([\#15163](matrix-org/synapse#15163))
- Add the `transaction_id` in the events included in many endpoints' responses. ([\#15174](matrix-org/synapse#15174))
- Fix a bug introduced in Synapse 1.78.0 where requests to claim dehydrated devices would fail with a `405` error. ([\#15180](matrix-org/synapse#15180))
- Stop applying edits when bundling aggregations, per [MSC3925](matrix-org/matrix-spec-proposals#3925). ([\#15193](matrix-org/synapse#15193))
- Fix a long-standing bug where the user directory search was not case-insensitive for accented characters. ([\#15143](matrix-org/synapse#15143))


Updates to the Docker image
---------------------------

- Improve startup logging in the with-workers Docker image. ([\#15186](matrix-org/synapse#15186))


Improved Documentation
----------------------

- Document how to use caches in a module. ([\#14026](matrix-org/synapse#14026))
- Clarify which worker processes the ThirdPartyRules' [`on_new_event`](https://matrix-org.github.io/synapse/v1.78/modules/third_party_rules_callbacks.html#on_new_event) module API callback runs on. ([\#15071](matrix-org/synapse#15071))
- Document using [Shibboleth](https://www.shibboleth.net/) as an OpenID Provider. ([\#15112](matrix-org/synapse#15112))
- Correct reference to `federation_verify_certificates` in configuration documentation. ([\#15139](matrix-org/synapse#15139))
- Correct small documentation errors in some `MatrixFederationHttpClient` methods. ([\#15148](matrix-org/synapse#15148))
- Correct the description of the behavior of `registration_shared_secret_path` on startup. ([\#15168](matrix-org/synapse#15168))


Deprecations and Removals
-------------------------

- Deprecate the `on_threepid_bind` module callback, to be replaced by [`on_add_user_third_party_identifier`](https://matrix-org.github.io/synapse/v1.79/modules/third_party_rules_callbacks.html#on_add_user_third_party_identifier). See [upgrade notes](https://github.com/matrix-org/synapse/blob/release-v1.79/docs/upgrade.md#upgrading-to-v1790). ([\#15044](matrix-org/synapse#15044))
- Remove the unspecced `room_alias` field from the [`/createRoom`](https://spec.matrix.org/v1.6/client-server-api/#post_matrixclientv3createroom) response. ([\#15093](matrix-org/synapse#15093))
- Remove the unspecced `PUT` on the `/knock/{roomIdOrAlias}` endpoint. ([\#15189](matrix-org/synapse#15189))
- Remove the undocumented and unspecced `type` parameter to the `/thumbnail` endpoint. ([\#15137](matrix-org/synapse#15137))
- Remove unspecced and buggy `PUT` method on the unstable `/rooms/<room_id>/batch_send` endpoint. ([\#15199](matrix-org/synapse#15199))


Internal Changes
----------------

- Run the integration test suites with the asyncio reactor enabled in CI. ([\#14101](matrix-org/synapse#14101))
- Batch up storing state groups when creating a new room. ([\#14918](matrix-org/synapse#14918))
- Update [MSC3952](matrix-org/matrix-spec-proposals#3952) support based on changes to the MSC. ([\#15051](matrix-org/synapse#15051))
- Refactor writing json data in `FileExfiltrationWriter`. ([\#15095](matrix-org/synapse#15095))
- Tighten the login ratelimit defaults. ([\#15135](matrix-org/synapse#15135))
- Fix a typo in an experimental config setting. ([\#15138](matrix-org/synapse#15138))
- Refactor the media modules. ([\#15146](matrix-org/synapse#15146), [\#15175](matrix-org/synapse#15175))
- Improve type hints. ([\#15164](matrix-org/synapse#15164))
- Move `get_event_report` and `get_event_reports_paginate` from `RoomStore` to `RoomWorkerStore`. ([\#15165](matrix-org/synapse#15165))
- Remove dangling reference to being a reference implementation in docstring. ([\#15167](matrix-org/synapse#15167))
- Add an option to force a rebuild of the "editable" complement image. ([\#15184](matrix-org/synapse#15184))
- Use nightly rustfmt in CI. ([\#15188](matrix-org/synapse#15188))
- Add a `get_next_txn` method to `StreamIdGenerator` to match `MultiWriterIdGenerator`. ([\#15191](matrix-org/synapse#15191))
- Combine `AbstractStreamIdTracker` and `AbstractStreamIdGenerator`. ([\#15192](matrix-org/synapse#15192))
- Automatically fix errors with `ruff`. ([\#15194](matrix-org/synapse#15194))
- Refactor database transaction for query users' devices to reduce database pool contention. ([\#15215](matrix-org/synapse#15215))
- Correct `test_icu_word_boundary_punctuation` so that it passes with the ICU versions available in Alpine and macOS. ([\#15177](matrix-org/synapse#15177))

<details><summary>Locked dependency updates</summary>

  - Bump actions/checkout from 2 to 3. ([\#15155](matrix-org/synapse#15155))
  - Bump black from 22.12.0 to 23.1.0. ([\#15103](matrix-org/synapse#15103))
  - Bump dawidd6/action-download-artifact from 2.25.0 to 2.26.0. ([\#15152](matrix-org/synapse#15152))
  - Bump docker/login-action from 1 to 2. ([\#15154](matrix-org/synapse#15154))
  - Bump matrix-org/backend-meta from 1 to 2. ([\#15156](matrix-org/synapse#15156))
  - Bump ruff from 0.0.237 to 0.0.252. ([\#15159](matrix-org/synapse#15159))
  - Bump serde_json from 1.0.93 to 1.0.94. ([\#15214](matrix-org/synapse#15214))
  - Bump types-commonmark from 0.9.2.1 to 0.9.2.2. ([\#15209](matrix-org/synapse#15209))
  - Bump types-opentracing from 2.4.10.1 to 2.4.10.3. ([\#15158](matrix-org/synapse#15158))
  - Bump types-pillow from 9.4.0.13 to 9.4.0.17. ([\#15211](matrix-org/synapse#15211))
  - Bump types-psycopg2 from 2.9.21.4 to 2.9.21.8. ([\#15210](matrix-org/synapse#15210))
  - Bump types-pyopenssl from 22.1.0.2 to 23.0.0.4. ([\#15213](matrix-org/synapse#15213))
  - Bump types-setuptools from 67.3.0.1 to 67.4.0.3. ([\#15160](matrix-org/synapse#15160))
  - Bump types-setuptools from 67.4.0.3 to 67.5.0.0. ([\#15212](matrix-org/synapse#15212))
  - Bump typing-extensions from 4.4.0 to 4.5.0. ([\#15157](matrix-org/synapse#15157))
</details>
Fizzadar added a commit to beeper/synapse-legacy-fork that referenced this pull request Apr 11, 2023
@Fizzadar
Merge tag 'v1.80.0' into merge-1.78
995114d 
Synapse 1.80.0 (2023-03-28)
===========================

No significant changes since 1.80.0rc2.

Synapse 1.80.0rc2 (2023-03-22)
==============================

Bugfixes
--------

- Fix a bug in which the [`POST /_matrix/client/v3/rooms/{roomId}/report/{eventId}`](https://spec.matrix.org/v1.6/client-server-api/#post_matrixclientv3roomsroomidreporteventid) endpoint would return the wrong error if the user did not have permission to view the event. This aligns Synapse's implementation with [MSC2249](matrix-org/matrix-spec-proposals#2249). ([\matrix-org#15298](matrix-org#15298), [\matrix-org#15300](matrix-org#15300))
- Fix a bug introduced in Synapse 1.75.0rc1 where the [SQLite port_db script](https://matrix-org.github.io/synapse/latest/postgres.html#porting-from-sqlite)
  would fail to open the SQLite database. ([\matrix-org#15301](matrix-org#15301))

Synapse 1.80.0rc1 (2023-03-21)
==============================

Features
--------

- Stabilise support for [MSC3966](matrix-org/matrix-spec-proposals#3966): `event_property_contains` push condition. ([\matrix-org#15187](matrix-org#15187))
- Implement [MSC2659](matrix-org/matrix-spec-proposals#2659): application service ping endpoint. Contributed by Tulir @ Beeper. ([\matrix-org#15249](matrix-org#15249))
- Allow loading `/register/available` endpoint on workers. ([\matrix-org#15268](matrix-org#15268))
- Improve performance of creating and authenticating events. ([\matrix-org#15195](matrix-org#15195))
- Add topic and name events to group of events that are batch persisted when creating a room. ([\matrix-org#15229](matrix-org#15229))

Bugfixes
--------

- Fix a long-standing bug in which the user directory would assume any remote membership state events represent a profile change. ([\matrix-org#14755](matrix-org#14755), [\matrix-org#14756](matrix-org#14756))
- Implement [MSC3873](matrix-org/matrix-spec-proposals#3873) to fix a long-standing bug where properties with dots were handled ambiguously in push rules. ([\matrix-org#15190](matrix-org#15190))
- Faster joins: Fix a bug introduced in Synapse 1.66 where spurious "Failed to find memberships ..." errors would be logged. ([\matrix-org#15232](matrix-org#15232))
- Fix a long-standing error when sending message into deleted room. ([\matrix-org#15235](matrix-org#15235))

Updates to the Docker image
---------------------------

- Ensure the Dockerfile builds on platforms that don't have a `cryptography` wheel. ([\matrix-org#15239](matrix-org#15239))
- Mirror images to the GitHub Container Registry (`ghcr.io/matrix-org/synapse`). ([\matrix-org#15281](matrix-org#15281), [\matrix-org#15282](matrix-org#15282))

Improved Documentation
----------------------

- Add a missing endpoint to the workers documentation. ([\matrix-org#15223](matrix-org#15223))

Internal Changes
----------------

- Add additional functionality to declaring worker types when starting Complement in worker mode. ([\matrix-org#14921](matrix-org#14921))
- Add `Synapse-Trace-Id` to `access-control-expose-headers` header. ([\matrix-org#14974](matrix-org#14974))
- Make the `HttpTransactionCache` use the `Requester` in addition of the just the `Request` to build the transaction key. ([\matrix-org#15200](matrix-org#15200))
- Improve log lines when purging rooms. ([\matrix-org#15222](matrix-org#15222))
- Improve type hints. ([\matrix-org#15230](matrix-org#15230), [\matrix-org#15231](matrix-org#15231), [\matrix-org#15238](matrix-org#15238))
- Move various module API callback registration methods to a dedicated class. ([\matrix-org#15237](matrix-org#15237))
- Configure GitHub Actions for merge queues. ([\matrix-org#15244](matrix-org#15244))
- Add schema comments about the `destinations` and `destination_rooms` tables. ([\matrix-org#15247](matrix-org#15247))
- Skip processing of auto-join room behaviour if there are no auto-join rooms configured. ([\matrix-org#15262](matrix-org#15262))
- Remove unused store method `_set_destination_retry_timings_emulated`. ([\matrix-org#15266](matrix-org#15266))
- Reorganize URL preview code. ([\matrix-org#15269](matrix-org#15269))
- Clean-up direct TCP replication code. ([\matrix-org#15272](matrix-org#15272), [\matrix-org#15274](matrix-org#15274))
- Make `configure_workers_and_start` script used in Complement tests compatible with older versions of Python. ([\matrix-org#15275](matrix-org#15275))
- Add a `/versions` flag for [MSC3952](matrix-org/matrix-spec-proposals#3952). ([\matrix-org#15293](matrix-org#15293))
- Bump hiredis from 2.2.1 to 2.2.2. ([\matrix-org#15252](matrix-org#15252))
- Bump serde from 1.0.152 to 1.0.155. ([\matrix-org#15253](matrix-org#15253))
- Bump pysaml2 from 7.2.1 to 7.3.1. ([\matrix-org#15254](matrix-org#15254))
- Bump msgpack from 1.0.4 to 1.0.5. ([\matrix-org#15255](matrix-org#15255))
- Bump gitpython from 3.1.30 to 3.1.31. ([\matrix-org#15256](matrix-org#15256))
- Bump cryptography from 39.0.1 to 39.0.2. ([\matrix-org#15257](matrix-org#15257))
- Bump pydantic from 1.10.4 to 1.10.6. ([\matrix-org#15286](matrix-org#15286))
- Bump serde from 1.0.155 to 1.0.157. ([\matrix-org#15287](matrix-org#15287))
- Bump anyhow from 1.0.69 to 1.0.70. ([\matrix-org#15288](matrix-org#15288))
- Bump txredisapi from 1.4.7 to 1.4.9. ([\matrix-org#15289](matrix-org#15289))
- Bump pygithub from 1.57 to 1.58.1. ([\matrix-org#15290](matrix-org#15290))
- Bump types-requests from 2.28.11.12 to 2.28.11.15. ([\matrix-org#15291](matrix-org#15291))
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@MatMaul MatMaul MatMaul approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unable to build synapse via podman on ppc64le
3 participants
@DMRobertson @atomdmac @MatMaul