Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DietPi-Software | Synapse: Fix install due to Rust integration #5778

Closed
wants to merge 1 commit into from

Conversation

Joulinar
Copy link
Collaborator

@Joulinar Joulinar commented Sep 28, 2022

STATUS - wip

DietPi-Software | Synapse - add rust as it seems to be required by install process now

@MichaIng
something we need to have a look into. We have a report on our forum about failing install due to missing rust install.
https://dietpi.com/forum/t/matrix-synapse-fails-to-update/14460

It's not 100% clear to me if this is now required or not because there is a statement that it is not required using PyPI.
https://matrix-org.github.io/synapse/v1.68/upgrade.html#rust-requirement-when-building-from-source

But there is no version for aarch64 on PyPI https://pypi.org/project/matrix-synapse/1.68.0/#files

On PiWheels build is failing as well https://www.piwheels.org/project/matrix-synapse/

Question is, should we add rust or hope it will be fixed upstream?? I just did a basic implementation without checking ram usage and other checks like architecture.

@Joulinar Joulinar marked this pull request as draft September 28, 2022 12:18
@Joulinar Joulinar self-assigned this Sep 28, 2022
@Joulinar Joulinar added this to the v8.10 milestone Sep 28, 2022
@MichaIng
Copy link
Owner

Dammit, previously there was only a single wheel for all architectures and system versions. With the implementation of Rust there are now a bunch of individual wheels required, and of course only x86 ones are available for now... So yes for ARM we need Rust, currently. We can report this to piwheels and contribute to requests for aarch64 wheels on PyPI as well, and see if/how fast this is done.

@MichaIng
Copy link
Owner

Let's see: matrix-org/synapse#13938, piwheels/packages#327

However, the aarch64 wheels at least won't be added before our next release, so good to keep this PR open. Just wrap Rust (un)install steps into ARM condition.

@Joulinar
Copy link
Collaborator Author

Ok I will have a look and add ARM condition + verify memory requirements.

@Joulinar
Copy link
Collaborator Author

ok on ARMv7 it requires gcc now and wheel bcrypt needs to be created. But it's missing/faild on piwheels as well due to rust https://www.piwheels.org/project/bcrypt/

@jahway603
Copy link

This issue I reported (matrix-org/synapse#14001) to upstream is related as I'm running Synapse on DietPi aarch64

@Joulinar
Copy link
Collaborator Author

Joulinar commented Oct 1, 2022

Your issue seems unrelated as you have a completely different behaviour. You seems to be missing setuptools

ModuleNotFoundError: No module named 'setuptools.command.build'

Did you tried DietPi install tools or the whole process manually by yourself?

@jahway603
Copy link

jahway603 commented Oct 3, 2022

I originally installed it with DietPi-software, but that was almost a year ago. I've been keeping synapse updated fine using pip until this recent 1.68.0 failed, the one that "requires rust"

@jahway603
Copy link

I tried following along with this DietPi forum post solution, but it still fails for me building wheels when after running pip3 install --no-cache-dir -U matrix-synapse psycopg2.

@Joulinar
Copy link
Collaborator Author

Joulinar commented Oct 3, 2022

An error message would be helpful

@jahway603
Copy link

@Joulinar same as in issue matrix-org/synapse#14001, but will paste it below too

$ sudo -u synapse pip3 install -U matrix-synapse
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: matrix-synapse in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (1.67.0)
Collecting matrix-synapse
  Using cached matrix-synapse-1.68.0.tar.gz (2.9 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.0.1)
Requirement already satisfied: pyasn1>=0.1.9 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.4.8)
Requirement already satisfied: sortedcontainers>=1.4.4 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (2.4.0)
Requirement already satisfied: phonenumbers>=8.2.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (8.12.33)
Requirement already satisfied: packaging>=16.1 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.0)
Requirement already satisfied: pyOpenSSL>=16.0.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.0.0)
Requirement already satisfied: Pillow>=5.4.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (8.3.2)
Requirement already satisfied: bcrypt>=3.1.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.2.0)
Requirement already satisfied: pyasn1-modules>=0.0.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.2.8)
Requirement already satisfied: pymacaroons>=0.13.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.13.0)
Requirement already satisfied: netaddr>=0.7.18 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.8.0)
Requirement already satisfied: msgpack>=0.5.2 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (1.0.2)
Requirement already satisfied: cryptography>=3.4.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (35.0.0)
Requirement already satisfied: attrs!=21.1.0,>=19.2.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.2.0)
Requirement already satisfied: frozendict!=2.1.2,>=1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (2.3.4)
Requirement already satisfied: jsonschema>=3.0.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.2.0)
Requirement already satisfied: ijson>=3.1.4 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.1.4)
Requirement already satisfied: prometheus-client>=0.4.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.11.0)
Requirement already satisfied: signedjson<2.0.0,>=1.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (1.1.1)
Requirement already satisfied: service-identity>=18.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.1.0)
Requirement already satisfied: bleach>=1.4.3 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (4.1.0)
Requirement already satisfied: pydantic>=1.7.4 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (1.10.1)
Collecting matrix-common<2.0.0,>=1.3.0
  Using cached matrix_common-1.3.0-py3-none-any.whl (11 kB)
Collecting setuptools_rust>=1.3
  Using cached setuptools_rust-1.5.2-py3-none-any.whl (23 kB)
Requirement already satisfied: unpaddedbase64>=2.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (2.1.0)
Requirement already satisfied: Twisted[tls]>=18.9.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.7.0)
Requirement already satisfied: treq>=15.1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (22.2.0)
Requirement already satisfied: PyYAML>=3.11 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (5.4.1)
Requirement already satisfied: canonicaljson<2.0.0,>=1.5.0 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (1.6.2)
Requirement already satisfied: typing-extensions>=3.10.0.1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (4.3.0)
Requirement already satisfied: cffi>=1.1 in /usr/local/lib/python3.9/dist-packages (from bcrypt>=3.1.7->matrix-synapse) (1.14.6)
Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.9/dist-packages (from bcrypt>=3.1.7->matrix-synapse) (1.16.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.9/dist-packages (from bleach>=1.4.3->matrix-synapse) (0.5.1)
Requirement already satisfied: simplejson>=3.14.0 in /usr/local/lib/python3.9/dist-packages (from canonicaljson<2.0.0,>=1.5.0->matrix-synapse) (3.17.5)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from Jinja2>=3.0->matrix-synapse) (2.0.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0.0->matrix-synapse) (58.0.4)
Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0.0->matrix-synapse) (0.18.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.9/dist-packages (from packaging>=16.1->matrix-synapse) (2.4.7)
Requirement already satisfied: PyNaCl<2.0,>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from pymacaroons>=0.13.0->matrix-synapse) (1.4.0)
Collecting semantic-version<3,>=2.8.2
  Using cached semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting setuptools
  Using cached setuptools-65.4.1-py3-none-any.whl (1.2 MB)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.9/dist-packages (from signedjson<2.0.0,>=1.1.0->matrix-synapse) (4.8.1)
Requirement already satisfied: incremental in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (21.3.0)
Requirement already satisfied: requests>=2.1.0 in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (2.26.0)
Requirement already satisfied: hyperlink>=21.0.0 in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (21.0.0)
Requirement already satisfied: Automat>=0.8.0 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (20.2.0)
Requirement already satisfied: constantly>=15.1 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (15.1.0)
Requirement already satisfied: zope.interface>=4.4.2 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (5.4.0)
Requirement already satisfied: idna>=2.4 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (3.2)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.1->bcrypt>=3.1.7->matrix-synapse) (2.20)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (2.0.6)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (1.26.7)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/dist-packages (from importlib-metadata->signedjson<2.0.0,>=1.1.0->matrix-synapse) (3.6.0)
Building wheels for collected packages: matrix-synapse
  Building wheel for matrix-synapse (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpgpd_o7pa
       cwd: /tmp/pip-install-zd_d2f6d/matrix-synapse_a58a0827ff0048ab8f3222aba7debffa
  Complete output (31 lines):
  A setup.py file already exists. Using it.
  Traceback (most recent call last):
    File "/tmp/pip-install-zd_d2f6d/matrix-synapse_a58a0827ff0048ab8f3222aba7debffa/setup.py", line 242, in <module>
      from build_rust import *
    File "/tmp/pip-install-zd_d2f6d/matrix-synapse_a58a0827ff0048ab8f3222aba7debffa/build_rust.py", line 6, in <module>
      from setuptools_rust import Binding, RustExtension
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/setuptools_rust/__init__.py", line 1, in <module>
      from .build import build_rust
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/setuptools_rust/build.py", line 23, in <module>
      from setuptools.command.build import build as CommandBuild  # type: ignore[import]
  ModuleNotFoundError: No module named 'setuptools.command.build'
  Traceback (most recent call last):
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/poetry/core/masonry/api.py", line 67, in build_wheel
      return WheelBuilder.make_in(poetry, Path(wheel_directory))
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 79, in make_in
      wb.build(target_dir=directory)
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 113, in build
      self._build(zip_file)
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 167, in _build
      self._run_build_command(setup)
    File "/tmp/pip-build-env-3nmbxrav/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 205, in _run_build_command
      subprocess.check_call(
    File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pip-install-zd_d2f6d/matrix-synapse_a58a0827ff0048ab8f3222aba7debffa/setup.py', 'build', '-b', '/tmp/pip-install-zd_d2f6d/matrix-synapse_a58a0827ff0048ab8f3222aba7debffa/build']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for matrix-synapse
Failed to build matrix-synapse
ERROR: Could not build wheels for matrix-synapse, which is required to install pyproject.toml-based projects

@Joulinar
Copy link
Collaborator Author

Joulinar commented Oct 3, 2022

As already stated above, your issue is different. At least from error message

ModuleNotFoundError: No module named 'setuptools.command.build'

@jahway603
Copy link

jahway603 commented Oct 3, 2022

As already stated above, your issue is different. At least from error message

ModuleNotFoundError: No module named 'setuptools.command.build'

sigh I already have that installed...

$ sudo apt-get install python3-setuptools
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-setuptools is already the newest version (52.0.0-4).

I've been running the DietPi Synapse package & setup for a year now. It is only this latest release that won't upgrade, the version that requires rust. I just tried updating pip to the latest and get the following, so same issue:

$ sudo -u synapse pip3 install -U matrix-synapse
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: matrix-synapse in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (1.67.0)
Collecting matrix-synapse
  Using cached matrix-synapse-1.68.0.tar.gz (2.9 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: treq>=15.1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (22.2.0)
Collecting matrix-common<2.0.0,>=1.3.0
  Using cached matrix_common-1.3.0-py3-none-any.whl (11 kB)
Requirement already satisfied: pymacaroons>=0.13.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.13.0)
Requirement already satisfied: attrs!=21.1.0,>=19.2.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.2.0)
Requirement already satisfied: service-identity>=18.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.1.0)
Requirement already satisfied: netaddr>=0.7.18 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.8.0)
Requirement already satisfied: phonenumbers>=8.2.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (8.12.33)
Requirement already satisfied: bleach>=1.4.3 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (4.1.0)
Requirement already satisfied: prometheus-client>=0.4.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.11.0)
Requirement already satisfied: pydantic>=1.7.4 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (1.10.1)
Requirement already satisfied: jsonschema>=3.0.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.2.0)
Requirement already satisfied: packaging>=16.1 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.0)
Requirement already satisfied: pyOpenSSL>=16.0.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.0.0)
Requirement already satisfied: PyYAML>=3.11 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (5.4.1)
Requirement already satisfied: Twisted[tls]>=18.9.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (21.7.0)
Requirement already satisfied: msgpack>=0.5.2 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (1.0.2)
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.0.1)
Requirement already satisfied: unpaddedbase64>=2.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (2.1.0)
Requirement already satisfied: frozendict!=2.1.2,>=1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (2.3.4)
Collecting setuptools_rust>=1.3
  Using cached setuptools_rust-1.5.2-py3-none-any.whl (23 kB)
Requirement already satisfied: typing-extensions>=3.10.0.1 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (4.3.0)
Requirement already satisfied: pyasn1>=0.1.9 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.4.8)
Requirement already satisfied: sortedcontainers>=1.4.4 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (2.4.0)
Requirement already satisfied: signedjson<2.0.0,>=1.1.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (1.1.1)
Requirement already satisfied: ijson>=3.1.4 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.1.4)
Requirement already satisfied: canonicaljson<2.0.0,>=1.5.0 in /mnt/dietpi_userdata/synapse/.local/lib/python3.9/site-packages (from matrix-synapse) (1.6.2)
Requirement already satisfied: cryptography>=3.4.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (35.0.0)
Requirement already satisfied: pyasn1-modules>=0.0.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (0.2.8)
Requirement already satisfied: bcrypt>=3.1.7 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (3.2.0)
Requirement already satisfied: Pillow>=5.4.0 in /usr/local/lib/python3.9/dist-packages (from matrix-synapse) (8.3.2)
Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.9/dist-packages (from bcrypt>=3.1.7->matrix-synapse) (1.16.0)
Requirement already satisfied: cffi>=1.1 in /usr/local/lib/python3.9/dist-packages (from bcrypt>=3.1.7->matrix-synapse) (1.14.6)
Requirement already satisfied: webencodings in /usr/local/lib/python3.9/dist-packages (from bleach>=1.4.3->matrix-synapse) (0.5.1)
Requirement already satisfied: simplejson>=3.14.0 in /usr/local/lib/python3.9/dist-packages (from canonicaljson<2.0.0,>=1.5.0->matrix-synapse) (3.17.5)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.9/dist-packages (from Jinja2>=3.0->matrix-synapse) (2.0.1)
Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0.0->matrix-synapse) (0.18.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0.0->matrix-synapse) (58.0.4)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.9/dist-packages (from packaging>=16.1->matrix-synapse) (2.4.7)
Requirement already satisfied: PyNaCl<2.0,>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from pymacaroons>=0.13.0->matrix-synapse) (1.4.0)
Collecting semantic-version<3,>=2.8.2
  Using cached semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting setuptools
  Using cached setuptools-65.4.1-py3-none-any.whl (1.2 MB)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.9/dist-packages (from signedjson<2.0.0,>=1.1.0->matrix-synapse) (4.8.1)
Requirement already satisfied: incremental in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (21.3.0)
Requirement already satisfied: requests>=2.1.0 in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (2.26.0)
Requirement already satisfied: hyperlink>=21.0.0 in /usr/local/lib/python3.9/dist-packages (from treq>=15.1->matrix-synapse) (21.0.0)
Requirement already satisfied: Automat>=0.8.0 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (20.2.0)
Requirement already satisfied: constantly>=15.1 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (15.1.0)
Requirement already satisfied: zope.interface>=4.4.2 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (5.4.0)
Requirement already satisfied: idna>=2.4 in /usr/local/lib/python3.9/dist-packages (from Twisted[tls]>=18.9.0->matrix-synapse) (3.2)
Requirement already satisfied: pycparser in /usr/local/lib/python3.9/dist-packages (from cffi>=1.1->bcrypt>=3.1.7->matrix-synapse) (2.20)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (2.0.6)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.1.0->treq>=15.1->matrix-synapse) (2021.5.30)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/dist-packages (from importlib-metadata->signedjson<2.0.0,>=1.1.0->matrix-synapse) (3.6.0)
Building wheels for collected packages: matrix-synapse
  Building wheel for matrix-synapse (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for matrix-synapse (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [31 lines of output]
      A setup.py file already exists. Using it.
      Traceback (most recent call last):
        File "/tmp/pip-install-uvge70xk/matrix-synapse_6bdb3fc52e5b40efa4ed02e8641205c8/setup.py", line 242, in <module>
          from build_rust import *
        File "/tmp/pip-install-uvge70xk/matrix-synapse_6bdb3fc52e5b40efa4ed02e8641205c8/build_rust.py", line 6, in <module>
          from setuptools_rust import Binding, RustExtension
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/setuptools_rust/__init__.py", line 1, in <module>
          from .build import build_rust
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/setuptools_rust/build.py", line 23, in <module>
          from setuptools.command.build import build as CommandBuild  # type: ignore[import]
      ModuleNotFoundError: No module named 'setuptools.command.build'
      Traceback (most recent call last):
        File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.9/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/poetry/core/masonry/api.py", line 67, in build_wheel
          return WheelBuilder.make_in(poetry, Path(wheel_directory))
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 79, in make_in
          wb.build(target_dir=directory)
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 113, in build
          self._build(zip_file)
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 167, in _build
          self._run_build_command(setup)
        File "/tmp/pip-build-env-y15p56_z/overlay/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 205, in _run_build_command
          subprocess.check_call(
        File "/usr/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/usr/bin/python3', '/tmp/pip-install-uvge70xk/matrix-synapse_6bdb3fc52e5b40efa4ed02e8641205c8/setup.py', 'build', '-b', '/tmp/pip-install-uvge70xk/matrix-synapse_6bdb3fc52e5b40efa4ed02e8641205c8/build']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for matrix-synapse
Failed to build matrix-synapse
ERROR: Could not build wheels for matrix-synapse, which is required to install pyproject.toml-based projects

Then I tried "updating" setuptools by running sudo -u synapse pip3 install -U pip setuptools & still fails with wheels stuff...

@Joulinar
Copy link
Collaborator Author

Joulinar commented Oct 3, 2022

How much memory your SBC has? At least 2 GB ram? If yes, as a workaround you could try to switch your system to this branch and try to install matrix-synapse.

G_CONFIG_INJECT 'DEV_GITBRANCH=' 'DEV_GITBRANCH=matrix-synapse' /boot/dietpi.txt
dietpi-update 1
dietpi-software reinstall 125

@jahway603
Copy link

jahway603 commented Oct 12, 2022

@Joulinar I'm not sure if I've ever used a SBC with less than 4GB RAM, but I don't want to test on my production matrix server, so instead I have followed your suggested instructions (with dietpi-software install 125 instead) on a brand new DietPi install.

When I first ran this, it failed and complained about my xterm-kitty terminal, which I filed as separate Issue #5800

When I tried running the install again in a different terminal emulator, alacritty, it was successful. This new server is federating according to https://federationtester.matrix.org/

My question now is, how to do this on a production system without borking it?

@Joulinar
Copy link
Collaborator Author

My question now is, how to do this on a production system without borking it?

you could wait on our next release. Atm we are waiting on matrix-org/synapse and piwheels to created missing wheels upstream. If they don't succeed, we will implement an own solution.

@MichaIng MichaIng changed the title DietPi-Software | Synapse DietPi-Software | Synapse: Fix install due to Rust integration Oct 15, 2022
DietPi-Software | Synapse - add rust as it seems to be required by install process now
@Joulinar
Copy link
Collaborator Author

@MichaIng
I guess we agreed to go with a global Rust install option for user root. Means we could close this PR as it would be obsolet?

@MichaIng
Copy link
Owner

You're right, easier to do this as new PR than reworking this one.

@jahway603
Copy link

jahway603 commented Oct 18, 2022

@Joulinar Question: On a production server, the command dietpi-software reinstall 125 won't mess up my postgresql, right?

I was successfully able to run that command on my DietPi test Matrix Synapse and it upgraded it from 1.68.0 to 1.69.0, so wanted to ask my above question as the test server has no users and no data on it, but my production server is being used by people right now.

@Joulinar
Copy link
Collaborator Author

won't mess up my postgresql, right?

Theoretically it should not touch any database data. But as we are on IT, I can't put my hand into the fire 😉
Definitely do a backup before going to update. But you could wait until our next release. There we plan to offer a Rust install option, to ensure propper install/reinstall.

@MichaIng
Copy link
Owner

I mark this PR as closed. aarch64 builds will be available soon: matrix-org/synapse#14212
I'll now try to get the same pushed for piwheels.

@MichaIng MichaIng closed this Oct 18, 2022
@Joulinar
Copy link
Collaborator Author

Joulinar commented Oct 20, 2022

@jahway603
We will not implement a fix for Synapse now as @MichaIng created a fix upstream to bypass the rust topic on aarch64. We would recommend waiting for Synapse 1.70 as this should contain the pre-compiled wheel. 😃

@MichaIng MichaIng deleted the matrix-synapse branch November 1, 2022 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants