Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DietPi-Software | Matrix Synapse Server #4526

Merged
merged 48 commits into from
Jul 11, 2021
Merged

DietPi-Software | Matrix Synapse Server #4526

merged 48 commits into from
Jul 11, 2021

Conversation

ravenclaw900
Copy link
Collaborator

@ravenclaw900 ravenclaw900 commented Jun 27, 2021

Status: Ready

  • Implement using PostgreSQL
  • Switch to building from source

Reference: #4525

Commit list/description:

  • Multiple | Add Synapse Server

+ Multiple | Add Synapse Server
@ravenclaw900 ravenclaw900 added this to the v7.4 milestone Jun 27, 2021
@ravenclaw900 ravenclaw900 self-assigned this Jun 27, 2021
@ravenclaw900 ravenclaw900 linked an issue Jun 27, 2021 that may be closed by this pull request
@MichaIng MichaIng marked this pull request as ready for review June 27, 2021 18:23
+ DietPi-Software | Implement PostgreSQL usage
+ DietPi-Software | Add and remove G_EXEC
+ DietPi-Software | Allow you to set server name and analytics
+ DietPi-Software | Add G_EXEC_OUTPUT=1 to some commands
+ DietPi-Software | Restart service before trying to register new user
+ DietPi-Software | Change bind addresses to bind to the public network
+ DietPi-Software | Use buster-backports instead of official repo
+ DietPi-Software | Change way synapse is installed
+ DietPi-Software | Add missing path to G_CONFIG_INJECT
+ DietPi-Software | Add pre-made config file
dietpi/dietpi-software Outdated Show resolved Hide resolved
+ DietPi-Software | Build synapse from source
+ DietPi-Software | Fix paths
+ DietPi-Software | Fix uninstall
+ DietPi-Software | Fix bind_addresses
+ DietPi-Software | Replace G_CONFIG_INJECT with sed
+ DietPi-Software | Fix indentation
+ DietPi-Software | cd to synapse directory before creating config files
+ DietPi-Services | Rename synapse service
+ DietPi-Software | Remove unneeded config_path
+ DietPi-Software | Fix dietpi_userdata path
+ DietPi-Software | Re-add apparently needed --config-file
+ DietPi-Software | PostgreSQL: postgresql-common prevents all other packages from being auto-removed, hence select all of them explicitly
+ DietPi-Software | Synapse: Fix build on ARMv8 where make is additionally required to compile PyNaCl
+ DietPi-Software | Synapse: Merge install and config steps, as we want to do for all software titles
+ DietPi-Software | Enhance logging software description
+ README | Add Synapse sources to 3rd party list
+ DietPi-Survey_report | Add support for PostgreSQL with v7.4
+ CHANGELOG | Synapse: A Matrix homeserver implementation has been added with software ID 125.
+ CHANGELOG | PostgreSQL: A persistent advanced object-relational database server has been added with software ID 194.
dietpi/dietpi-software Outdated Show resolved Hide resolved
ravenclaw900 and others added 3 commits July 10, 2021 09:00
+ DietPi-Software | Rename Synapse Server to Synapse
+ DietPi-Software | PostgreSQL: Disable TCP/IP listener by default and instead assure that UNIX domain socket is enabled at expected location
+ DietPi-Software | Synapse: Start server in systemd unit via python3 call directly and omit synctl, which saves ~70 MiB memory and allows to pass custom command line parameters easily. Use this to load additional override configs from homeserver.yaml.d directory.
+ DietPi-Software | Synapse: Connect to PostgreSQL via UNIX domain socket, which provides ~30% faster queries.
+ DietPi-Software | Synapse: Apply database access details via override config, which is much safer than editing YAML with repeating and multi-line config blocks.
+ DietPi-Software | Synapse: Log to journal (journalctl -u synapse) by default and reduce log level to WARNING
+ DietPi-Software | Synapse: Switch to Type=notify, so that "systemctl start synapse" finishes only after the service is ready to listen. Else register_new_matrix_user can fail when the server was not yet ready. Taken from upstream: https://github.com/matrix-org/synapse/blob/develop/debian/matrix-synapse.service
+ DietPi-Software | Synapse: Implement quick reload with systemd unit, taken from upstream: https://github.com/matrix-org/synapse/blob/develop/debian/matrix-synapse.service
+ DietPi-Software | PostgreSQL: Remove possible left configuration files on uninstall
@MichaIng
Copy link
Owner

Btw, what was the reason for switching away from the official APT repository?

+ DietPi-Software | Synapse: Add forgotten service type change to Type=notify
+ DietPi-Services | Add PostgreSQL control
+ DietPi-Software | PostgreSQL: Fix override config creation
+ DietPi-Software | Synapse: Before checking and altering the database, assure that PostgreSQL is started, essential on reinstalls
@ravenclaw900
Copy link
Collaborator Author

It only has amd64 and i386 packages.

@MichaIng
Copy link
Owner

Ah okay, makes totally sense then. And Buster backports is not available on 32-bit RPi images.

+ DietPi-Software | Synapse: Clarify in comment a little why image codecs are required on ARMv6/7 (but not on ARMv8 and x64_64) 
+ DietPi-Software | Synapse: Error-handle most steps of the uninstall
+ DietPi-Software | Python 3: Remove obsolete pip cache on uninstall
@MichaIng
Copy link
Owner

Mystery solved why on ARMv6/7 additional (image codec) libraries are required:

+ DietPi-Software | Synapse: Since PyPI does not ship a Python 3.5 cffi wheel anymore, it needs to be compiled on Debian Stretch
+ DietPi-Software | PostgreSQL: Add a commit about why we don't disable the TCP/IP listener on Stretch
@MichaIng
Copy link
Owner

MichaIng commented Jul 11, 2021

Ready from my end. Tested on all distro versions, ARMv6/7/8 and x86_64 and:

  • Build dependencies minimised.
  • Changed PostgreSQL default (on Buster and up) to not listen on TCP/IP anymore but UNIX domain socket only and adjusted Synapse accordingly. This reduces overhead/memory usage a little and provides ~30% faster communication. Since Synapse is the first and only dependant, this is a unique chance to implement this as default right from the start. Something we need to add to our online docs.
  • Tweaked systemd unit to use Type=notify and not use synctl. The first is to assure that the server has fully started (listens on 8008) before systemctl start command returns, else the user registration on install can fail, on slow devices or when input is done fast. The second reduces memory usage by ~70 MiB used by synctl itself. I think this is a console tool and intended only as such to interactively manage Synapse processes, which is why it defaults to daemonise the server etc. It's much more direct to call the Python module itself. Or did you face any issue with that?

@MichaIng
Copy link
Owner

Merging now to avoid conflicts with youtube-dl.

@MichaIng MichaIng merged commit 6527ece into dev Jul 11, 2021
@MichaIng MichaIng deleted the synapse branch July 11, 2021 12:38
@tandy-1000
Copy link

* Changed PostgreSQL default (on Buster and up) to not listen on TCP/IP anymore but UNIX domain socket only and adjusted Synapse accordingly. This reduces overhead/memory usage a little and provides ~30% faster communication. Since Synapse is the first and only dependant, this is a unique chance to implement this as default right from the start. Something we need to add to our online docs.

This caused some issues for me on non-DietPi services which used PostgreSQL until I found this PR.

@Joulinar
Copy link
Collaborator

You are on 7.4 Beta?

@MichaIng
Copy link
Owner

Yes, this has not yet been released and it won't affect manually installed instances of PostgreSQL (e.g. apt install postgresql). While it is generally recommended to use UNIX sockets for local service connections, it will be easy and documented how to re-enable it and how to allow also remote TCP/IP connections to PostgreSQL, as for security reasons on Debian it listens only on localhost/loopback IP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DietPi-Software | Matrix Synapse Server
4 participants