Release v1.5.0 - S3 Storage backend, storage migration script & adding new translations

[elrido]

This release adds an S3 storage backend, a storage migration script, and new translations.

This minor release adds support for Simple Storage Service (S3), a storage migration script, adds four new languages to the translations and includes updated libraries.

The new S3 storage backend can be used with Ceph, Amazon Web Services or other S3 providers. A migration script was also provided, which can migrate existing instances from one storage backend to another, including in between the same storage backend types, but with different configurations (i.e. from one directory to another or one database to another).

The release includes several smaller improvements to the MariaDB and MySQL support. It reverts to CREATE INDEX without IF NOT EXISTS clauses (introduced in 1.4.0) which are not supported in MySQL. It also avoids requiring the SUPER privilege for the sql_mode added in 1.4.0. Indexes now also make use of the table prefix, to support multiple instances sharing a single database.

Jdenticons were added as a fourth option for comment icons, in addition to identicons and vizhash (and using none). The new icon type adds a different style and they don't require the PHP GD module to be generated. They are larger in size than identicons, a bit smaller than vizhash and slower to generate than either.

If using the YOURLS URL shortener with a signature and would like to keep it hidden, as of this release a server side integration via a proxy can be used, storing the signature only in configuration.

Benefits of switching to the new release

We recommend to upgrade 1.4.x instances using MariaDB backend and any instance that would like to make use of any of the new features or translations.

Update procedure

As usual, you can download the archive for a manual upgrade and can find more details in the installation instructions.

We also offer a Docker container that includes the recommended secure setup with the non-essential files and data outside of the web servers document root.

Changes since version 1.4.0

• ADDED: script for data storage backend migrations (script for storage backend migrations #1012)
• ADDED: Translations for Turkish, Slovak, Greek and Thai
• ADDED: S3 Storage backend (implemented S3 storage backend #994)
• ADDED: Jdenticons as an option for comment icons (identicon library is abandoned - find and switch to a replacement #793)
• CHANGED: Avoid SUPER privilege for setting the sql_mode for MariaDB/MySQL (MariaDB x PDO::MYSQL_ATTR_INIT_COMMAND (1.4.0) #919)
• CHANGED: Upgrading libraries to: DOMpurify 2.4.6, jQuery 3.6.1, Showdown 2.1.0 & zlib 1.2.13
• FIXED: Revert to CREATE INDEX without IF NOT EXISTS clauses, to support MySQL (MySQL doesn't support CREATE INDEX IF NOT EXISTS #943)
• FIXED: Apply table prefix to indexes as well, to support multiple instances sharing a single database (MySQL doesn't support CREATE INDEX IF NOT EXISTS #943)
• FIXED: YOURLS integration via new proxy, storing signature in configuration (Privatebin/YOURLS integration: API call to YOURLS not completing as intended #725)

Help wanted & greatly appreciated

Apart from the large tasks that require deeper insight and time, there are also smaller issues were help is wanted, topics open to debate and of course many languages that still remain to be translated. We are also still looking for additional long term maintainers among our frequent issue helpers.

If you are interested in helping with any of these points, we have prepared a development guide including design goals, code structure and tools that should get you started. For any questions, you can also chat with the maintainers in the discussion area.

Plans for future releases

The next regular release will focus on user interface improvements.

---

This discussion was created from the release Release v1.5.0 - S3 Storage backend, storage migration script & adding new translations.

[calvinbui]

are there any performance differences from using a database or s3 compared to flat files?

alternatively, is there a recommended option?

[elrido]

The following isn't based on actual benchmarks: Assuming a fast local disk access and a single instance, I'd expect to see the best performance in reading from local files over a locally running database using a UNIX socket, simply because no extra service is involved and the files can be cached in RAM (in Linux, at least).

When using a database over a network socket, S3 or GCS the network latency gets added as well.

I assume one would prefer to use S3, GCS or a database if one wanted to scale beyond a single node or the writeable filesystem is located on an NFS (for example a kubernetes persistant storage shared in read-write-many mode) or because it is well integrated with the hosting environment (cloud native, maintained by hoster, etc.) or existing tooling (backup scripts, etc.).

As one datapoint to compare with your own setup, the privatebin.net demo instance currently hosts around 35000 pastes consuming 1 GiB of disk space using the Filesystem backend on a single instance, 4 CPU, 512 MiB RAM VM at a 1m load average of 0.01 and network traffic average of 600 kbps out / 80 kbps in and 1m peaks of 17.5 Mbps out / 9 Mbps in.