Database Lab Engine v3.2.0
Release v3.2.0 delivers a good number of new features and improvements. DLE UI is extended with two new tabs, "Logs" and "Configuration", to improve the user experience when configuring or reconfiguring DLE for managed Postgres services such as AWS RDS and Google Cloud SQL:
This allows users to see PGDATA initialization errors when setting up DLE, then adjust the configuration and retry initialization right in the UI – without any need to connect to the DLE machine using SSH.
The demo DLE instance can be found here: https://demo.aws.postgres.ai:446/instance (token: demo_token
).
Postgres 15 is already supported!
PostgreSQL 15 is already supported by DLE and included in the list of default images, with many extensions shipped with it: https://postgres.ai/docs/database-lab/supported-databases.
AWS Marketplace – the easiest way to get DLE for any Postgres database
DLE in AWS Marketplace has now General Availability (GA) status, it has already been updated to version 3.2.0: https://aws.amazon.com/marketplace/pp/prodview-wlmm2satykuec, offering the Standard license, with support, embedded monitoring, simplified installation, and access to a wider variety of supported Postgres images (including extensions not available by default). An instance with DLE delivers "dev/test databases as a service" and levels up all dev&test processes. For example, on an r5.xlarge instance (4 vCPUs, 32 GiB RAM) with 200 GiB disk space, it is possible to run up to 30 database clones simultaneously (and even more, if shared_buffers
is adjusted to lower values) spending as low as $360 per month – less than for a single regular RDS clone of the same size.
New contributor
🙇 The development team greets a new contributor: @Adrinlol. This release wouldn't be possible without other contributors: @agneum, @vitabaks, @DmitryFomin1, @ane4ka, @NikolayS, @denis256, @yuridevx.
What's new
New features
- New tab in UI, "Configuration" and corresponding API to allow editing of certain configuration parameters (only in logical mode; secret values are masked) and viewing the current config (in any mode): !561, !575, !594, !601, !612, !613, !614 (@yuridevx, @Adrinlol), !608, !609, !612 (@Adrinlol), !580, !615
- New tab in UI, "Logs" to allow seeing DLE log messages to support troubleshooting (secret values are masked): !579 (@Adrinlol), !585, !593
- Manage logs of PostgreSQL containers to simplify diagnostics (previously, if an error happens inside Postgres container and the container is destroyed, it was impossible to find the details of the error): !558, !570, !598, https://gitlab.com/postgres-ai/docs/-/merge_requests/481 (@denis256)
- Support
queryPreprocessing
inlogicalRestore
!576 - Allow using inline SQL in
queryPreprocessing
(available in sectionslogicalRestore
,logicalSnapshot
,physicalSnapshot
) !576 - Simplify Postgres image customization – for initialization in logical mode, perform comprehensive testing of the source database and compare a container created using the Postgres image provided in configuration with the source DB: !573, !578 (:bow: contribution by @Adrinlol), !596
- Show details of continuous synchronization of PGDATA (API+CLI; currently only for physical mode): !562, !595 (@denis256)
- Show the progress of the full refresh process !574, !581, !584, !587, !588, !591, !610 (@Adrinlol)
- Configuration option
excludeTables
– allow excluding certain tables for data retrieval in logical mode: !536, https://gitlab.com/postgres-ai/database-lab/-/issues/348 (was also backpatched to 3.1.2) - Support PostgreSQL 15: !599, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/42, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/44, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/45, https://gitlab.com/postgres-ai/docs/-/merge_requests/504
Improvements and fixes
- Redesign the Retrieval service: !549
- Mounting of socket directory for non-guest virtualization roles: !557 (@denis256)
- Add a link to the guide "PostgreSQL configuration in clones": !559
- Add session manager to CF template: !560
- Add
maintenance_work_mem
for Logical mode: !567 - Report the type of DLE subscription in UI: !563
- Detect if DLE is running in a Docker container (this resolves issues of running DLE on KVM): !571, !603
- Apply the configuration and then start Postgres on the restore stage, to reduce the number of restarts needed: !518, !547 (@yuridevx), !558, !572
- Fix the creation of a restricted user: !586
- Build CE UI Docker image after merging into the master branch: !590
- Adjust container readiness health checks: !597, !602
- Remove deprecated eslint analyzers: !605
- Increase the number of parallel jobs in logical mode from 2/2 to 4/4 – usually, many more CPU cores are available: !611
Diff showing all changes
Diff between versions v3.1.2 and v3.2.0: v3.1.2...v3.2.0.
Migration from DLE v3.1.x and configuration notes
There are no special actions that are required to migrate from DLE 3.1.x to 3.2.0: you need to stop the DLE container, change the image, and start a new one using the new image. All snapshots should remain available, and if, at stopping time, some clones were present, they should be automatically recreated by DLE.
Note: if you have an older version, check the previous release notes as well: https://gitlab.com/postgres-ai/database-lab/-/releases.
Configuration directory
The config directory, by default, is ~/.dblab/engine/configs
and the corresponding mounted volume must be writable:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs
...
Logs directory
Make sure that ~/.dblab/engine/logs
is mounted as a volume to the DLE container:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs \
--volume ~/.dblab/engine/meta:/home/dblab/meta \
--volume ~/.dblab/engine/logs:/home/dblab/logs \
...
Alternatively, you can mount the entire directory (including configs
and meta
– but make sure that these subdirectories exist, DLE doesn't create them yet):
...
--volume ~/.dblab/engine:/home/dblab
...
Query preprocessing
The queryPreprocessing
section supports inline SQL now. Examples can be found here: !576#examples
Configuration UI
Because the configuration can be viewed and edited in the UI, make sure:
- secret values are masked (e.g.,
verificationToken
, PlatformaccessToken
, you can find all masking rules here: !608#masking-rules) - DLE configuration matches the reference configuration files in structure and naming; it is not recommended to have custom names and sections
Where to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Provide feedback, contribute
Any feedback will be greatly appreciated:
- Twitter: @Database_Lab
- Database Lab Community Slack: https://slack.postgres.ai/
If you want to help the project grow, consider various ways of contributing:
- Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
- Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
- Consider translating the README.md to one or several languages you know
- Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab