Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying auto config path #6601

Merged
merged 1 commit into from
Jun 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

### Added
- Added FlowDB table `infrastructure.invalid_cell_info` for recording cell information that could not be included in `infrastructure.cell_info` (including cells with null or duplicate cell IDs). [#6626](https://github.com/Flowminder/FlowKit/issues/6626)
- The file name of FlowDB's automatically generated at init config file can now be specified by setting the `AUTO_CONFIG_FILE_NAME` environment variable. By default this is `postgresql.configurator.conf`.

### Changed
- FlowDB now triggers an ANALYZE on newly created cache tables to generate statistics rather than waiting for autovacuum
- FlowDB now produces JSON formatted logs by default. Set `FLOWDB_LOG_DEST=csvlog` for the old default behaviour.
- The logging destination of FlowDB can now be configured at init by setting the `FLOWDB_LOG_DEST` environment variable, valid options are `stderr`, `csvlog`, and `jsonlog`.
- The location inside the container of FlowDB's automatically generated config file has changed to `/flowdb_autoconf/$AUTO_CONFIG_FILE_NAME`.



### Fixed
Expand Down
2 changes: 2 additions & 0 deletions docs/source/administrator/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ You may also provide the following environment variables:
| FLOWDB_ENABLE_POSTGRES_DEBUG_MODE | When set to TRUE, enables use of the [pgadmin debugger](https://www.pgadmin.org/docs/pgadmin4/4.13/debugger.html) | FALSE |
| MAX_LOCKS_PER_TRANSACTION | Controls the maximum number of locks one transaction can take, you may wish to reduce this on low-memory servers. | 36500 |
| FLOWDB_LOG_DEST | Controls the logging destination, may be be one of `stderr`, `jsonlog`, `csvlog`. | `jsonlog` |
| AUTO_CONFIG_FILE_NAME | The path in the container of the automatically generated config file located at /flowdb_autoconf | `postgresql.configurator.conf` |


However in most cases, the defaults will be adequate.

Expand Down
5 changes: 5 additions & 0 deletions flowdb.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,9 @@ ENV LOCATION_TABLE=infrastructure.cells
# Default logging destination
ENV FLOWDB_LOG_DEST=jsonlog

# Default autoconfig file
ENV AUTO_CONFIG_FILE_NAME=postgresql.configurator.conf

#
# Copy file spinup build scripts to be execed.
#
Expand All @@ -155,5 +158,7 @@ COPY --chown=postgres ./flowdb/bin/build/* /docker-entrypoint-initdb.d/
ADD --chown=postgres ./flowdb/data/* /docker-entrypoint-initdb.d/data/csv/
# Need to make postgres owner
RUN chown -R postgres /docker-entrypoint-initdb.d
# Make postgres owner of the autoconf directory
RUN mkdir /flowdb_autoconf && chown -R postgres /flowdb_autoconf && chmod a+w /flowdb_autoconf

EXPOSE 5432
2 changes: 1 addition & 1 deletion flowdb/bin/build/0010_tweak_config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ set -e
# to have it off from the start for the benefit of the testdata containers
# for increased performance during startup.
python3 /docker-entrypoint-initdb.d/configurate.py
echo "include 'postgresql.configurator.conf'" >> /var/lib/postgresql/data/postgresql.conf
echo "include '/flowdb_autoconf/$AUTO_CONFIG_FILE_NAME'" >> /var/lib/postgresql/data/postgresql.conf
pg_ctl -D "$PGDATA" \
-o "-c listen_addresses=''" \
-w restart
4 changes: 1 addition & 3 deletions flowdb/bin/build/configurate.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,7 @@ def bool_env(var):
max_locks = int(os.getenv("MAX_LOCKS_PER_TRANSACTION", 365 * 5 * 4 * (1 + 4)))


config_path = os.getenv(
"AUTO_CONFIG_PATH", "/var/lib/postgresql/data/postgresql.configurator.conf"
)
config_path = f"/flowdb_autoconf/{os.getenv('AUTO_CONFIG_FILE_NAME', 'postgresql.configurator.conf')}"

preload_libraries = ["pg_stat_statements"]
if bool_env("FLOWDB_ENABLE_POSTGRES_DEBUG_MODE"):
Expand Down