Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Antidote configuration #397

Merged
merged 32 commits into from
Nov 14, 2019
Merged

Antidote configuration #397

merged 32 commits into from
Nov 14, 2019

Conversation

albsch
Copy link
Member

@albsch albsch commented Oct 31, 2019

Unified and simplified Antidote configuration.

  • Changed shell command to use the release shell instead
  • Added part on how to use the relase in the Readme
  • Rebar now understands symlinks for _checkouts, fixed readme
  • Removed deprecated env entrypoint script
  • Removed launch-nodes script, uses legacy RELX_REPLACE_OS_VARS functionality
  • Removed all schema files except the riak_core schema for compatibility purposes. Especially the erlang-vm.schema file caused problems when trying to use the NODE_NAME env variable
  • Configuration uses the new .src functionality to enable native OS variable replacement in the root configuration file
  • Split up smaller configuration files, currently only for the network configuration
  • Enabled packing files (data antidote, data riak, logging files) in a root directory if specified via ROOT_DIR_PREFIX
  • Same for logging and data dirs (LOGGER_DIR_PREFIX, DATA_DIR_PREFIX)
  • Removed vars.config, it was only used for the Mustache templating system of Rebar, which was only used for the schema files of cuttlefish, which we do not use
  • Fixed ranch to version 1.7.1 for now, otherwise it would upgrade to a release candidate with breaking API changes
  • Bumped antidote to version 0.2.1 (both rebar.config and antidote.app.src)
  • Moved documentation from antidote.app.src to config/sys.config where all configuration for the release resides
  • Added a validation function to ensure the antidote data directory exists
  • Commented out stats calls which are not implemented yet in the antidote_stats module (metadata_updated, metadata_update_stable)
  • Removed partially obsolete max ETS tables argument. The amount is not limited anymore, and 8000 is a reasonable default size of the internally used table for named tables.
  • Removed the ERL_MAX_PORTS env parameter. This is no longer used and is replaced by vm argument +Q. Furthermore, the default is much higher (65536) than this value (4096).
  • Added kernel network config for documentation purposes
  • Removed eunit_opts
    • Verbose output is worse than default output, as it doesn't show failures at the end
    • Eunit_logs is not useful
  • Added eunit.coverdata to coverage stats, coverage has increased by ~7%
  • Moved monitoring setup to antidote_stats repository
  • Fixed logging output path in systests and removed copied erlang code for ct_slave_ext
  • Redirected lager output to logger via custom lager backend

Consequences:

  • Update readme for monitoring setup in antidote_stats repository
  • Update documentation for monitoring setup in Antidote documentation
  • Docker files have to be updated
  • Update documentation for Antidote on how to use a release, go into more detail than the readme
  • Some configuration parameters cannot be directly configured by OS variables and have to be configured dynamically via the -Application Key Value parameter tuple, for which Value can be used for OS variable configuration
  • VM arguments are copy pasted from a rebar default from many years ago. Check if vm args improve performance, if not disable them
  • Check if there is a way to specify coverage output path for ct_run

@albsch
Copy link
Member Author

albsch commented Nov 4, 2019

The new dockerfile can be seen here.

@albsch albsch marked this pull request as ready for review November 4, 2019 14:10
@bieniusa
Copy link
Contributor

bieniusa commented Nov 4, 2019

Why are we bumping from v0.1.1. to v0.4.0?

@albsch
Copy link
Member Author

albsch commented Nov 4, 2019

Why are we bumping from v0.1.1. to v0.4.0?

There are multiple versions everywhere and I didn't know which version to take. The docker-antidote version is later than v0.2.0 and the release configuration (rebar) is 0.0.2. So I guess the version should be something like v0.2.1 everywhere?

Actually, I don't know how the rebar release version and the app.src version relates to the release tagged version/dockerfile version. Are they related at all?

Albert Schimpf added 11 commits November 4, 2019 17:43
…s replaced by vm argument +Q. Furthermore, the default is much higher (65536) than this value (4096).
… limited anymore, and 8000 is a reasonable default size of the internally used table for named tables.
* Changed shell command to use the release shell instead
* Added part on how to use the relase in the Readme
* Rebar now understands symlinks for _checkouts, fixed readme
* Removed deprecated env entrypoint script
* Removed launch-nodes script, uses legacy RELX_REPLACE_OS_VARS functionality
* Removed all schema files except the riak_core schema for compatibility purposes. Especially the erlang-vm.schema file caused problems when trying to use the NODE_NAME env variable
* Configuration use the new .src functionality to enable native OS variable replacement in the root configuration file
* Split up smaller configuration files, currently only for the network configuration
* Enabled packing files (data antidote, data riak, logging files) in a root directory if specified via ROOT_DIR_PREFIX
* Same for logging and data dirs (LOGGER_DIR_PREFIX, DATA_DIR_PREFIX)
* Removed vars.config, it was only used for the Mustache templating system of Rebar, which was only used for the schema files of cuttlefish, which we do not use
* Fixed ranch to version 1.7.1 for now, otherwise it would upgrade to a release candidate with breaking API changes
* Bumped antidote to version 0.4.0 (both rebar.config and antidote.app.src)
* Moved documentation from antidote.app.src to config/sys.config where all configuration for the release resides
* Added a validation function to ensure the antidote data directory exists
* Commented out stats calls which are not implemented yet in the antidote_stats module (metadata_updated, metadata_update_stable)

Consequences:

* Docker files have to be updated
…_opts. Verbose output is worse than default output, as it doesn't show failures at the end. Eunit_logs is not useful. Added eunit.coverdata to coverage stats. Silenced added test case
…cify export_cover_name with ct_run directly
@albsch
Copy link
Member Author

albsch commented Nov 11, 2019

New statistics:
image
image
image

* Redirected lager output to logger via lager backend.
* Added expensive statistics enabled by `extended_stats` application
setting
* Added warning monitor for logger warning messages
* Bumped antidote_stats version
* Added `antidote_stats` back again into the supervision tree, removed
by mistake
@albsch albsch merged commit ec98c26 into master Nov 14, 2019
@albsch albsch deleted the antidote-configuration branch November 15, 2019 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants