From a33bdb7f8c15ba7179f7ad27e31fd9ddff3d7a69 Mon Sep 17 00:00:00 2001 From: Alexandre Aubin Date: Mon, 30 Oct 2023 18:38:28 +0100 Subject: [PATCH] Iterate on config panel documentation --- .../20.config_panels/config_panels.md | 314 +++++++++++++----- 1 file changed, 223 insertions(+), 91 deletions(-) diff --git a/pages/06.contribute/10.packaging_apps/60.advanced/20.config_panels/config_panels.md b/pages/06.contribute/10.packaging_apps/60.advanced/20.config_panels/config_panels.md index 610895a0c..1698deb69 100644 --- a/pages/06.contribute/10.packaging_apps/60.advanced/20.config_panels/config_panels.md +++ b/pages/06.contribute/10.packaging_apps/60.advanced/20.config_panels/config_panels.md @@ -1,5 +1,5 @@ --- -title: Configuration panel for apps +title: Config Panels template: docs taxonomy: category: docs @@ -7,15 +7,23 @@ routes: default: '/packaging_config_panels' --- -Configuration panels allow to let admins manage parameters or runs actions for which the upstream's app doesn't provide any appropriate UI itself. It's a good way to reduce manual change on config files and avoid conflicts on it. +From a practical point of view, one of the main purpose of config panel is to "expose" settings from the app's configuration files to YunoHost's admins which are therefore able to manipulate them from a nice web ui. This is especially relevant for apps which do not provide any sort of admin UI and expect admins to manually edit the configuration files. -Those panels can also be used to quickly create interfaces that extend the capabilities of YunoHost (e.g. VPN Client, Hotspost, Borg, etc.). +Technically speaking, config panels are used both for apps packaging and also core features (domain configuration, global settings) ! Please: Keep in mind the YunoHost spirit, and try to build your panels in such a way as to expose only really useful, "high-level" parameters, and if there are many of them, to relegate those corresponding to rarer use cases to "Advanced" sub-sections. Keep it simple, focus on common needs, don't expect the admins to have 3 PhDs in computer science. -## `config_panel.toml`'s principle and general format +## Community examples -To create configuration panels for apps, you should at least create a `config_panel.toml` at the root of the package. For more complex cases, this TOML file can be paired with a `config` script inside the scripts directory of your package, which will handle specific controller logic. +- [Check the basic example at the end of this doc](#basic-example) +- [Check the example_ynh app toml](https://github.com/YunoHost/example_ynh/blob/master/config_panel.toml.example) and the [basic `scripts/config` example](https://github.com/YunoHost/example_ynh/blob/master/scripts/config) +- [Check config panels of other apps](https://grep.app/search?q=version&filter[repo.pattern][0]=YunoHost-Apps&filter[lang][0]=TOML) +- [Check `scripts/config` of other apps](https://grep.app/search?q=ynh_app_config_apply&filter[repo.pattern][0]=YunoHost-Apps&filter[lang][0]=Shell) + + +## Overview + +From an app packager perspective, config panels are defined in `config_panel.toml` at the root of the app repository. It is coupled to the `scripts/config` script, which may be used to define custom getters/setters/validations/actions. However, most use cases should be covered automagically by the core, thus it may not be necessary to define such a `config` script at all! The `config_panel.toml` describes one or several panels, containing sections, each containing questions generally binded to a params in the app's actual configuration files. @@ -27,7 +35,7 @@ max_rate: 10 max_age: 365 ``` -We could for example create a simple configuration panel for it like this one, by following the syntax `[PANEL.SECTION.QUESTION]`: +A simple configuration panel can be created with this syntax: ```toml version = "1.0" [main] @@ -44,7 +52,7 @@ version = "1.0" choices = ["white", "dark"] bind = ":__INSTALL_DIR__/config.yml" - [main.limits] + [main.limits] [main.limits.max_rate] ask.en = "Maximum display rate" type = "number" @@ -56,58 +64,38 @@ version = "1.0" bind = ":__INSTALL_DIR__/config.yml" ``` -Here we have created one `main` panel, containing the `main` and `limits` sections, containing questions according to params name of our `config.yml` file. Thanks to the `bind` properties, all those questions are bind to their values in the `config.yml` file. - -### Questions short keys have to be unique - -For performance reasons, questions short keys have to be unique in all the `config_panel.toml` file, not just inside its panel or its section. Hence it's not possible to have: -```toml -[manual.vpn.server_ip] -[advanced.dns.server_ip] -``` -In which two questions have "real variable name" `is server_ip` and therefore conflict with each other. - -! Some short keys are forbidden cause it can interfer with config scripts (`old`, `file_hash`, `types`, `binds`, `formats`, `changed`) and you probably should avoid to use common settings name to avoid to bind your question to this settings (e.g. `id`, `install_time`, `mysql_pwd`, `path`, `domain`, `port`, `db_name`, `current_revision`, `admin`) - -### Supported questions types and properties - -See [the full list of questions types and properties](/dev/forms) +Here, a `main` panel is created, containing the `main` and `limits` sections, containing questions corresponding to app options in `config.yml` file. Thanks to the `bind` property, all those questions are read/written to their values in the actual `config.yml` file. +## Supported questions types and properties -### Reading and writing values +You can learn more about the full list of option types and their properties in [their dedicated page](/dev/forms). -You can read and write values with 2 mechanisms: the `bind` property in the `config_panel.toml` and for complex use cases the getter/setter in a `config` script. -### `bind` property +## The "`bind`" statement -The `bind` property allows to define where read and write the value bind to the question. +Without any `bind` statement attached to a config panel property, values are only read/written from/to the app's settings file (`/etc/yunohost/$app/settings.yml`). This is usually not very useful in practice. -#### Default behaviour +Using `bind = ":/some/config/file"`, one declares that the actual truth used by the app lives in `/some/config/file`. The config panel will read/write the value from/to this file. YunoHost will automagically adapt to classic formats such as YAML, TOML, JSON, INI, PHP, `.env`-like and `.py`. (At least, assuming we're dealing with simple key<->value mappings) -If you did not define a specific getter/setter (see below), and no `bind` argument was defined, YunoHost will read/write the value from/to the app's `/etc/yunohost/$app/settings.yml` file. - -#### Read / write into a var of an actual configuration file - -If you want to read/write the value from/to the app's actual configural file (be it `.env`-like, JSON, YAML, INI, PHP, `.py`, ...): +A simple real-life example looks like: ```toml [main.main.theme] -# (other properties ommited) +type = "string" bind = ":__INSTALL_DIR__/config.yml" ``` -In which case, YunoHost will look for something like a key/value, with the key being `theme`. +In which case, YunoHost will look for something like a key/value, with the key being `theme` inside the app's `config.yml`. + +If the question id in the config panel (here, `theme`) differs from the key in the actual conf file (let's say it's not `theme` but `css_theme`), then the syntax becomes: -If the question id in the config panel (here, `theme`) differs from the key in the actual conf file (let's say it's not `theme` but `css_theme`), then you can write: ```toml [main.main.theme] -# (other properties ommited) +type = "string" bind = "css_theme:__FINALPATH__/config.yml" ``` -!!!! Note: This mechanism is quasi language agnostic and will use regexes to find something that looks like a key=value or common variants. However, it does assume that the key and value are stored on the same line. It doesn't support multiline text or file in a variable with this method. If you need to save multiline content in a configuration variable, you should create a custom getter/setter (see below). - -Nested syntax is also supported, which may be useful for example to remove ambiguities about stuff looking like: +You may also encounter situations such as: ```json { "foo": { @@ -119,33 +107,43 @@ Nested syntax is also supported, which may be useful for example to remove ambig } ``` -which we can `bind` to using: +In which case if we want to interface with foo's `max` value, let's write: ```toml bind = "foo>max:__INSTALL_DIR__/conf.json" ``` -#### Read / write an entire file +### "Binding" to an entire file Useful when using a question `file` or `text` for which you want to save the raw content directly as a file on the system. + +For example to be able to manipulate an image: + ```toml -[main.main.logo] -# (other properties ommited) -bind = "__INSTALL_DIR__/img/logo.png" +[panel.section.logo] +type = "file" +bind = "__INSTALL_DIR__/assets/logo.png" ``` -### Custom getter / setter +Or an entire text file: -Sometimes the `bind` mechanism is not enough: - * the config file format is not supported (e.g. xml, csv) - * the data is not contained in a config file (e.g. database, directory, web resources...) - * the data should be written but not read (e.g. password) - * the data should be read but not written (e.g. fetching status information) - * we want to change other things than the value (e.g. the choices list of a select) - * the question answer contains several values to dispatch in several places - * and so on +```toml +[panel.section.config_content] +type = "text" +bind = "__INSTALL_DIR__/config.ini" +default = "This is the default content" +``` + + +### Custom getters/setters/validators (a.k.a `bind=null`) -You can create specific getter/setters functions inside the `scripts/config` of your app to customize how the information is read/written. +More complex use-case may appear, such as: +- you want to expose some "dynamic" information in the config panel, such as computed health status, computed disk usage, dynamic list of options, ... +- password handling, where the data may be written but can't be read +- the config file format is not supposed (e.g. xml, csv, ...) +- ... + +You can create specific getter/setters functions inside the `config` script of the app to customize how the information is read/written. The basic structure of the script is: ```bash #!/bin/bash @@ -153,19 +151,20 @@ source /usr/share/yunohost/helpers ynh_abort_if_errors -# Put your getter, setter and validator here +# Put your getter, setter, validator or action here # Keep this last line ynh_app_config_run $1 ``` -#### Getter +#### Custom getters -A question's getter is the function used to read the current value/state. Custom getters are defined using bash functions called `getter__QUESTION_SHORT_KEY()` which returns data through stdout. +A question's getter is the function used to read the current value/state. Custom getters are defined using bash functions called `getter__QUESTION_SHORT_KEY()` which returns data through stdout. Stdout can generated using one of those formats: - 1) either a raw format, in which case the return is binded directly to the value of the question - 2) or a yaml format, in this case you dynamically provide properties for your question (for example the `style` of an `alert`, the list of available `choices` of a `select`, etc.) + 1) either just the raw value, + 2) or a yaml, containing the value and other metadata and properties (for example the `style` of an `alert`, the list of available `choices` of a `select`, etc.) + [details summary="Basic example with raw stdout: get the timezone on the system" class="helper-card-subtitle text-muted"] @@ -190,10 +189,10 @@ get__timezone() { `config_panel.toml` ```toml - [main.plugins.plugins] - ask = "Plugin to activate" - type = "tags" - choices = [] +[main.plugins.plugins] +ask = "Plugin to activate" +type = "tags" +choices = [] ``` `scripts/config` @@ -211,26 +210,25 @@ get__plugins() { `config_panel.toml` ```toml - [main.cube.status] - ask = "Custom getter alert" - type = "alert" - style = "info" - bind = "null" # no behaviour on +[main.cube.status] +ask = "Custom getter alert" +type = "alert" +style = "info" +bind = "null" # no behaviour on ``` `scripts/config` - ```bash get__status() { if [ -f "/sys/class/net/tun0/operstate" ] && [ "$(cat /sys/class/net/tun0/operstate)" == "up" ] then - cat << EOF + cat << EOF style: success ask: en: Your VPN is running :) EOF else - cat << EOF + cat << EOF style: danger ask: en: Your VPN is down @@ -240,12 +238,14 @@ EOF ``` [/details] -#### Setter + +#### Custom setters A question's setter is the function used to set new value/state. Custom setters are defined using bash functions called `setter__QUESTION_SHORT_KEY()`. In the context of the setter function, variables named with the various quetion's short keys are avaible ... for example the user-specified date for question `[main.main.theme]` is available as `$theme`. When doing non-trivial operations to set a value, you may want to use `ynh_print_info` to inform the admin about what's going on. + [details summary="Basic example : Set the system timezone" class="helper-card-subtitle text-muted"] `config_panel.toml` @@ -266,21 +266,23 @@ set__timezone() { ``` [/details] -## Validation -You will often need to validate data answered by the user before to save it somewhere. -Validation can be made with regex through `pattern` argument +## User input validations + +You will sometimes need to validate data provided by the user before saving it. + +Simple validation can be achieved using a regex pattern: ```toml - pattern.regexp = '^.+@.+$' - pattern.error = 'An email is required for this field' +pattern.regexp = '^.+@.+$' +pattern.error = 'An email is required for this field' ``` -You can also restrict several types with a choices list. +You can also restrict the accepted values using a choices list. ```toml - choices.foo = "Foo (some explanation)" - choices.bar = "Bar (moar explanation)" - choices.loremipsum = "Lorem Ipsum Dolor Sit Amet" +choices.foo = "Foo (some explanation)" +choices.bar = "Bar (moar explanation)" +choices.loremipsum = "Lorem Ipsum Dolor Sit Amet" ``` Some other type specific argument exist like @@ -290,26 +292,136 @@ Some other type specific argument exist like | `file` | `accept` | | `boolean` | `yes` `no` | -Finally, if you need specific or multi variable validation, you can use custom validators function: +See also : custom validators + +### Custom validators + +In addition to the "simple" validation mechanism (see the 'option' doc), custom validators can be defined in a similar fashion as custom getters/setters: + + ```bash validate__login_user() { - if [[ "${#login_user}" -lt 4 ]]; then echo 'User login is too short, should be at least 4 chars'; fi + if [[ "${#login_user}" -lt 4 ]] + then + echo 'User login is too short, should be at least 4 chars' + fi } ``` -## Other actions than read, validate and save -### Restart a service at the end -You can use the services key to specify which service need to be reloaded or restarted. + + + +## `visible` & `enabled` expression evaluation + +Sometimes we may want to conditionaly display a message or prompt for a value, for this we have the `visible` prop. +And we may want to allow a user to trigger an action only if some condition are met, for this we have the `enabled` prop. + +Expressions are evaluated against a context containing previous values of the current section's options. This quite limited current design exists because on the web-admin or on the CLI we cannot guarantee that a value will be present in the form if the user queried only a single panel/section/option. +In the case of an action, the user will be shown or asked for each of the options of the section in which the button is present. + +The expression has to be written in javascript (this has been designed for the web-admin first and is converted to python on the fly on the cli). + +Available operators are: `==`, `!=`, `>`, `>=`, `<`, `<=`, `!`, `&&`, `||`, `+`, `-`, `*`, `/`, `%` and `match()`. + +#### Examples + +```toml +# simple "my_option_id" is thruthy/falsy +visible = "my_option_id" +visible = "!my_option_id" +# misc +visible = "my_value >= 10" +visible = "-(my_value + 1) < 0" +visible = "!!my_value || my_other_value" +``` + +For a more complete set of examples, [check the tests at the end of the file](https://github.com/YunoHost/yunohost/blob/dev/src/tests/test_questions.py). + +#### match() + +For more complex evaluation we can use regex matching. + +```toml +[my_string] +default = "Lorem ipsum dolor et si qua met!" + +[my_boolean] +type = "boolean" +visible = "my_string && match(my_string, '^Lorem [ia]psumE?')" +``` + +Match the content of a file. + +```toml +[my_file] +type = "file" +accept = ".txt" +bind = "/etc/random/lorem.txt" + +[my_boolean] +type = "boolean" +visible = "my_file && match(my_file, '^Lorem [ia]psumE?')" +``` + +with a file with content like: +```txt +Lorem ipsum dolor et si qua met! +``` + + +## Actions + +"Actions" correspond to config panel buttons triggering specific pieces of code. For example, one could implement an action to trigger a scan of Nextcloud files, or install a plugin inside an app that does not already provide an interface to do so. In core config panels, buttons are for example used to trigger certificate renewal. + +The most basic example looks like this, using `type = 'button'`: ```toml -services = [ 'nginx', '__APP__' ] +[panel.section.my_action] +type = "button" +ask = "Run action" +# (NB: no need to set `bind` to "null") ``` -This argument can be set on a single question, to a section, or to an entire panel. +And then defining the controller, prefixed by `run__` inside the app's `config` script: +```bash +run__my_action() { + ynh_print_info "Running 'my_action'..." +} +``` -### Overwrite config panel mechanism +You may build more complex actions, where the actions uses other form inputs: + +```toml +[panel.my_action_section] +name = "Action section" + [panel.my_action_section.my_repo] + type = "url" + bind = "null" # this value won't be saved as a setting, it's ephemeral and only relevant for the action + ask = "gimme a repo link" + + [panel.my_action_section.my_repo_name] + type = "string" + bind = "null" # this value won't be saved as a setting, it's ephemeral and only relevant for the action + ask = "gimme a custom folder name" + + [panel.my_action_section.my_action] + type = "button" + ask = "Clone the repo" + # the button is clickable only once the previous values are provided + enabled = "my_repo && my_repo_name" +``` + +```bash +run__my_action() { + ynh_print_info "Cloning '$my_repo'..." + cd /tmp + git clone "$my_repo" "$my_repo_name" +} +``` + +## Overwrite config panel mechanism All main configuration helpers are overwritable, example: @@ -329,7 +441,7 @@ ynh_app_config_apply() { } ``` -List of main configuration helpers +List of main configuration helpers: * `ynh_app_config_get` * `ynh_app_config_show` * `ynh_app_config_validate` @@ -337,3 +449,23 @@ List of main configuration helpers * `ynh_app_config_run` More info on this can be found by reading [vpnclient_ynh config script](https://github.com/YunoHost-Apps/vpnclient_ynh/blob/master/scripts/config) + + + +## Important technical notes + +### Options short keys have to be unique + +For performance reasons, questions short keys have to be unique in all the `config_panel.toml` file, not just inside its panel or its section. Hence it's not possible to have: +```toml +[manual.vpn.server_ip] +[advanced.dns.server_ip] +``` +In which two questions have "real variable name" `is server_ip` and therefore conflict with each other. + +! Some short keys are forbidden cause it can interfer with config scripts (`old`, `file_hash`, `types`, `binds`, `formats`, `changed`) and you probably should avoid to use common settings name to avoid to bind your question to this settings (e.g. `id`, `install_time`, `mysql_pwd`, `path`, `domain`, `port`, `db_name`, `current_revision`, `admin`) + +### `bind` versus app settings + +! IMPORTANT: with the exception of `bind = "null"` options, options ids should almost **always** correspond to an app setting initialized/reused during install/upgrade. +Not doing so may result in inconsistencies between the config panel mechanism and the use of ynh_add_config. See also discussions in https://github.com/YunoHost/issues/issues/1973