Merge pull request #1727 from AppDaemon/toml_docs

Add Docs for TOML

docs/APPGUIDE.rst

@@ -149,10 +149,17 @@ Configuration of Apps
 ---------------------
 
 Apps are configured by specifying new sections in an app configuration
-file. The App configuration files exist under the apps directory and can be called anything as long as they end in ``.yaml``. You can have one single file for configuration of all apps, or break it down to have one ``yaml`` file per App, or anything in between. Coupled with the fact that you can have any number of subdirectories for apps and ``yaml`` files, this gives you the flexibility to structure your apps as you see fit.
+file. These configuration files can be written in either YAML or TOML, but must be the same type as the appdaemon configuration file, and which variant is used depends on the ``--toml`` flag
+supplied to AppDaemon at startup.
+
+The App configuration files exist under the apps directory and can be called anything as long as they end in ``.yaml`` or ``.toml``.
+You can have one single file for configuration of all apps, or break it down to have one configuration file per App, or anything in between.
+Coupled with the fact that you can have any number of subdirectories for apps and configuration files, this gives you the flexibility to structure your apps as you see fit.
+
 It should also be noted that a "dot" ``.`` is not allowed in the app name.
 
-The entry for an individual App within a ``yaml`` file is simply a dictionary entry naming the App, with subfields to supply various parameters. The name of the section is the name the App is referred to within the system in log files etc. and must be unique.
+The entry for an individual App within a configuration file is simply a dictionary entry naming the App, with subfields to supply various parameters.
+The name of the section is the name the App is referred to within the system in log files etc. and must be unique.
 
 To configure a new App you need a minimum of two directives:
 
@@ -164,15 +171,23 @@ To configure a new App you need a minimum of two directives:
 Although the section/App name must be unique, it is possible to re-use a
 class as many times as you want, and conversely to put as many classes
 in a module as you want. A sample definition for a new App might look as
-follows:
+follows in YAML:
 
 .. code:: yaml
 
     newapp:
       module: new
       class: NewApp
 
-When AppDaemon sees the following configuration, it will expect to find a
+The TOML equivqlent would look like this:
+
+.. code:: toml
+
+    [newapp]
+    module = "new"
+    class = "NewApp"
+
+When AppDaemon sees the above configuration, it will expect to find a
 class called ``NewApp`` defined in a module called ``new.py`` in the
 apps subdirectory. Apps can be placed at the root of the Apps directory
 or within a subdirectory, an arbitrary depth down - wherever the App is,
@@ -182,6 +197,7 @@ information about the path, just the name of the file itself (without
 the ``.py``) is sufficient. If names in the subdirectories overlap,
 AppDir will pick one of them but the exact choice it will make is
 undefined.
+
 When starting the system for the first time or when reloading an App or
 Module, the system will log the fact in its main log. It is often the
 case that there is a problem with the class, maybe a syntax error or
@@ -221,7 +237,7 @@ new App has been added, or if one has been removed, and it will act
 appropriately, starting the new App immediately and removing all
 callbacks for the removed App.
 
-The suggested order for creating a new App is to first add the apps.yaml entry
+The suggested order for creating a new App is to first add the app configuration file entry
 then the module code and work until it compiles cleanly. A good workflow is to
 continuously monitor the error file (using ``tail -f`` on Linux for
 instance) to ensure that errors are seen and can be remedied.
@@ -245,6 +261,16 @@ directives like so:
       param1: spam
       param2: eggs
 
+Or in TOML:
+
+.. code:: toml
+
+    [MyApp]
+    module = "myapp"
+    class = "MyApp"
+    param1 = "spam"
+    param2 = "eggs"
+
 Within the Apps code, the 2 parameters (as well as the module and class)
 are available as a dictionary called ``args``, and accessed as follows:
 
@@ -277,6 +303,28 @@ like this:
       sensor: binary_sensor.garage
       light: light.garage
 
+In TOML this would be:
+
+.. code:: toml
+
+    [downstairs_motion_light]
+    module = "motion_light"
+    class = "MotionLight"
+    sensor = "binary_sensor.downstairs_hall"
+    light = "light.downstairs_hall"
+
+    [upstairs_motion_light]
+    module = "motion_light"
+    class = "MotionLight"
+    sensor = "binary_sensor.upstairs_hall"
+    light = "light.upstairs_hall"
+
+    [garage_motion_light]
+    module = "motion_light"
+    class = "MotionLight"
+    sensor = "binary_sensor.garage"
+    light = "light.garage"
+
 Apps can use arbitrarily complex structures within arguments, e.g.:
 
 .. code:: yaml
@@ -321,6 +369,12 @@ An example ``secrets.yaml`` might look like this:
 
     application_api_key: ABCDEFG
 
+The equivalent ``secrets.toml`` would be:
+
+.. code:: toml
+
+    application_api_key = "ABCDEFG"
+
 The secrets can then be referred to in the ``apps.yaml`` file as follows:
 
 .. code:: yaml
@@ -330,6 +384,15 @@ The secrets can then be referred to in the ``apps.yaml`` file as follows:
       module: appmodule
       application_api_key: !secret application_api_key
 
+Or in TOML:
+
+.. code:: toml
+
+    [appname]
+    class = "AppClass"
+    module = "appmodule"
+    application_api_key = "!secret application_api_key"
+
 In the App, the api_key can be accessed like every other argument the App can access.
 
 Environment Variables
@@ -355,7 +418,15 @@ The variables can also be referred to in the ``apps.yaml`` file as follows:
       module: appmodule
       application_api_key: !env_var application_api_key
 
-In the App, the api_key can be accessed like every other argument the App can access.
+In the App, the api_key can be accessed like every other argument the App can access. This also works for TOML files:
+
+.. code:: toml
+
+    [appname]
+    class = "AppClass"
+    module = "appmodule"
+    application_api_key = "!env_var application_api_key"
+
 
 Include YAML Files
 ~~~~~~~~~~~~~~~~~~~~~
@@ -379,7 +450,7 @@ The tag can also be referred to in the ``apps.yaml`` file as follows:
       module: appmodule
       app_users: !include /home/ubuntu/dev/conf/app_users.yaml
 
-In the App, the app_users can be accessed like every other argument the App can access.
+In the App, the app_users can be accessed like every other argument the App can access, this works for TOML files also.
 
 App Dependencies
 ----------------
@@ -433,6 +504,16 @@ It is also possible to have multiple dependencies, added as a yaml list
         - Sound
         - Global
 
+In TOML this would be:
+
+.. code:: toml
+
+    [Consumer]
+    module = "sound"
+    class = "Sound"
+    dependencies = [ "Sound", "Global" ]
+
+
 AppDaemon will write errors to the log if a dependency is missing and it
 will also detect circular dependencies.
 

docs/CONFIGURE.rst

@@ -3,7 +3,23 @@ Configuration
 *************
 
 When you have AppDaemon installed using either Docker or ``pip``, you are ready to
-start working on the ``appdaemon.yaml`` file, its main configuration file.
+start working on the ``appdaemon`` configuration file, its main configuration file.
+
+Appdaemon Configuration File Format
+===================================
+
+The AppDaemon configuration file is usually a ``YAML`` file, however from appdaemon 4.3.0 and onwards, appdaemon's configuration file
+as well as the app configuration files can be spedicied in ``TOML`` rather than YAML. This behavior
+is global for all files and is turned on and off by the ``--toml`` flag when appdaemon is invoked. This behavior
+enables the user to easily switch between YAML and TOML files, although all config files muct be converted at the same time when moving from YAML to TOML.
+YAML and TOML configuration files are identical in function and capabilities, it is a matter of personal preference which format is used. At this time,
+TOML configuration is not available for HADashboard.
+
+A useful online resource for converting from YAML to TOML and back can be found at `transform tools <https://transform.tools/yaml-to-toml>`_.
+
+Configuring AppDaemon
+=====================
+
 If you are using Docker, you should already have a skeleton configuration generated the first-time the container is run, under the bind-mounted configuration directory.
 For ``pip`` users, create a configuration directory somewhere where all the AppDaemon data will be stored (e.g., ``/home/homeassistant/conf``) and create a file in there called ``appdaemon.yaml``.
 
@@ -24,7 +40,7 @@ The ``plugins`` section configures the communication with Home Assistant.
 
 .. code:: yaml
 
-    # appdaemon.yml
+    # appdaemon.yaml
     appdaemon:
       time_zone: CET
       latitude: 51.725
@@ -36,6 +52,24 @@ The ``plugins`` section configures the communication with Home Assistant.
           ha_url: <home_assistant_base_url>
           token: <some_long_lived_access_token>
 
+The same configuration in a TOML file would be called ``appdaemon.toml`` and would look like this:
+
+.. code:: toml
+
+  [appdaemon]
+  time_zone = "CET"
+  latitude = 51.725
+  longitude = 14.3434
+  elevation = 0
+
+  [appdaemon.plugins.HASS]
+  type = "hass"
+  ha_url = "<home_assistant_base_url>"
+  token = "<some_long_lived_access_token>"
+
+Both YAML and TROML files work in similar ways to express atomic values, lists and dictionaries, from this point on, some examples will be given in both formats, but the end-user
+is encouraged to learn the ins and outs of both formats to help in converting configurations from one format to another.
+
 Plugins
 -------
 
@@ -122,21 +156,21 @@ The following options are available under the ``appdaemon`` section:
   * - missing_app_warnings
 
       .. TODO: reference to ``apps.yaml` without having introduced it before
-    - AppDaemon by default outputs a warning if it finds a Python file that has no associated configuration in an ``apps.yaml`` file.
+    - AppDaemon by default outputs a warning if it finds a Python file that has no associated configuration in an app config file.
 
       Set this parameter to ``0`` to suppress the warning. This is useful for instance to distribute Python files not strictly related to AppDaemon along with AppDaemon apps.
     - No
 
   * - invalid_config_warnings
-    - AppDaemon by default outputs a warning if it finds an ``apps.yaml`` or ``apps.toml`` file that doesn’t include ``class`` and ``module`` for an app.
+    - AppDaemon by default outputs a warning if it finds an app config file file that doesn’t include ``class`` and ``module`` for an app.
 
       Set this parameter to ``0`` to suppress the warning.
       This is intended to ease the distribution of additional files along with apps.
     - No
 
   * - production_mode
-    - - ``false``: AppDaemon checks for changes in Apps and ``apps.yaml`` files every second. This can save some processing power on busy systems.
-      - ``true``: AppDaemon checks for changes in Apps and ``apps.yaml`` files only on restart
+    - - ``false``: AppDaemon checks for changes in Apps and app config files every second. This can save some processing power on busy systems.
+      - ``true``: AppDaemon checks for changes in Apps and app config files only on restart
 
       Defaults to ``false``.
 
@@ -417,13 +451,11 @@ AppDaemon's default time format is ``%Y-%m-%d %H:%M:%S.%f%z``.
 secrets
 =======
 
-AppDaemon supports the use of `secrets` in the configuration file, to allow separate storage of sensitive information such as passwords. For this to work, AppDaemon expects to find a file called ``secrets.yaml`` in the configuration directory, or a named file introduced by the top level ``secrets:`` section. The file should be a simple list of all the secrets. The secrets can be referred to using a !secret value in the configuration file.
-
 AppDaemon supports the use of `secrets` in the configuration file, to allow separate storage of sensitive information such as passwords.
-By default AppDaemon looks for a file called ``secrets.yaml`` in the configuration directory.
-You can configure AppDaemon to load a different secrets file by defining its path by defining a top-level ``secrets:`` configuration.
+By default AppDaemon looks for a file called ``secrets.yaml`` or ``secrets.toml`` in the configuration directory.
+You can configure AppDaemon to load a different secrets file by defining its path by defining a top-level ``secrets`` configuration.
 
-The file should be a simple list of all the secrets. The secrets can be later referred to using the ``!secret`` YAML directive in the configuration file.
+The file should be a simple list of all the secrets. The secrets can be later referred to using the ``!secret`` directive in the configuration file, this works for both YAML and TOML.
 
 An example ``secrets.yaml`` might look like this:
 
@@ -489,7 +521,7 @@ For example:
 
     app_dir: /etc/appdaemon/apps
 
-An example of the HASS plugin could look like the following:
+An example of the HASS plugin configured with YAML could look like the following:
 
 .. code:: yaml
 
@@ -516,6 +548,35 @@ An example of the HASS plugin could look like the following:
           cert_verify: True
           namespace: default
 
+Or in TOML:
+
+.. code:: toml
+
+  secrets = "/some/path"
+
+  [log]
+  accessfile = "/export/hass/appdaemon_test/logs/access.log"
+  errorfile = "/export/hass/appdaemon_test/logs/error.log"
+  logfile = "/export/hass/appdaemon_test/logs/appdaemon.log"
+  log_generations = 3
+  log_size = 1_000_000
+
+  [appdaemon]
+  threads = 10
+  time_zone = "<time zone>"
+  api_port = 5_000
+  api_key = "api_key"
+  api_ssl_certificate = "<path/to/root/CA/cert>"
+  api_ssl_key = "<path/to/root/CA/key>"
+
+  [appdaemon.plugins.HASS]
+  type = "hass"
+  ha_url = "<some_url>"
+  token = "<token>"
+  cert_path = "<path/to/root/CA/cert>"
+  cert_verify = true
+  namespace = "default"
+
 
 Authentication
 ^^^^^^^^^^^^^^
@@ -681,6 +742,28 @@ An example of the MQTT plugin could look like the following:
            - hermes/intent/#
            - hermes/hotword/#
 
+Or in TOML:
+
+.. code:: toml
+
+  [MQTT]
+  type = "mqtt"
+  namespace = "mqtt"
+  verbose = true
+  client_host = "Broker IP Address or DNS"
+  client_port = "Broker PORT Number"
+  client_id = "Client_ID"
+  client_user = "username"
+  client_password = "password"
+  ca_cert = "ca_cert"
+  tls_version = "auto"
+  client_cert = "mycert"
+  client_key = "mykey"
+  verify_cert = true
+  event_name = "MQTT_EVENT"
+  client_topics = [ "hermes/intent/#", "hermes/hotword/#" ]
+
+
 Creating a test app
 ===================
 
@@ -715,6 +798,14 @@ Then, we can create a file called apps.yaml in the apps directory and add an ent
       module: hello
       class: HelloWorld
 
+Or if we are using TOML:
+
+.. code:: toml
+
+  [hello_world]
+  module = "hello"
+  class = "HelloWorld"
+
 App configuration is fully described in the `API doc <AD_API_REFERENCE.html>`__.
 
 With this app in place we will be able to test the App part of AppDaemon

docs/INSTALL.rst

@@ -204,7 +204,8 @@ The following CLI arguments are available:
     -D {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --debug {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                             global debug level
     -m MODULEDEBUG MODULEDEBUG, --moduledebug MODULEDEBUG MODULEDEBUG
-    -v, --version         show program's version number and exit
+    -v, --version           show program's version number and exit
+    --toml                  use toml file format for configuration files
 
 A brief description of them follows:
 
@@ -216,7 +217,7 @@ A brief description of them follows:
 
     If no file is found in either location, AppDaemon will raise an exception. In addition, AppDaemon expects to find a dir named ``apps`` immediately subordinate to the config directory.
 
-``-C`` name of the configuration file (default: ``appdaemon.yaml``)
+``-C`` name of the configuration file (default: ``appdaemon.yaml`` or ``appdaemon.toml`` depending on the value of the ``--toml`` flag)
 
 .. TODO: document -d in appdaemon help text