Merge pull request #4777 from tonhuisman/feature/TAR-archive-upload-a…

…nd-download

[TAR] Add support for downloading and uploading files via .tar archive

docs/source/Plugin/P011.rst

@@ -1,4 +1,4 @@
-.. include:: ../Plugin/_plugin_substitutions_p01x.repl
+.. include:: ../Plugin/_plugin_substitutions_p01x.repl
 .. _P011_page:
 
 |P011_typename|
@@ -26,22 +26,83 @@ Supported hardware
 
 |P011_usedby|
 
-.. Commands available
-.. ^^^^^^^^^^^^^^^^^^
+This plugin supports the Pro Mini Extender, that's a software solution installed on an Arduino Nano, providing 5 available analog input/output pins and 14 digital input/output pins.
 
-.. .. include:: P011_commands.repl
+The Pro Mini Extender software has to be manually installed on the Arduino Nano (or a Chinese clone), using the ``MiniProExtender`` software project from this `ESPEasySlaves repository <https://github.com/letscontrolit/ESPEasySlaves>`_ via the Arduino IDE.
+
+After installing the software on the Arduino Nano, the simplest solution is to power the PME with 3.3V, even if it's specified for 5V, so no level converter is needed for the I2C connection with the ESP, that only allows for 3.3V signal levels. The 3.3V power has to be connected to the 5V pin on the Arduino Nano, **not** on the 3.3V pin, as that's a low-power output-only pin!
+
+NB: This software can also be installed on other Arduino models, that support I2C, Analog and Digital pins, but this hasn't been actively tested. Depending on the available IO pins, some of the features may not match with the Arduino Nano.
+
+Device Configuration
+--------------------
+
+.. image:: P011_DeviceConfiguration.png
+
+* **Name** A unique name should be entered here.
+
+* **Enabled** The device can be disabled or enabled. When not enabled the device should not use any resources.
+
+Sensor
+^^^^^^
+
+* **Port** Select the port (pin) of the Pro Mini Extender that is addressed by this task. For a *Digital* or *Input (Switch)* the range is 0..13, for an *Analog* input, the available pins are 0..3 and 6..7, as A4 and A5 are the (fixed) I2C pins of the Arduino Nano.
+
+I2C Options 
+^^^^^^^^^^^^
+
+The available settings here depend on the build used. At least the **Force Slow I2C speed** option is available, but selections for the I2C Multiplexer can also be shown. For details see the :ref:`Hardware_page`
+
+Device Settings
+^^^^^^^^^^^^^^^
+
+* **Port Type**:
+
+.. image:: P011_PortTypeOptions.png
+
+* *Digital*: An On/Off Input/Output type of port, every **Interval** the pin is read and the state is made available in the **Value** field.
+
+* *Analog*: Read the current analog value of the port.
+
+* *Input (switch)*: Act like an input switch, the pin is read every 20 msec, and if the state changes, the new state is reported as an event, with the new value. The **Interval** is ignored when this Port Type is selected.
+
+Data Acquisition
+^^^^^^^^^^^^^^^^
+
+This group of settings, **Single event with all values**, **Send to Controller** and **Interval** settings are standard available configuration items. Send to Controller is only visible when one or more Controllers are configured.
+
+**Interval** By default, Interval will be set to 60 sec. It is the frequency used to read sensor values and send these to any Controllers configured for this device.
+
+Values
+^^^^^^
+
+The single **Value** available holds the last digital, analog or input (switch) state value.
+
+Commands available
+^^^^^^^^^^^^^^^^^^
+
+.. include:: P011_commands.repl
 
 .. Events
 .. ~~~~~~
 
 .. .. include:: P011_events.repl
 
+Get Config Values
+^^^^^^^^^^^^^^^^^
+
+Get Config Values retrieves values or settings from the sensor or plugin, and can be used in Rules, Display plugins, Formula's etc. The square brackets **are** part of the variable. Replace ``<taskname>`` by the **Name** of the task.
+
+.. include:: P011_config_values.repl
+
 Change log
 ----------
 
 .. versionchanged:: 2.0
   ...
 
+  |added| 2024-03: Add Input (switch) option.
+
   |added|
   Major overhaul for 2.0 release.
 

docs/source/Plugin/P011_commands.repl

@@ -0,0 +1,53 @@
+.. csv-table::
+        :header: "Command", "Extra information"
+        :widths: 20, 30
+
+        "
+        ``extgpio,<pin>,<0|1>``
+
+        ``<pin>`` : The IO pin on the PME board, range 0..13 for Digital output.
+
+        ``<0|1>`` : Select 0 for Off (low), and 1 for On (high) level output
+
+        ","
+        Switch the pin to either low or high level output.
+        "
+        "
+        ``extpwm,<pin>,<level>``
+
+        ``<pin>`` : The IO pin on the PME board, range 0..7 for Analog output.
+
+        ``<level>`` : The PWM level, range 0..255
+
+        ","
+        Set the Analog pin (0..7) to the PWM level, where 0 = 0% and 255 = 100% of the VCC voltage the PME is running at.
+        "
+        "
+        ``extpulse,<pin>,<0|1>,<duration>``
+
+        ``<pin>`` : The IO pin on the PME board, range 0..13 for Digital output.
+
+        ``<0|1>`` : Select 0 for Off (low), and 1 for On (high) level output
+
+        ``<duration>`` : The time in milliseconds the IO pin state should be set, after which it is restored in the previous state.
+        ","
+        Switch the pin to either low or high level output, and after the duration has passed, return to the previous state.
+        "
+        "
+        ``extlongpulse,<pin>,<0|1>,<duration>``
+
+        ``<pin>`` : The IO pin on the PME board, range 0..13 for Digital output.
+
+        ``<0|1>`` : Select 0 for Off (low), and 1 for On (high) level output
+
+        ``<duration>`` : The time in **seconds** the IO pin state should be set, after which it is restored in the previous state.
+        ","
+        Switch the pin to either low or high level output, and after the duration (seconds!) has passed, return to the previous state.
+        "
+        "
+        ``status,ext,<pin>``
+
+        ``<pin>`` : The IO pin on the PME board, range 0..13 for Digital output, range 20..27 for the Analog pins A0..A7.
+        ","
+        Report the current state/value for the pin selected.
+        "

docs/source/Plugin/P011_config_values.repl

@@ -0,0 +1,20 @@
+.. csv-table::
+    :header: "Config value", "Information"
+    :widths: 20, 30
+
+    "
+    ``[<taskname>#D,<port>]``
+
+    ``<port>`` : Range 0..13, corresponding with D0..D13 on the PME.
+    ","
+    Returns the current status (0/1) from the requested Digital port (pin) of the Pro Mini Extender.
+    "
+    "
+    ``[<taskname>#A,<port>]``
+
+    ``<port>`` : Range 0..3 and 6..7, corresponding with A0..A7 on the PME.
+    ","
+    Returns the current value (0..1023) at the requested Analog port (pin) of the Pro Mini Extender.
+
+    Port A4 and A5 should be avoided, as that's occupied by the I2C connection at the PME. There is no check in the code to block those pins!
+    "

docs/source/Reference/Command.rst

@@ -195,10 +195,10 @@ P009 :ref:`P009_page`
 .. .. include:: ../Plugin/P010_commands.repl
 
 
-.. P011 :ref:`P011_page`
-.. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+P011 :ref:`P011_page`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. .. include:: ../Plugin/P011_commands.repl
+.. include:: ../Plugin/P011_commands.repl
 
 
 P012 :ref:`P012_page`

docs/source/Tools/Tools.rst

@@ -233,9 +233,7 @@ Rules Settings
 --------------
 
 * Rules - Check to enable rules functionality (on next page load, extra Rules tab will appear)
-* Old Engine - Default checked.
 * Enable Rules Cache - Rules cache will keep track of where in the rules files each ``on ... do`` block is located. This significantly improves the time it takes to handle events. (Enabled by default, Added 2022/04/17)
-* Allow Rules Event Reorder - It is best to have the rules blocks for the most frequently occuring events placed at the top of the first rules file. (also for frequently happening events, which you don't want to act on) The cached event positions can be reordered in memory based on how often an event was matched.  (Enabled by default, Added 2022/04/17, disabled 2022/06/24)
 * Tolerant last parameter - When checked, the last parameter of a command will have less strict parsing.
 * SendToHTTP wait for ack - When checked, the command SendToHTTP will wait for an acknowledgement from the server.
 * SendToHTTP Follow Redirects - When checked, HTTP calls may follow redirects. Strict RFC2616, only requests using GET or HEAD methods will be redirected (using the same method), since the RFC requires end-user confirmation in other cases.
@@ -469,6 +467,19 @@ Added: 2023-07-20
 
 When Rules auto-completion, also including syntax highlighting, is available in the build, some users have difficulty working with the auto-completion. This option disables the auto-completion, and that also inhibits the syntax highlighting as these 2 features are closely integrated.
 
+Disable Save Config as .tar
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Added: 2023-08-25
+
+Only available in builds that have .tar support included!
+
+By default, using the Tools/Save button, the complete configuration will be downloaded as a single .tar archive, that includes all configuration files (``config.dat``, ``security.dat``, ``provisioning.dat``, ``notification.dat``, ``rules1.txt`` .. ``rules4.txt`` and any task-specific CustomSettings ``extcfg<NN>.dat``).
+
+Enabling this option allows to download *only* the ``config.dat`` file (renamed to include unit name, unit number, buildnumber and current date/time), to accommodate external systems/scripts that expect only the .dat file.
+
+The Tools/Backup files feature will still download all files stored on the Flash file system, independent of this setting, and also the Tools/Load and Tools/File browser/Upload buttons will extract the included files from an uploaded .tar file.
+
 Deep Sleep Alternative
 ^^^^^^^^^^^^^^^^^^^^^^
 
@@ -888,6 +899,8 @@ Then it does not make sense to have the client timeout of that controller set to
 System Variables
 ================
 
+Interfaces
+==========
 
 I2C Scan
 ========
@@ -910,6 +923,33 @@ Example scan using an I2C multiplexer, showing multiple devices across multiple
 .. note:: On builds that have ``LIMIT_BUILD_SIZE`` set, like the ESP8266 Collection and Display builds, the names of the supported devices and plugins are **not** included in the output, only the address(es) are listed.
 
 
+Settings
+========
+
+The :cyan:`Load` button will allow to load files onto the Flash file system. If you want to restore a previously saved ``config.dat``, the downloaded file has to be renamed to exactly ``config.dat`` and uploaded.
+
+Since 2023-08-25, .tar archive support has been added and made available in most builds, allowing to download and upload the complete configuration, and even all files on the flash file system, as a single archive, for backup and restore/clone purposes. This makes it possible to more easily deploy a unit using a pre-configured configuration.
+
+Uploading an earlier created backup as a .tar file, will unpack all files in the root of the archive to the flash file system, *overwriting* any files that already exist. If the archive includes ``config.dat`` and the Extended CustomTaskSettings feature is available, any already existing ``extcfg<NN>.dat`` file that's not included in the archive will be removed, as that is part of the configuration, and these files can not be deleted manually.
+
+Any files in subdirectories in the archive will be ignored, as directories are not supported on the flash filesystem.
+
+The :cyan:`Save` button offers to download the configuration of the unit. If .tar file support is included in the build, by default all configuration files (``config.dat``, ``security.dat``, ``provisioning.dat``, ``notification.dat``, ``rules1.txt`` .. ``rules4.txt`` and any task-specific CustomSettings ``extcfg<NN>.dat``) will be included, if they exist, in the .tar archive that can be downloaded.
+
+If .tar file support is not included, or the Tools/Advanced option **Disable Save Config as .tar** is enabled, only the ``config.dat`` file will be downloaded.
+
+The :cyan:`Backup files` button is only available if .tar file support is included in the build, and offers to download a .tar archive containing all files on the flash file system. These can be stored as a backup and restored in case of some configuration or system failure, or used to create 1 or multiple clones of the unit for multi-deployment. Uploading can also be started from an automation system or script, POST-ing the .tar archive from an external source.
+
+Firmware update
+===============
+
+Via the :cyan:`Update Firmware` button, you can browse for an updated firmware, downloaded from the Releases page, an Actions run, or self-built, and install that. When using the same flash configuration (``4M1M``, ``4M316k``, ``8M1M``, etc.) all settings will be preserved. When uncertain, the configuration can be saved using either the Save (or Backup files if available) button above.
+
+File system
+===========
+
+Via :cyan:`File browser` you can browse the files on the flash file system, download them separately, upload additional files, or delete any non-system files.
+
 Factory Reset
 =============