2.12.0

Updated dependencies

• Netty 4.1.111->4.1.112
• Apache commons 3.14.0 -> 3.15.0
• MSSQL 12.6.2.jre11 -> 12.6.3.jre11

Minor changes

• Telnet, rtvals cmd now has alternating color for the val listing
• admin:phypower now supports rtl chips and checks if root privilege first.
• Taskmanagers can now be under dcafs node instead of settings or taskmanagers
• Removed IssuePool, wasn't used anyway.

Fixes

• i2c:id,xml didn't reload if the script already exists

MQTT

• Increased retry interval to 25s because lower causes 'already in progress' error.
• Ttl can be set broker wide and the status message gives more info.
• Added mqtt:brokerid,debug and mqtt:brokerid,debug,true/false to read and set debugging. For now the only
  difference with enabled is log messages if data is received or send.
• Result of mqtt:brokerid,stores is now sorted on topic.

Paths

• Changed how 'if' nodes work to be more in line with regular programming. Successive nodes will be executing
  instead of previously more like switch case.
• Added 'case' node to mimic how if used to work.
• Rewrote how the steps are interconnecting.
• Added some filter/if options
  • minitems : minimum amount of items after split
  • maxitems : maximum amount of items after split
  • !contains: doesn't contain the given data
• Added aliases for some filters
  • !start alias for notstart
  • !contains alias for contains
  • includes alias for contains

Rtvals

• Improved dynamic units, but breaking change:
  • Now allows for starting with any of the units instead of only base.
  • Can now go up or down a unit instead of only up.
  • Replaced attribute 'from' with max/till, should be a bit clearer (this is the breaking part).
  • Can now have a scale for each individual unit.
  • It's possible to repeat units to add different scale settings.
  • Div is set globally or per unit.
  • Attribute digits and scale are the same thing, use according to preference.

<unit base="Wh" div="1000"> <!-- default divider/multiplier is 1000 -->
  <level max="100"  scale="2" >mWh</level> <!-- Up to 100mWh use two digits, no div done to next step -->
  <level max="1000" digits="1" >mWh</level> <!-- From 100 till 1000mWh use one digit -->
  <level max="1500" scale="2" >Wh</level>
  <level            scale="3" >kWh</level>
</unit>

• Fixed, reloading a store first removed the rtvals defined in it from the general pool. However this was also done if
  that rtval was defined in the general node. Now those aren't removed anymore.
• After a rtvals:reload, the paths and databases will also be reloaded.

DBM

• When editing a sqlite db in another program, this causes the current connection to break (errorcode 8). Reconnecting resolves this, so
  now dcafs will try this. If this fails, query content is dumped to a file tablename_dump.csv in the same folder as the
  sqlite.
• Fixed, Mariadb doesn't like the use of ", replaced with `. Which also works for sqlite.
• Fixed, MariaDB returns a different error code on batch errors, now properly handled
• Fixed, Localnow column wasn't local when send to sql server.
• dbm:prep existed but wasn't documented. Gives total procesed queries count.
• Added dbm:reloadall to reload all databases in one go
• Added dbm:clearerrors to clear the error counters

2.11.0

Earlier then planned because of a telnet issue.

• Will now log an error if a commandable with same id is added. For example, taskmanager and stream share an id.
• Fixed, editor just kept working with the data of the previous step if a step failed, now aborts.
• Fixed, tablename's can't start with a number if not surrounded by " in queries.

Telnet

• Fixed, telnet start cmd was executed but result not requested nor worked well with default id.
• Added id? cmd to request current id.
• Added es, adds the elapsed time since last received data.
• Fixed, if the result is 25 lines or longer the output was colored by line. This shouldn't be done
  if it contains telnet codes (fe. st reply).
• Fixed, if dcafs starts without settings.xml, path for tinylog remained null. Because of this, telnet clients
  couldn't connect.

MQTT

• Fixed, addbroker cmd didn't write to xml.
• Removed use of label.
• If a topic contains \ it will be replaced with /
• Added cmd mqtt:brokerid,generate,topic this will generate store entries for messages received
  that match the topic (use wildcard!). Data received will determine data type: int,real or txt.
• Breaking, removed the use of defaulttopic. Added more possible confusion than actual use.
• Can use a group node in store node like global rtvals.

Valstore

• db attribute now allows multiple tables and multiple id:table sets. Can be handy if a single store
  needs to write to multiple tables.Mo
  • For example db="db1:table1,table2" or db="db1:table1;db2:table2"

Rtvals

• Fixed, realval couldn't get an op via node.
• Vals now share the code that reads group/name and name can be attr,content or node.
• Unit node can now be used as default unit/scale setup for vals.

<rtvals>
    <!-- If the name of the val matches the nameregex, the unit/scale will be applied -->
    <!-- This won't overwrite unit/scale settings --> 
    <unit base="°C" nameregex="temp.*" scale="1" />
    <unit base="%"  nameregex="hum.*"  scale="1" />
</rtvals>

2.10.0

This release is mainly a rewrite (again) of the i²c code and expanding on the gpio code. Next major point is adding
more feedback in the status report for possible issues.

• (If) dcafs runs with root permissions files created are only writable by root. This is not needed,
  so now new xml files are created with 'others write' permission. But owner remains.
• Changed default telnet port to 2323, because 23 requires root on linux.
• Happened that tinylog stopped writing for unknown reason, added the age of the daily raw
  file to the status report. Default is one hour, can be changed with maxrawage node in settings.
• Base path of tinylog can be set in xml using tinylog node in the settings node..
• By default, dcafs now runs a global check every hour to see if there are (possibly unreported issues). Interval can
  be altered or set to 0s to disable it. Only works if email/matrix is setup, because no use checking if can't be reported.

 <settings>
   <!-- Settings related to the telnet server -->
   <telnet port="2323" title="DCAFS">
     <textcolor>lightgray</textcolor>
   </telnet>
   <maxrawage>10m</maxrawage> <!-- Max age of raw data to consider good -->
   <tinylog>/mnt/sd</tinylog> <!-- Store logs on sd card -->
   <statuscheck>
     <interval>1h</interval> <!-- time interval to run a general check -->
     <email>admin</email> <!-- id of email recipient -->
     <matrix>playground</matrix> <!-- id of the matrix room -->
   </statuscheck>
 </settings>

• Status report has been updated to include some more info and unified appearance.
• Fixed, matrix:roomid,txt/say required the room id to have matrix prepended

Telnet

• CTRL+s can be used to send things to streams without eol sequence
• Using the ESC key will actually send and ESC
• Fixed, rewrote part of the cli code to make it less error prone. Things tended to break
  when the length of the text equals the size of the buffer and edits are done.
• Will show message on opening session if raw data is abnormally old.

Interrupts

• Can now add gpio if the board isn't recognised by diozero.
• Added admin:chexkpins to check which gpio are recognized
• Breaking, changed the xml format to fit this extra info

<gpios chiplines="32">
  <!-- defining a gpio if board is unknown -->
  <gpio name="GPIO2_A3" chip="2" line="4">
    <physical header="X1" pin="13"/>
    <modes>digital_input</modes>
    <interrupt edge="falling">
      <cmd>info:hello?</cmd>
    </interrupt>
  </gpio>
  <!-- defining a gpio board is known -->
  <gpio name="GPIO2_A4"> <!-- name needs to match -->
    <interrupt edge="falling">
      <cmd>info:hello 2?</cmd>
    </interrupt>
  </gpio>
</gpios>

• Added more actions to do on an interrupt:
  • counter: counts the amount of pulses detected
  • frequency: calculates the frequency of the pulses, based on moving average or single interval
  • period: measures the length of a pulse of minimum length of 500µs with 100µs accuracy.

    <gpio chip="2" line="3" name="GPIO2_A3">
      <physical header="X1" pin="13"/>
      <modes>digital_input</modes>
      <interrupt edge="rising">
	    <counter>aws_rain</counter> <!-- count the amount of pulses, stores it in a int rtval -->
		<frequency samples="5">aws_frequency</frequency> <!-- Calculates the frequency of the pulses with moving average -->
      </interrupt>
    </gpio>

• Because these update an rtval, the triggers on said rtval can be used.

I2C

• Added attribute 'datatype' to the write op. Allows to change datatype of the content to dec or ascii instead
  of the default hex.
• Added attribute 'addsize' to write node, default false. If true appends the amount of bytes to send
  after the register.
• Debug state is now carried over on reload.
• When adding devices, now there's a check for duplicate address&bus.

I2C Uart

• Basic implementation to test https://github.com/michieltjampens/i2c_uart_stm32
• For now, only sending and receiving data works.

  <i2c>
    <bus controller="1">
      <uart id="uart1">        
        <address>0x1b</address>
		<eol>crlf</eol>
		<irq>GPIO2_A3</irq>
		<serialsettings>38400,8,1,none</serialsettings>
      </uart>
	  <uart id="uart2">        
        <address>0x2b</address>
		<eol>crlf</eol>
		<irq>GPIO2_A3</irq>
		<serialsettings>38400,8,1,none</serialsettings>
      </uart>
    </bus>
  </i2c>

DBM

• Database can now have the node maxinsertage which determines what the max time since the last insert is. When beyond
  this, the status will turn red in the st cmd result.
• When the amount of queries buffered is higher than the max, this will also turn red.

2.9.1

Mainly bug fix release.

Tasks

• Task with output stream allows for waiting for reply. The window could be altered with replywindow. But this
  changed attempts to one. Now the attribute allows for attempts to be specified replywindow="5s,3" meaning 3 attempts wih 5s window.
  If no amount is given, it's set to 1.
• Fixed, args given when running a cmd included the command instead of actual first arg.
• Added {mint2s16b:operation} to calculate 2s complement of a 16b operation.

Paths

• Fixed, pf:id,delete now works again. Was changed to pf:id,delete,all/last.
• Fixed, pf:id,reload and pf:id,list weren't altered properly from pf:reload,id
• Fixed, if as first step wasn't working properly
• Paths will now be reloaded after changing them (instead of having to do manually).
• pf:id,list now shows db the store's are writing to and textcolor is now default instead of all green.
• Fixed, delimiter in a edit node wasn't processed for potential chars like \t etc.
• Added cmd to request the data of a step in a path, (similar to debug) pf:pathid,stepid

2.9.0

Streams

• Fixed, changing the ttl of a stream no longer reloads it, applies it instead.
• Can now send data to a stream using the id instead of Sy.
• Can now send ESC to a stream using the esc key (eol won't be appended)
• SerialStream, Added the attribute flush to ttl node. If true, this means that any data in the buffer will be forwarded
  as if the eol was received if the ttl has passed. This can be useful for situations in which a device sends a prompt without eol.
  Or if unsure about the eol but still want data to be more or less per message instead of as received.

TaskManager

• Added {utc:format} as possible fill in value for a task value. This will get replaced with the current utc time
  according to the format (if valid).
• Can now use {group_name} in the content of a task instead of having to specify type of val. It will just take
  a bit longer because it has to search in more collections.

Store

• Added store:streamid,alterval,valname,op,operation to add ops to a val.
• If no index is provided for the addi/addr/addf or addt, one will be generated:
  • If it's the first node, it will be 0.
  • If other nodes are already present, it will be the index of the last one +1.

Database

• Replaced the use of the enum Rolloverunit with the java ChronoUnit.
• Fixed, filename of the sqlite with rollover in status overview didn't match the actual name on short periods.
• Breaking, combined rollover attributes to make it a bit more straightforward. Bot SQLite and FileCollector use this.

<!-- Previous -->
<rollover count="5" unit="minutes">_HHmm</rollover>
<!-- Now -->
<rollover period="5 minutes">_HHmm</rollover>

Other

• Editor redate now supports epochmillis and epochseconds. Use 'epochmillis' or epochsec(onds) as inputformat.
• Fixed, flagval and textval update command was checking for id in old index so didn't work.
• RealVal and Intval can now be created with a command.
• Updated dependencies:
  • netty 4.1.109
  • jSerialcom 2.11.0
  • mssql 12.6.1.jre11
  • postgresql 42.7.3

2.8.1

• Updated netty,activation and json dependency
• Can now request realtime updates of flags with flag:id.
• The !! function of the telnet interface now works additive. So if a prefix was active, it can be appended to.

>dbm:rolls,!!

Prefix changed to 'dbm:rolls,'
dbm:rolls,>addcal,dice,!!

Prefix changed to 'dbm:rolls,addcal,dice,'
dbm:rolls,addcal,dice,>

I2C

• Changed i2c code to using doubles to actually work with numbers of 32bits till 63bit.
• Changed i2c:reload to do a full reload including looking for devices on the bus
• Fixed, requesting data in telnet couldn't be stopped

2.8.0

• Fixed, Matrix out of bounds when sending something without room url etc.
• IntVal, now accepts real for parsing if 'allowreal' attribute is set to true. Will round according to math rules. Will
  now give an error if this or regular parsing fails.
• Rewrote I2C code

Dependencies

• Netty 4.1.101 -> 4.1.106
• jSerialCom 2.10.3 -> 2.10.4
• commons-lang3 3.12.0 -> 3.14.0

I2C

• Rewrite in progress, mainly to make the code easier to understand and adapt. All the basics work.
  • Read/Write/alter registers, delay before executing read.
• Now ops are stored inside the device instead of globally, so each device has their own set.
  Doesn't make a difference if only one device was using the script. But now multiple can use the same
  script but with custom store's.
• Now allow for read data to be split according to bit list, fe. 8,20,20,8 means read 7 bytes and split those according
  to that sequence.
• Removed use of label.
• Can now use math and store node inside the scripts.
  Breaking changes
• The return attribute in the read node, now is based on the set amount of bits. So return now refers to the amount of
  times that bits is returned.... So if bits is 16 and you read one of that, return used to be two and now it's one.
• The main node is renamed from commandset to i2cscript and script attribute to id.
• The subnodes are changed to 'i2cop' from 'command' because command is already in use for other things

Admin cmds

• Added a cmd that uses phytool to power down or up a phy, could be used to save power.
  User must install the tool first.

Vals

• Added 'Dynamic Units' so it's possible to alter the unit depending on the amount.
  There are two options, either 'step' for integers or 'level' for real

 <!-- For example the unit is a time period in seconds -->
  <rtvals>
    <unit base="s"> <!-- the unit used  -->
      <step cnt="60">m</step> <!-- the next step up 60s  to 1m -->
      <step cnt="60">h</step> <!-- the next step up 60m to 1h -->
    </unit>
    <unit base="Hz">
      <level div="1000" from="1500">kHz</level> <!-- Use A if the value is higher than 1500 and use 1000 as divider -->
      <level div="1000">MHz</level> <!-- No 'from' so same as div, so kHz -->
      <level div="1000">GHz</level>
    </unit>
</rtvals>
<!--
So an input of 3962s will result in 1h6m2s shown instead.
Or an input of 1400Hz will result in 1400Hz 
    but 1840mA will become 1.840kHz
    and 1245358Hz will become 1.245MHz, scale is taken from the realval 
-->

Paths

• Changed pf:list so it actually gives a list of active paths instead of also listing steps in it
• Added pf:id,list to give all the steps in the path with the provided id

TaskManager

• Reordered parameters in some commands, so it's id first if it's interacting with that id, so tm:id,cmd.
• Interval without an initial delay is now actually without a delay... (was 10% of interval)
• Made it possible to refer to a trigger with an attribute of the type instead of the trigger attr. Should
  be a bit more intuitive.

<tasklist>
  <!-- Both tasks do the same -->
  <task output="stream:powmon" trigger="interval:10m">ac1</task>
  <task output="stream:powmon" interval="10m">ac1</task>
</tasklist>

2.7.0

Biggest changes are a rewrite of the code that reads the path xml to allow for nested 'if's and the functionality to provide arguments when starting a task through cmd.

• Fixed, TCP streams no longer had the label applied.
• Fixed, the reply to a cmd in telnet wrote on the same line as the given cmd
• Fixed, serialport buffers seem to be filled even if nothing is listening, all that data gets dumped on connection. So
  flush the buffers on opening the port.
• Added ss:id,port,newport to change the port of the stream
• Added option to prefix received data lines with the id of their origin. Activated by adding the node in
  the stream with true as content. Can be handy if you get data from different src's at the same time.
• Telnet, Command history (up to 20) in telnet is now saved in ram on ip basis, can be cleared by sending >>clearhistory or >>clrh
• Telnet, When cmds are issued from non-telnet, the telnet escape codes are removed. Adding -r at the end has the same result.

Matrix

• Room text updates can now be requested with the 'matrix:roomid' cmd.
• Changed cmd matrix:say,roomid,message to matrix:roomid,say,message.
• Allow for cmds to be executed on joining a room with <cmd></cmd node inside room node.

I2C

• It's now possible to add extra arguments to a i2c cmd to set data send. Next step applying a
  math operation to the argument first.

	<command id="init" info="Set default control, not enabled, no oc retry, retry on ov and uv">
		<write reg="0xD0">i0</write> <!-- i0 will get replaced with the first arg -->
		<write reg="0xD4">0x13</write>
	</command> 

Path

• Rewrote the code that reads paths from xml
• Fixed, type nmea was ignored because it was expecting a value.
• It's now possible to add filter rules with attributes
• Added a new tag 'if', this is actually a filter but one that allows nesting, this should make it a bit more intuitive... right?
  • The only node that expects other forwards to be inside it. In other words, it allows nesting.
  • Given how to code works, it should be possible to add multiple levels...
  • A If doesn't pass data on to a next step, instead the step after the if, get the same data as the if

<if start="$GPVTG">
  <store>
    <real def="0" i="1" unit="°">cog</real>
    <real def="0" i="5" unit="kn">sogknots</real>
    <real def="0" i="7" unit="m/s">sogms</real>
  </store>
</if>

• Added new filter rule 'atx', check if value at index is a certain value

<if at1="hello"> <!--start at 0 -->
</if>

• Filter often has to send to multiple writables, so now uses concurrency to speed things up a bit. Which means that if
  the path contains multiple

Store

It's now possible to add math operations with values that are in the store.

<store>
      <real group="o1" i="1" unit="V">voltage</real>
      <real group="o1" i="2" unit="mA">current</real>
     <!-- calculate the product of the voltage and current to get the power in watt -->
      <real group="o1" o="(o1_voltage*o1_current)/1000" unit="W">power</real> 
</store>

TaskManager

• Allow for tasks started from cmd to include arguments to replace i0, i1 etc. This includes the
  functionality to do math operations.
  For example:

<!-- Part of the taskmanager 'io' -->
<!--
{mint:formula} -> calculate the formula and return an integer 
{math:formula} -> calculate the formula and return a real
-->
<task id="go" output="stream:test" >ovli0:{mint:i1*(65535/32)}</task>

When using the cmd io:go,1,3.3 this will result in ovl1:6758 being written to test.

2.6.0

Fixes

• FlagVals weren't sending updates to targets.
• ss:id1,tunnel,id2 command was using wrong indexes.
• Mathforward didn't set all values in the store if they weren't used by it.
• I2C cmds weren't read correctly from script, kept using the write/read etc from the first cmd.
• Telnet text color was read from attribute instead of element.

Updated deps

• jSerialcom
• Netty
• SQLite

Store implementation

• General code cleanup
• Removed 'store' from the BaseStream and replaced this with a new collector 'StoreCollector'. That way it's works the same
  as any other data processing method (making it easier to understand and debug).
• A forward or storecollector can now request a link to a specific sqltable to trigger an insert.

Database

• Rewrote the way the 'store' and 'database' interact. This was based on cmd but this gives issues when processing
  at high speed. Now an insert is triggered directly instead when using forwards and streams.

MQTT

• Added cmds for the mqtt provide and store
• Changed subscribe/unsubscribe cmds to no longer need to save afterwards
• Removed cmd to save settings (old store)

2.5.2

• Fixed, taskmanager scripts had different node paths for storing and reading.

MQTT

• Added option to have mqtt topic linked to rtvals, no cmds for it yet

<broker id="mosq">
      <store> <!-- to store the value of a topic in a rtval -->
        <int topic="dice/rolled" group="dice">rolled</int>
      </store>
      <provide> <!-- to provide a rtval when updated this will be send to mqtt broker -->
        <rtval>dice_rolled</rtval> <!-- uses group/name as topic so dice/rolled -->
        <rtval topic="dice/d6">dice_rolledd6</rtval> <!-- or alternative topic -->
      </provide>
</broker>