TerkinData: Adjust compatibility for AVR, Espressif, and ARM

.gitmodules

@@ -60,10 +60,6 @@
 [submodule "tools/makeEspArduino"]
 	path = tools/makeEspArduino
 	url = https://github.com/plerup/makeEspArduino
-[submodule "libraries/ArduinoSTL"]
-	path = libraries/ArduinoSTL
-	url = https://github.com/hiveeyes/ArduinoSTL
-	branch = operator-fixes
 [submodule "libraries/Adafruit_MQTT_Library"]
 	path = libraries/Adafruit_MQTT_Library
 	url = https://github.com/adafruit/Adafruit_MQTT_Library

doc/source/resources.rst

@@ -132,6 +132,7 @@
 .. _UUID4: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_.28random.29
 .. _ELF: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
 .. _STL: https://en.wikipedia.org/wiki/Standard_Template_Library
+.. _ArduinoSTL: https://github.com/mike-matera/ArduinoSTL
 
 .. SDKs
 .. _Arduino: https://www.arduino.cc/

libraries/TerkinData/README.rst

@@ -16,60 +16,73 @@ TerkinData C++
 
 .. tip::
 
-    You might want to `read this document on our documentation space <https://hiveeyes.org/docs/arduino/TerkinData/README.html>`_,
+    You might want to `read this document on our documentation space
+    <https://hiveeyes.org/docs/arduino/TerkinData/README.html>`_,
     all inline links will be working there.
 
 
 ************
 Introduction
 ************
+
 TerkinData C++ is a convenient library for handling sensor readings.
-It helps to decouple the sensor reading domain from the
-telemetry domain in a typical data logger application.
-While providing a generic interface, it can serialize
-measurement values to CSV, x-www-form-urlencoded and
-JSON formats.
 
-The library solves some important obstacles usually encountered when doing telemetry:
+It helps to decouple the sensor reading domain from the telemetry domain in a
+typical data logger application. While providing a generic interface, it can
+serialize measurement values to CSV, x-www-form-urlencoded and JSON formats.
+
+The library solves some important obstacles usually encountered when doing
+telemetry:
 
 - When serializing to CSV, the order of fields is important.
   Therefore, an ordered list of field names is required for proper operation.
-  When serializing a CSV data line and some sensor values are missing due to sensor
-  defects or other errors, this must not have an impact on the CSV data line.
-  Optionally, the CSV header line can be prefixed with an arbitrary string on serialization.
+  When serializing a CSV data line and some sensor values are missing due to
+  sensor defects or other errors, this must not have an impact on the CSV data
+  line. Optionally, the CSV header line can be prefixed with an arbitrary
+  string on serialization.
 - Some sensors like the DHT33_ (RHT04_) digital humidity/temperature sensor
   deliver two values in a single reading cycle: Humidity and temperature.
-  The data channeling from reading the sensor to telemetry submission has to account for that.
-  As with the DS18B20_ digital thermometer or other `1-Wire`_ devices,
-  multiple sensors of the same type might be attached, the data channeling also has to account for that.
+  The data channeling from reading the sensor to telemetry submission has to
+  account for that. As with the DS18B20_ digital thermometer or other `1-Wire`_
+  devices, multiple sensors of the same type might be attached, the data
+  channeling also has to account for that.
   This is achieved by one level of indirection between the sensor reading and
   the telemetry domain by introducing a convenient mapping between lowlevel
-  sensor values and the designated highlevel telemetry field names,
-  which is resolved and applied on serialization.
-- A timestamp string in arbitrary format can be attached to a single measurement.
+  sensor values and the designated highlevel telemetry field names, which is
+  resolved and applied on serialization.
+- A timestamp string in arbitrary format can be attached to a single
+  measurement.
 
-It relies on a reasonable STL_ implementation as it uses the ``map`` and ``vector``
-data containers and its corresponding iterators under the hood.
+The software is currently alpha quality as we didn't hunt down potential memory
+leaks yet. It has also not been run on embedded MCU hardware yet, as this is
+just a design draft by now. Helping hands on that are very welcome!
 
 
 ***********
 Environment
 ***********
-TerkinData C++ compiles with *avr-g++ 4.9.1* (AVR) and *g++ 5.3.0* (MacPorts)
-as well as *xtensa-lx106-elf-g++ 4.8.2* (xtensa-lx106, crosstool-NG 1.20.0) on Mac OS X.
-It currently does **not** compile with *clang++ 3.4*. Patches are welcome!
 
-The example programs `<csv_basic.cpp_>`_, `<urlencoded_basic.cpp_>`_ and `<json_basic.cpp_>`_ have been tested
-successfully at runtime on x86_64 (Mac OS X).
+TerkinData C++ has been compiled successfully for/with:
+
+- AVR ATmega328: avr-g++ 7.3.0
+- ARM: xPack GNU Arm Embedded GCC 9.3.1
+- ESP8266: xtensa-lx106-elf-gcc 4.8.2
+- ESP32: xtensa-esp32-elf-gcc 8.4.0
+- macOS: Apple clang version 12.0.0
 
-It is currently alpha quality as we didn't hunt down potential memory leaks yet.
-It has also not been run on embedded MCU hardware yet, as this is just a design draft by now.
-Helping hands on that are very welcome!
+On AVR, the framework relies on the STL_ implementation ArduinoSTL_ as it uses
+the ``map`` and ``vector`` data containers and its corresponding iterators
+under the hood.
+
+The example programs `<csv_basic.cpp_>`_, `<json_basic.cpp_>`_,
+`<urlencoded_basic.cpp_>`_ and have been tested successfully at runtime on
+x86_64 (macOS).
 
 
 ********
 Synopsis
 ********
+
 .. highlight:: cpp
 
 Setup
@@ -165,20 +178,36 @@ JSON
     delete measurement;
 
 
+
+*****
+Usage
+*****
+
+.. highlight:: bash
+
+::
+
+    # Get source code.
+    git clone --recursive https://github.com/hiveeyes/arduino
+    cd arduino/libraries/TerkinData/examples
+
+    # Build examples for all embedded platforms.
+    make
+
+    # Build and run examples on host platform.
+    make run
+
+
 ********
 Examples
 ********
-.. highlight:: bash
 
 .. _terkindata-csv-example:
 
 CSV
 ===
 ::
 
-    cd examples
-    make csv
-
     ======================
     TerkinData CSV example
     ======================
@@ -199,10 +228,8 @@ CSV
 
 x-www-form-urlencoded
 =====================
-::
 
-    cd examples
-    make urlencoded
+::
 
     ========================================
     TerkinData x-www-form-urlencoded example
@@ -221,10 +248,8 @@ x-www-form-urlencoded
 
 JSON
 ====
-::
 
-    cd examples
-    make json
+::
 
     =======================
     TerkinData JSON example
@@ -261,7 +286,7 @@ Dependencies
 
 ArduinoSTL
 ==========
-| ArduinoSTL 1.0.2 by Mike Matera
+| ArduinoSTL by Mike Matera
 | https://github.com/mike-matera/ArduinoSTL
 |
 
@@ -274,9 +299,10 @@ ArduinoJson
 |
 
 
-****
-Todo
-****
+*******
+Backlog
+*******
+
 .. todo::
 
     - Add :ref:`BERadio` serialization format

libraries/TerkinData/TerkinData.cpp

@@ -1,9 +1,9 @@
 /**
  *
- * TerkinData: Flexible data collection for decoupling sensor reading and telemetry domains
+ * TerkinData: Flexible data collection for decoupling sensor reading and
+ *             telemetry domains.
  *
- *
- *  Copyright (C) 2017  Andreas Motl <andreas.motl@elmyra.de>
+ *  Copyright (C) 2017-2021  Andreas Motl <andreas.motl@elmyra.de>
  *
  *  This program is free software; you can redistribute it and/or modify
  *  it under the terms of the GNU Lesser General Public License as published by

libraries/TerkinData/TerkinData.h

@@ -1,9 +1,9 @@
 /**
  *
- * TerkinData: Flexible data collection for decoupling sensor reading and telemetry domains
+ * TerkinData: Flexible data collection for decoupling sensor reading and
+ *             telemetry domains.
  *
- *
- *  Copyright (C) 2017  Andreas Motl <andreas.motl@elmyra.de>
+ *  Copyright (C) 2017-2021  Andreas Motl <andreas.motl@elmyra.de>
  *
  *  This program is free software; you can redistribute it and/or modify
  *  it under the terms of the GNU Lesser General Public License as published by
@@ -22,7 +22,7 @@
 
     // Standard C++ (STL) for Arduino for dynamic data structures like vector and map.
     // Required for AVR only, STL already seems to be included in Espressif SDK.
-    #ifndef ARDUINO_ARCH_ESP8266
+    #if ARDUINO_ARCH_AVR
         #include <ArduinoSTL.h>
     #endif
 
@@ -34,9 +34,11 @@
 #endif
 
 // Dynamic data containers
+#include <string>
 #include <vector>
 #include <map>
 
+
 namespace TerkinData {
 
     // For storing a list of field names
@@ -193,10 +195,26 @@ namespace TerkinUtil {
         #endif
     }
 
-    #if not defined(ARDUINO)
-
     // http://www.cplusplus.com/reference/ctime/strftime/
     static std::string now_iso() {
+        #if defined(ARDUINO)
+        return std::string("FIXME: ISO timestamp on AVR");
+        /*
+        //#include <time.h>
+
+        //TimeElements tm;
+        //tm tms;
+        //time_t t = makeTime(tm);
+        time_t t = mktime();
+        //#include "time.h"
+        time_t t = now();
+        char buff[] = "20180810T143000Z";
+        sprintf(buff, "%02d-%02d-%02dT%02d:%02d:%02d", year(t), month(t), day(t), hour(t), minute(t), second(t));
+        //return std::string(isotime().c_str());
+        //return std::string(t);
+        return std::string(buff);
+        */
+        #else
         time_t rawtime;
         struct tm * timeinfo;
         char buffer [80];
@@ -207,9 +225,9 @@ namespace TerkinUtil {
         strftime(buffer, 80, "%FT%TZ", timeinfo);
 
         return std::string(buffer);
+        #endif
     }
 
-    #endif
 
 
     // https://github.com/zenmanenergy/ESP8266-Arduino-Examples/blob/master/helloWorld_urlencoded/urlencode.ino

libraries/TerkinData/examples/Makefile

@@ -1,24 +1,32 @@
-all: csv urlencoded json
+$(eval venvpath  := $(PWD)/.venv)
+$(eval python    := $(venvpath)/bin/python)
+$(eval pip       := $(venvpath)/bin/pip)
+$(eval pio       := $(venvpath)/bin/pio)
 
-run:
-	@./csv_basic
-	@./json_basic
-	@./urlencoded_basic
+export PLATFORMIO_WORKSPACE_DIR := $(PWD)/.pio
 
-csv:
-	@./build csv_basic.cpp
-	@./csv_basic
+build: setup-virtualenv
+	cd json; $(pio) run --project-conf=../platformio.ini # --verbose
+	cd csv; $(pio) run --project-conf=../platformio.ini # --verbose
+	cd urlencoded; $(pio) run --project-conf=../platformio.ini # --verbose
 
-json:
-	@./build json_basic.cpp
-	@./json_basic
+upload: setup-virtualenv
+	$(pio) run --target upload --upload-port=${MCU_PORT}
 
-urlencoded:
-	@./build urlencoded_basic.cpp
-	@./urlencoded_basic
+run:
+	mkdir -p build
+	./compile csv/csv_basic.cpp
+	./compile json/json_basic.cpp
+	./compile urlencoded/urlencoded_basic.cpp
+	@echo
 
-clean:
-	rm csv_basic
-	rm json_basic
-	rm urlencoded_basic
+	./build/csv_basic
+	@echo
+	./build/json_basic
+	@echo
+	./build/urlencoded_basic
+	@echo
 
+setup-virtualenv:
+	@test -e $(python) || `command -v virtualenv` --python=python3 $(venvpath)
+	$(pip) install platformio

libraries/TerkinData/examples/build → libraries/TerkinData/examples/compile

@@ -1,10 +1,9 @@
 #!/bin/bash
-#
-# usage: ./build csv_basic.cpp
-# The executable will be saved in the current directory
+# Build example programs against glibc.
+# Usage: ./compile csv_basic.cpp
 
 INPUT=$1
-OUTPUT=$(basename $INPUT ".cpp")
+OUTPUT=build/$(basename $INPUT ".cpp")
 
 LIB_DIR="../.."
 TERRINE="${LIB_DIR}/Terrine"
@@ -13,18 +12,15 @@ ARDUINOJSON="${LIB_DIR}/ArduinoJson"
 INCLUDE="-I$TERRINE -I$TERKINDATA -I$ARDUINOJSON"
 
 #CPP=g++
-#CPP=clang++
-#CPP=/opt/local/bin/g++-mp-4.9
-CPP=/opt/local/bin/g++-mp-5
-#CPP=/opt/local/bin/gcc-mp-5
-#CPP=/opt/local/bin/clang++-mp-3.4
+CPP=clang
+#CPP=/opt/local/bin/g++-mp-5
 
 # ./sim/compat.h
-CPP_FLAGS="-x c++ -lstdc++ -std=gnu++11 -Os -fno-exceptions -ffunction-sections -fdata-sections -pedantic -fpermissive"
+CPP_FLAGS="-x c++ -lstdc++ -std=gnu++11 -Os -fno-exceptions -ffunction-sections -fdata-sections -fpermissive"
 CPP_FLAGS_MORE="-Wall -Wextra -Wno-builtin-macro-redefined -Wno-unused-function -fdiagnostics-color -DHAVE_ARDUINO_JSON"
+# -pedantic
 
 $CPP -g -I. $INCLUDE $CPP_FLAGS $CPP_FLAGS_MORE \
     $INPUT \
     $TERRINE/Terrine.cpp $TERKINDATA/TerkinData.cpp \
     -o $OUTPUT
-