EmonESP Changes

Objective: include a class in the EmonESP software to read its variables

EmonESP

ESP8266 WIFI serial to emoncms link

For applications that only require basic posting of data from one emonTx to a remote server such as Emoncms.org an emonTx with this WiFi module provides a lower cost route than an emonBase or emonPi base-station installation.

The core of EmonESP is also used for emonPixel and OpenEVSE ESP WiFi 2.0.

Requirements

ESP-12E module with 4M Flash

EmonESP User Guide

Hardware Setup

Purchase a pre-loaded ESP8266
To connect an ESP to emonTx see This User Guide Section

First Setup

On first boot, ESP should broadcast a WiFI AP emonESP_XXX. Connect to this AP and the captive portal should forward you to the log-in page. If this does not happen navigate to http://192.168.4.1

Note: You may need to disable mobile data if connecting via a Android 6 device

1. WiFi Connection

Select your WiFi network from list of available networks
Enter WiFi PSK key then click Connect

emonESP should now connect to local wifi network and return local IP address.
Browse to local IP address by clicking the hyperlink (assuming your computer is on the same WiFi network) On future boots EmonESP will automatically connect to this network.

Note: on some networks it's possible to browse to the EmonESP using hostname http://emonesp or on windows http://emonesp.local

If re-connection fails (e.g. network cannot be found) the EmonESP will automatically revert back to WiFi AP mode after a short while to allow a new network to be re-configued if required. Re-connection to existing network will be attempted every 5min.

Holding the boot button at startup (for about 10's) will force AP mode. This is useful when trying to connect the unit to a new WiFi network.

2. Emoncms

EmonESP can post data to emoncms.org or any other Emoncms server (e.g. emonPi) using Emoncms API.

Data ca be posted using HTTP or HTTPS. For HTTPS the Emoncms server must support HTTPS (emoncms.org does, emonPi does not).Due to the limited resources on the ESP the SSL SSH-1 fingerprint for the Emoncms server must be manually entered and regularly updated.

Note: the emoncms.org fingerprint will change every 90 days when the SSL certificate is renewed.

3. MQTT

EmonESP can post data to an MQTT server. Each data key:pair value will be published to a sub-topic of base topic.E.g data CT1:346 will results in 346 being published to <base-topic>/CT1

Enter MQTT server host and base-topic
(Optional) Enter server authentication details if required
Click connect
After a few seconds Connected: No should change to Connected: Yes if connection is successful. Re-connection will be attempted every 10s.

Note: emon/xxxx should be used as the base-topic if posting to emonPi MQTT server if you want the data to appear in emonPi Emoncms. See emonPi MQTT docs.

5. Admin (Authentication)

HTTP Authentication (highly recomended) can be enabled by saving admin config by default username and password

HTTP authentication is required for all HTTP requests including input API

7. System

Displays free system memory and firmware version

OTA Firmware Update

TBC

HTTP API Examples

View units status:

http://<IP-ADDRESS>/status

Example return in JSON:

{"mode":"STA","networks":[],"rssi":[],"ssid":"OpenEnergyMonitor","srssi":"-58","ipaddress":"10.0.1.93","emoncms_server":"emoncms.org","emoncms_node":"emonesp","emoncms_apikey":"xxxxxxxx","emoncms_connected":"0","packets_sent":"0","packets_success":"0","mqtt_server":"emonpi","mqtt_topic":"emonesp","mqtt_user":"emonpi","mqtt_pass":"xxxxxx","mqtt_connected":"0","free_heap":"25040"}

4. Data Input

Data can be inputed to EmonESP via serial UART or HTTP API.

UART Input

Data in serial:pairs string format can be inputed to EmonESP via serial UART (115200 baud) e.g:

ct1:3935,ct2:325,t1:12.5,t2:16.9,t3:11.2,t4:34.7

HTTP API

Data in string:pairs can be sent to EmonESP via HTTP API. This is useful to emulate the serial string data function while using the UART for code upload and debug. API example:

http://<IP-ADDRESS>/input?string=ct1:3935,ct2:325,t1:12.5,t2:16.9,t3:11.2,t4:34.7

Save Emoncms server details

http://<IP-ADDRESS>/saveemoncms?&server=emoncms.org&apikey=xxxxxxxxxxxxxxxxxx&node=emonesp&fingerprint=7D:82:15:BE:D7:BC:72:58:87:7D:8E:40:D4:80:BA:1A:9F:8B:8D:DA

SSL SHA-1 fingerprint is optional, HTTPS connection will be enabled if present

Save Emoncms MQTT server details

http://<IP-ADDRESS>/savemqtt?&server=emonpi&topic=emonesp&user=emonpi&pass=emonpimqtt2016

MQTT user and pass are optional, leave blank for connection to un-authenticated MQTT servers

Installation

EmonESP uses ESP8266 Arduino core

Firmware can be compiled and uploaded either using PlatfomIO (see blog post) or Arduino IDE.

Option 1: Using PlatformIO

For more detailed ESP8266 Arduino core specific PlatfomIO notes see: https://github.com/esp8266/Arduino#using-platformio

1a. Install PlatformIO command line

The easiest way if running Linux is to install use the install script, this installed pio via python pip and installs pip if not present. See PlatformIO installation docs. Or PlatformIO IDE can be used :

$ sudo python -c "$(curl -fsSL https://raw.githubusercontent.com/platformio/platformio/master/scripts/get-platformio.py)"

1b. And / Or use PlatformIO IDE

Standalone built on GitHub Atom IDE, or use PlatformIO Atom IDE plug-in if you already have Atom installed. The IDE is nice, easy and self-explanitory.

Download PlatfomIO IDE

2. Clone this repo

$ git clone https://github.com/openenergymonitor/EmonESP

3. Compile

$ cd EmonESP
$ pio run

The first time platformIO is ran the espressif arduino tool chain and all the required libs will be installed if required.

3. Upload

Put ESP into bootloader mode
- On Heatpump monitor use jumper to pull GPIO0 low then reset then connect power (simulates reset) or pull RST pin low.
- On other ESP boards (Adafruit HUZZAH) press and hold GPIO0 button then press Reset, LED should light dimly to indicate bootloader mode

a.) Upload main program:

$ pio run -t upload

b.) Upload data folder to the file system (html, CSS etc.) (SPIFFS):

Put ESP back into bootloder mode, see above

$ pio run -t uploadfs

See PlatfomrIO docs regarding SPIFFS uploading

Or upload all in one go

This wil upload both the fimware and fs in a single command

Put ESP into bootloader mode

esptool.py write_flash 0x000000 .pioenvs/emonesp/firmware.bin 0x300000 .pioenvs/emonesp/spiffs.bin

c.) OTA upload over local network

$ pio run -t upload --upload-port <LOCAL-ESP-IP-ADDRESS>

Upload SPIFFS filesystem over OTA (and don't flash):

pio run -e emonesp_spiffs -t upload --upload-port <LOCAL-ESP-IP-ADDRESS>

OTA uses port 8266. See PlatformIO ESP OTA docs:

Troubleshooting Upload

Erase Flash

If you are experiancing ESP hanging in a reboot loop after upload it may be that the ESP flash has remnants of previous code (which may have the used the ESP memory in a different way). The ESP flash can be fully erased using esptool. With the unit in bootloder mode run:

$ esptool.py erase_flash

sudo maybe be required

Output:

esptool.py v1.2-dev
Connecting...
Running Cesanta flasher stub...
Erasing flash (this may take a while)...
Erase took 8.0 seconds

Fully erase ESP-12E

To fully erase all memory locations on an ESP-12 (4Mb) we neeed to upload a blank file to each memory location

esptool.py write_flash 0x000000 blank_1MB.bin 0x100000 blank_1MB.bin 0x200000 blank_1MB.bin 0x300000 blank_1MB.bin

4. Debugging ESP subsystems

The ESP subsystems have a lot of logging that can be enabled via setting various build options.

Using Platform IO the easiest way to configure these is via the PLATFORMIO_BUILD_FLAGS environment variable.

First select the serial port to output debug;

-DDEBUG_ESP_PORT=Serial
-DDEBUG_ESP_PORT=Serial1

Then add one or more of the debug options;

-DDEBUG_ESP_CORE
-DDEBUG_ESP_WIFI
-DDEBUG_ESP_HTTP_CLIENT
-DDEBUG_ESP_HTTP_SERVER
-DDEBUG_ESP_HTTP_UPDATE
-DDEBUG_ESP_UPDATER
-DDEBUG_ESP_OTA
-DDEBUG_ESP_SSL
-DDEBUG_TLS_MEM

For example from the Windows Power shell you may do something like;

$env:PLATFORMIO_BUILD_FLAGS="-DDEBUG_ESP_PORT=Serial1 -DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI"
pio run -t clean
pio run
pio run -t upload --upload-port 172.16.0.80

Option 2: Using Arduino IDE

1. Install ESP for Arduino IDE with Boards Manager

Install steps from: https://github.com/esp8266/Arduino

Install Arduino IDE 1.6.8 from the Arduino website.
Start Arduino and open Preferences window.
At the bottom of this window find the field “Additional Board Manager URLs” and enter ‘http://arduino.esp8266.com/stable/package_esp8266com_index.json`. You can add multiple URLs, separating them with commas.
Open Tools > Board > Board Manager, scroll down and click on esp8266 platform, select version then install
Select Tools > Board > Generic ESP8266 Module (required for EmonESP)

2. Install ESP filesystem file uploader

Required to include data folder with HTML etc in the upload

Follow esp8266 filesystem instructions (copied below):

Download the Arduino IDE plug-in (.zip)
Navigate to the tools folder in your Arduino sketchbook directory, (create directory if it doesn't exist)
Create tools > ESP8266FS folder
Unpack the plug-in into ESP8266FS directory (the path will look like <home_dir>/Arduino/tools/ESP8266FS/tool/esp8266fs.jar)
Restart Arduino IDE

3. Clone this repo