eGeoffrey
Developing my weather station, Work in Progress #140
Replies: 47 comments
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-07-17 12:04:53] I think that I have enough insight in the way eGeoffrey is working to proceed with my own weather station. My intention is to inform you about the progress of the development every now and then. At the hardware side I have had some difficulties. They were in the area of creating the PCB for the weather station. This PCB is required to combine the different sensors with the ESP8266 module, a Wemos D1 mini. Because this is my first attempt to integrate a bunch of different sensors with different interfaces I made mistakes in the design of the PCB. But I am getting along. Because there are more ways to transmit the data from a sensor with the MQTT protocol to the broker a decision had to be taken which way to go. Next to that I have been thinking about how to allocate topics to the sensors. Because the weather station is only the first step in developing my house automation system I developed a scheme for it. I would like to share my thoughts about this: Sensor network Transmission network In essence there are two different ways to exchange data between sensors and eGeoffrey using MQTT via the broker, Mosquitto:
The mesh network is self establishing. The only user data required to build the network are data related to the WiFi network and the broker the mesh should connect to. Adding and removing nodes from the network is taken care of by the network without user intervention. As long as a module can establish a single link with one of the other modules, it can exchange data with Mosquitto. Even at houses with larger plots it is possible to create a sensor network without adding extra hardware to the WiFi network. The sensor modules themselves create the radio network. The module with the best WiFi signal creates the single WiFi connection of the network. Sensor control Each sensor is controlled by an ESP8266 module. Each module can handle several sensors. Topic allocation Each module is on a plot. This plot has a name. This name is the root topic. Examples A temperature sensor in one of the two weather stations in the garden of our house would report its data under topic “/ourhouse/garden/weatherstation/2/bme280/temp”. A humidity sensor in the living of our holiday home would report its data under topic “/holidayhome/living/general/bme/hum”. The topic for the data of the radar sensor of the same module would be “/holidayhome/living/general/radar”. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-07-31 14:16:34] I am integrating eGeoffrey with my weather station. Some early "lessons learned" are already there. To investigate what happens with data being processed, it is helpful to filter out certain data. Heartbeat data are the first to be filtered out. The topic structure used does not allow this. The topic structure needs amendment The old topic was: "ourhouse/garden/weatherstation/heartbeat The new topic is "ourhouse/haertbeat/weatherstation" Now the heartbeats of all modules can be filtered out easily. Furthermore it is useful to be able to adapt the interval between transmission of sensor data. In the software of each module each sensor has a transmission interval for normal operation and one for testing. A variable "testFlag" controls which of both intervals is used. Topic /onshuis/test/"module-ID" By filtering topics specific data can be examined. Switching data transmission of a sensor on or off is not useful. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-01 10:54:12] I have added an extra feature. During normal operations the data rate is rather slow for most sensors. For testing purposes it might be handy to have sensor data transmitted at a higher pace. This is implemented by a "control an on/off switch" widget. The sensor of this widget sends a "1" or "0" in the topic "/mesh-in/module-ID/ourhouse/test/weatherstation". The software of the weather-station receives this message and instead of the operations interval the test interval is used for data transmission. The weather station sends the received message return in the topic "/mesh-out/module-ID/ourhouse/test/weatherstation. Egeoffrey has a sensor which receives data in the topic "/mesh-out/+/ourhouse/test/weatherstation". |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-01 11:06:10] I have a question about data rates of sensors. Where you operate your own weather-station also you might be able to help me. Which intervals do you apply for temperature, windspeed, wind direction, light and uv? |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-02 08:24:32] Gaining experience with the weather station having multiple sensors measuring the same quantity an other idea popped-up. Adjusting sensor data level Having multiple sensors reading the same quantity it is clear that those sensors deliver data that do not match each other. To be able to compare the results of the sensors measuring the same quantity it is necessary to adjust the readings of each sensor within certain limits. The result is that the curves of the presented data are moved higher or lower on the screen. They can be put over each other. Each sensor needs to be adjustable. Each change in value is published. To confirm the settings an acknowledgement has to be given. For each sensor we need a widget that displays the adjustment applied to the reading. Per reading an eGeoffrey sensor is needed receiving the adjustment values of the sensors. Data display To gain a better overview of the data presented some grouping of data is needed. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-08-03 10:38:39]
Personally I do not go below the 10 minutes interval just because there are few use cases you need so many data points unless you have critical actions to be executed based on those values and their accuracy. I have also a bunch of zigbee-based xiaomi devices for which the interval cannot be configured and they push new measures every now and then but I'm still fine with it. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-04 07:30:19] @user2684 At the moment under "allowed values" e.g. ON,OFF" is possible. It would be helpful if a range for numbers could be inserted here. In my case it would prevent inputting wrong numbers for the selection of a sensor controlled by one module. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-04 08:48:57] I do have a serious problem with ESP628MQTTMESH. Outgoing everything is OK but incoming the messages are arriving but the callback routine is not triggered anymore. I put an issue in Github, but nowadays reactions take a rather long time. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-08-05 11:00:56]
This is not related to eGeoffrey right? Just to understand if I can do something or not. Thx |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-05 11:06:41] @user2684 |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-06 14:11:52] I fixed the mesh problem. I am now working on controlling the sensors via MQTT by means of eGeoffrey. I remark about sensor configuration: When one later on changes the "actuator" to "push" the data on the "configuration" tab are deleted. Would be handy if these data would be retained and not deleted. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-07 17:06:23] The tsl2561 sensor turns out to be overloaded very fast. That means the reading is 6555 and that is it. I ordered a different type the VEML 7700 together with a I2C type UV index sensor the SI1145. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-08 09:12:06] Surveying the CPU temperature is a good idea.
The casing of my choice is:
I purchased it from "reichelt.de" |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-08 11:25:08] Using MQTT in two directions, from the sensor module to eGeoffrey and the other way around, turned out to be very simple. During development and testing it is handy to control several aspects of each individual sensor.
To send all these parameters from within eGeoffrey a sensor control page is made. The sensor module receives the MQTT messages which are transmitted by eGeoffrey. As soon as the “ack” is received the data are processed and stored into EEPROM to be used from now on. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-10 07:16:43] The idea explained above is working. eGeoffrey sends data to the sensor module and the module sends the requested data in return. That means that with the sensor control page from 12 sensors five aspects can be controlled with eGeoffrey. To give you an impression of how that is achieved a little bit more detail is below. eGeoffrey sending data Topic structure onshuis/sensor sensors publishing with topic: mesh-in/XXXX/onshuis/status mesh-in/XXXX/onshuis/opsinterval mesh-in/XXXX/onshuis/testinterval mesh-in/XXXX/onshuis/adjust mesh-in/XXXX/onshuis/ack widgets: onshuis/status onshuis/opsinterval onshuis/testinterval onshuis/opstest onshuis/adjust onshuis/ack To be sure that the data are received correctly the sensor module sends the data back to eGeoffrey. eGeoffrey receiving data sensors receiving with topic: mesh-out/+/onshuis/status mesh-out/+/onshuis/opsinterval mesh-out/+/onshuis/testinterval mesh-out/+/onshuis/opstest mesh-out/+/onshuis/adjust widgets: onshuis/status onshuis/opsinterval onshuis/testinterval onshuis/opstest onshuis/adjust Also it is possible to interrogate a module and retrieve the settings from a sensor. eGeoffrey retrieving sensor settings Topic structure: onshuis/retrieve sensors publishing with topic: mesh-in/XXXX/onshuis/retrieve mesh-in/XXXX/onshuis/ack widgets: onshuis/ack eGeoffrey resetting sensor module After finishing a test series it is handy to reset the module to the default values with only two actions. Topic structure onshuis/default sensors publishing with topic: mesh-in/XXXX/onshuis/default mesh-in/XXXX/onshuis/ack widgets: onshuis/ack |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-27 16:38:31] The test regarding controlling the individual sensors is concluded successfully. Controlling the complete module still has some issues. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-28 14:44:34] Issues solved. Put the weather-station on line. Some sensors have to be replaced but for the time being let it run. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-08-29 16:46:48] To give an idea of how the screens are looking now: Controlling the weather-station
Display of local weather data
|
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-08-30 15:16:56] Uau! very comprehensive page! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-12-05 09:43:13] Having gained experience now some remarks can be made. As far as the UV sensors are concerned one has to be aware that acrylic covers and normal glass block UV light rather effectively. The only glass which does not do that would be quartz glass. The tipping bucket has still an issue. The summing over time by eGeoffrey does not work for this application. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2020-12-07 14:54:00] @eporocrail said in Developing my weather station, Work in Progress:
Had something similar happening to me with always a 0 from those sensors since they have to be kind of fully exposed to the light of the sun without nothing interfering...
Still tracked with egeoffrey/egeoffrey-controller#23. Could take a while to implement it though. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-02-10 08:18:04] Egeoffrey is not able to deal with data from a "tipping bucket" properly at the moment. The accumulation of the data of the bucket has to be done by the weather station. To create a real link between the gathered rain information and the time the rain was falling, the weather station needs a more or less accurate clock. It is possible to run a clock on the weather station which provides time information that is accurate enough. The only thing required from the outside world is a sync signal. My modules communicate via MQTT. The most flexible way to implement a time sync signal is distributing time via MQTT. One single broker is used at my place. One time sync module is implemented. It is not incorporated into the mesh. This module synchs its own clock via the NTP protocol with an internet time-server. It sends time to Mosquitto in a readable format with a ten seconds interval. (year, month, day, hour, minute, second) Mosquitto distributes the time message at the same pace. Time is sent with topic: "in/decoder ID/time". Any other module using a clock does sync time during booting. After this first sync, time can be updated as required by the application. The time decoder is displaying the time using an TFT screen. This option is implemented. Extra functionality. The first one is an alarm clock. The function of the earlier developed data display and system control module can be taken over. The TFT screen used now is larger than the one from the data display module. Re-design. Re-designed are the display screens, the data exchange and the control. GUI Two pages are foreseen. The first one for time and weather information and the second one for alarm system information. Default the time page is displayed. When an alarm occurs the alarm page is displayed on reception of the alarm data. Next to this automatic pop-up the alarm page can be activated with the remote control. Data Data are broadcasted by eGeoffrey. Data are broadcasted after processing newly received data. A topic structure needs to be designed. Data are received and stored by the module. Depending on the active page data are displayed and refreshed at regular intervals. Control Commands are given with a TV IR-remote control. The commands are received by the module directly without data exchange via MQTT. Control is as easy as possible. Mainly the number pad is used. A command starts with the "red" key and ends with the "yellow" key. The command given is executed with the "green" key. Remote control functions: |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2021-02-13 10:17:07] Ok, I see this tipping bucket issue to be something I'd rather need to re-prioritize high despite you have found out a brillant solution IMHO. Try giving me some hit regarding the expected behaviour on egeoffrey/egeoffrey-controller#23 and I'll see if I can work on it ASAP. I expect this to be something like measures get in as usual and at any time, hourly aggregates are the sum of those values per hour, daily aggregates are the sum of the hourly measures. Is it the case? Regarding instead your approach, why not using a RTC module and every here there your station ask for a time sync to re-align the RTC if needed? Thanks |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-02-13 11:17:47] The software clock of the ESP8266 is rather stable (deviation seems to be a second in several hours). For that reason I do not need an RTC module. In the mesh I can only deal with MQTT messages. UDP is not supported. Therfore I have to send a sync signal via MQTT. Once this is implemented it is a general feature for my system. Your perception of the tipping bucket problem is OK. I will put a remark in the issue. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-02-15 13:54:40] The integration of the IR remote is done. A piezo buzzer is used to produce the alarm clock sound. Next challenge is the data exchange. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2021-02-15 14:51:44] Great piece of work as usual |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-02-15 15:12:03] I will do the test when I finished my solution. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-02-19 14:11:50] The software modules are ready. The next step is putting the hardware together. Then the software integration can be finished. Have fun. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2021-03-11 12:05:38] Quite a storm hit our country. That is the end of the project. Have fun. |
Beta Was this translation helpful? Give feedback.
-
|
[This post has been migrated from the old forum, it was originally sent by user2684 on 2021-03-15 16:06:28] @eporocrail said in Developing my weather station, Work in Progress:
Really sorry to hear that Thanks again for the contribution |
Beta Was this translation helpful? Give feedback.
-
[This post has been migrated from the old forum, it was originally sent by eporocrail on 2020-06-28 18:36:52]
Developing my weather-station
The aim of this project is to create a weather-station from scratch.
Several sensors will be integrated. One of them is of an experimental nature. An other one is self-made. The data of some will be compared to see the differences between the different types of sensor measuring the same quantity.
The mechanical part is not described here.
Sensor integration, data transmission, data processing and data presentation are addressed.
Sensors
The following sensors are applied:
BME280 : pressure, humidity and temperature
TSL2561 : light level
ML8511 : UV level
DS18B20 : temperature
HMC5883: compass for the wind direction
YL38 : rain start detection
Tipping bucket rain volume sensor.
Windsensor Rev. P : a hot wire anemometer with temperature sensor
They will be combined in one single mechanical construction.
They are hooked-up to one LOLIN D1 mini. This module contains a microcontroller able to transmit via incorporated WiFi capabilities. Several GPIO and communication protocols are available.
The following sensors use the I2C protocol and need 3.3V power:
BME280, TSL2561, HMC5883
The following sensors provide an analog output and need 3.3V power:
ML8511, YL38
The following sensor uses the "one wire" protocol and needs 3.3V power:
DS18B20
The Windsensor Rev. P has two analog outputs, one for the temperature and one for the windspeed.
It requires 12V power.
The tipping bucket works with two simple switching signals of 3.3V.
All sensors have 3.3V outputs. One requires 12V powers supply, the rest need 3.3V power supply.
Datacollection
Per sensor quantity it is adjustable which interval between measurements is applied.
It might happen that values change suddenly. To be able to catch those rapidly changing values also, per quantity an adjustable difference between sensed values is applied to collect and transmit data. This results in a regular update of data per sensor. On a sudden change in value each sensor transmits extra data.
Datatransmission
For data-transmission the MQTT protocol is used. Sensor values are published in each sensor's unique topic.
Data processing and presentation
eGeoffrey is used for data processing and presentation.
Beta Was this translation helpful? Give feedback.
All reactions