-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added FDRS Timekeeping #142
Conversation
I love that you did it all without the time.h library. That definitely is better than I could have made it. My (literally) only concern presently is the DNS config. Is there any way to do that automatically? We'll at least have to make the definition a string probably, to be beginner-friendly and match MQTT config. |
Yeah, I wanted to write the code with as little dependencies as possible to simplify the compile process and size of the code
We can default the DNS server to a public DNS server like Google or some other DNS server if you'd like. That way we minimize the chance that people would have issues but still let them set the DNS server if they'd like. If it's the format of the IP address with the commas I'll have to go back and look at how I can change it into a string. It's most likely possible if that is desired. Kinda related to this, I'm thinking about adding optional Static IP address configuration as an advanced option to the default DHCP configuration.
Sounds good! |
Fixed in commit cce5fe5 |
In my commit dcda381 in the gateway_time.h and node_time.h files there is a couple of loadFDRS() calls with the current time and slew rate. I've disabled those by default as I think casual users would not care about that information. But just FYI if you want to do something with those like only enable when DEBUG is enabled and disable otherwise? My thoughts on next feature adds in this topic:
I'm not sure how soon I'll get to those features. I have a couple of others I want to work with such as enable OTA updates for WiFi enabled gateways and perhaps BLE UART communication. I'll try to add the ESP-NOW part in the next few days and then the others may be a week or two or so. |
In a27d34c, I removed the DNS server because the WiFi.useDNS() function does not work the way I expected it. Also, we'll just use the DNS server that is provided by the DHCP server. We will just make the assumption that the DHCP server has a valid DNS server entered ;) Once the branch with OTA and static IP are merged with this one users will have the option to define their own DNS server along with static IP address. |
@timmbogner, question on events/actions. Based on the repeater examples it seems that the convention is that ESPNOW1_ACT and LORA1_ACT are directed towards the nodes and, oppositely, ESPNOW2_ACT and LORA2_ACT are directed towards the front end? I'm thinking in context of periodically sending the time from the front-end to the nodes that the actions to be taken should be under ESPNOW1_ACT and LORA1_ACT but the documentation doesn't really specify any directionality but the examples seem to imply what I stated above. I think up to now we really haven't had much or any traffic in the front-end to node direction so it hasn't been important to clarify that. So I'm thinking that I'm going to use ESPNOW1_ACT and LORA1_ACT as events/actions to determine where to direct the time data. Let me know your thoughts please. Thanks! The other challenge is that those events are really meant to handle DataPackets and not SystemCommands so there will be that challenge to distinguish between what type of information is being sent. |
Great question. It's actually the big question for me in terms of timekeeping. So I reckon there are two paths to think down here...
Let me know what you think! |
I totally agree with you that we should keep it as flexible as possible for any future expansion and your item 2 above is the way to go. I think for now I'll set up time updates for ESP-NOW to go to Neighbor 2 (acting as repeater) and peers (acting as gateway). For LoRA time updates, I will send to Neighbor 2 (acting as repeater) and broadcast (acting as gateway). If, in the future, you tweak how things work it can be easily changed as I will try to neatly put the code in functions. On a slightly different topic, as the code matures I think we may need to revisit function names to match what LoRa is doing to what ESP-NOW is doing as closely as possible for clarity. We have functions for initialization, functions for looping, lower level functions that handle the nitty gritty data encapsulation details and higher level functions that call the lower level functions. as more functionality is added it might be good to standardize on the names. Something that came to my mind while reviewing the code and something to ponder for the future. |
I think it should send to all neighbors, just in case. We may need to better define a condition where a neighbor is not present in configs, which would actually be good. |
I'll definitely ponder this. I noticed when making functions for the recent reshuffle that I had begun |
I'm just afraid that if we send data back in the other direction, towards the front end, that will cause a loop. In general, all of the data should be sent in one direction, but there is no directionality in the code, because up to now there was no reason to send information from the front end down to the node. |
I'll admit, that possibility is why I'm not 100% sure of myself. Preventing the time data from being returned to its source is my means of preventing this. So if the time was received from ESPNOW1, it would never be sent back to that device, only ESPNOW2 and peers. |
I should say though, you're welcome to keep it how you think is best for the time-being. |
Going through the code to implement timekeeping via ESP-NOW seems like there is quite a bit to be modified. I wonder if this would be better as a different PR. The issue is that this PR was started before the last major commit to main branch so there are things that were committed to main that are conflicting with this code. So I think the next related commit to this might be better committed against main and wait till this PR is also completed. |
Dangit... holdupasec |
…17/Farm-Data-Relay-System into pr/142" This reverts commit 8238ebd, reversing changes made to c5f3796.
Soo... I thought that first commit had resolved the conflicts, then it added the other when I pushed it. My damage control didn't help. I dunno 😣
|
No problem. I think I can reject the pr and push again against the new main
branch. I'll look at it probably tomorrow.
…On Mon, Mar 13, 2023, 7:28 PM Timm Bogner ***@***.***> wrote:
Soo... I thought that first commit had resolved the conflicts, then it
added the other when I pushed it. My damage control didn't help. I dunno
😣
Here is a branch
<https://github.com/timmbogner/Farm-Data-Relay-System/tree/timekeeping-fixed>
without the extra stuff that I think should be okay? Sorry... dunno what to
do at this point. I honestly didn't think I was pushing it directly to your
fork, I sorta thought there was an intermediate branch for this pull
request.
—
Reply to this email directly, view it on GitHub
<#142 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOTXENIOVEVOD775GGCU6WLW363SPANCNFSM6AAAAAAVI5QZ3Q>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
I think there was a hook in LoRa that I forgot to implement. I think I have it almost ready to go but haven't submitted it yet. I'm trying to think whether it was in the gateway code, I believe. So gateways wouldn't be able to send to other gateways without the hook. I'll try to get that submitted tonight. Sorry about that. |
No worries! I have been working to get my T-Watch going on the system. One thing I've changed for this is to have the gateway send the time to a controller immediately after registering. I'll push that commit later if it's alright with you, as well as adding the following sketch to the controller examples. It works, but in UTC. Can you point me in the right direction? Here is the RTC library being used, I think // FARM DATA RELAY SYSTEM
//
// LilyGo T-Watch 2020 Example
//
// Developed by Timm Bogner (timmbogner@gmail.com) in Urbana, Illinois, USA.
//
#define LILYGO_WATCH_2020_V1
#include <LilyGoWatch.h>
#include "fdrs_node_config.h"
#include <fdrs_node.h>
TTGOClass *ttgo;
TFT_eSPI *tft;
PCF8563_Class *rtc;
uint32_t interval = 0;
void fdrs_recv_cb(DataReading theData) {
DBG("ID: " + String(theData.id));
DBG("Type: " + String(theData.t));
DBG("Data: " + String(theData.d));
}
void setup() {
Serial.begin(115200);
//Get watch instance
ttgo = TTGOClass::getWatch();
// Initialize the hardware
ttgo->begin();
// Turn on the backlight
ttgo->openBL();
// Receive as a local variable for easy writing
rtc = ttgo->rtc;
tft = ttgo->tft;
beginFDRS();
pingFDRS(1000);
addFDRS(fdrs_recv_cb);
subscribeFDRS(READING_ID);
delay(100);
rtc->syncToRtc();
}
void loop() {
loopFDRS();
if (millis() - interval > 1000) {
interval = millis();
tft->setTextColor(TFT_BLUE, TFT_BLACK);
tft->drawString(rtc->formatDateTime(PCF_TIMEFORMAT_MM_DD_YYYY), 50, 200, 4);
tft->setTextColor(TFT_GREEN, TFT_BLACK);
tft->drawString(rtc->formatDateTime(PCF_TIMEFORMAT_HMS), 5, 75, 7);
}
} |
Sounds good!
syncToRTC() in your example will take the system time and set it in the watch RTC. I would wrap that call with an
syncToSystem() will read the time from the RTC and set it in the running program
|
Sorry, I should have been more specific with my issue. Everything in that sketch works, except the display shows the time in UTC, which is what syncToRtc() sets it as. Is there an easy way to set it to the date/time adjusted for timezone and DST? |
The way I've coded the time stuff in FDRS is to store the time in UTC and then convert to local time based on the configured offset and whether the current time is in DST or STD time. The NTP information is in UTC and I think that's the most flexible way to store the time. So if you're seeing the time in UTC then it sounds like it's not being converted to your local time. The printTime() function is what takes the UTC time and converts the output to local time to be displayed. So I guess the question is whether that library expects local time to be stored in RTC or if it expects UTC. How does the library handle changeover from DST to STD time and vice versa? It's almost like it expects the RTC time to be in the local time zone. So in order to handle that you'd have to get the system time and then offset it by your time zone and then store it to the watch RTC. I guess what I would probably do is just replace the formatDateTime lines by your own function calls. So the class calls:
and
so I would substitute those calls with the strftime function call similar to the printTime() function.
If there is a delay you can use two char arrays and generate both strings before displaying them. I think something like that should work. |
I think the RTC does expect it to be local time. I guess we ought to find out what the more "standard" way of operating is. It would make sense that a device such as a clock or watch would set to local time internally. Would a datalogger be similar or would it use UTC? It seems like since you did all the work to make a nice DST routine, we should make the result a little more accessible to users. I have two ideas, let me know if either one seems good. One is to simply make the final offset (after DST adjustment) accessible through a function. The other is to just have two matching functions that return tm structs of either local or UTC. What do you think? I may do option 2 either way. And we can also do neither or both. |
I started out using local time as the RTC time but I think it's more universal to use the UTC time and just convert to whatever standard you want to use. I could see the datalogger data going either way. It really depends upon how "global" you want the data to be. After starting with local time I switched to UTC time because that is how it is utilized in NTP and I think the DST/STD conversion is way easier because you really do not have to fool with going backwards in time in the local time. I have no idea what the Arduino standard is but in the building controls and industrial controls world I think UTC is the standard. Also, pilots use UTC as the standard but they are traversing between different time zones all the time. I think an argument can be made either way but I think storing as UTC is better and calculating the offset to print the local time is the way to go.
I guess I would prefer a function that outputs local time in whatever manner is desired given my preference to have the underlying time in UTC. |
We could put a pin in the idea of functions that return tm structs for later. I'm not sure yet if that will be very complicated or useful. In unrelated news that I find sort of funny: |
OOPS, SORRY! I just noticed Yeah I totally misread your example. My bad. |
…Data-Relay-System into dev_timekeeping
stylin'
It was being called twice when ESP-NOW and LoRa were both enabled
I think ESP-NOW is pretty much solid. I can't get devices to receive a LoRa time, though. I've got Gateway 0x01 connected to MQTT and LoRa, then 0x03 is straight LoRa. 0x01 says it is sending, but I can't get the repeater 0x03 or a node to receive it. DataReadings and pings still make it okay. Any ideas? I'm going to start working through it with DBGs to find where things are going wrong. |
Sorry, I've been neglecting this as I've been busy with other things. Also been troubleshooting issues and have limited time. I think LoRa is working on my code but I'm troubleshooting multiple instances of time being sent. I'll submit what I have and then troubleshoot later. |
As far as LoRa goes the commit from a few days ago should work. Are you using the latest commits in this branch? I'll send my output if I have time later on tonight but I'm not sure that I'll have time. |
Here is my debug output from: ESPNOW -> LoRa gateway - address 0x22 (time received from 0x21):
LoRa -> LoRa Gateway - address 0x23 (time received from 0x22):
|
Well please don't feel like you need to do anything in a hurry. I'm also a bit preoccupied, so nothing is time-critical. No pun intended. In fact, if you do get some free time you should check out this series about the radio systems on the Voyager probes from EEVblog! Just saw your latest post: |
I've got it working on a different board. For some reason the TTGO LoRa32 v1 has an issue where it fully appears to transmit the time, but doesn't actually send anything. It's now running okay on the T3S3. The T3S3 seemed like it had issues at first, but dropped the issues and started working after some troubleshooting and I'm unclear what fixed them. |
Hey Jeff, Using T7-S3 and SX1278 it seems to send only one of the three LoRa time packets. This is similar to what the T3-S3 (with SX1262) did at first, though I'm not sure how I corrected it. I think this could be an issue with timing, and several receptions/transmissions happening too quickly. The packet that it does receive is the one destined for neighbor#2, which is the last packet broadcasted. I think these are two separate issues, but I'm not positive. Could you share the the setup(s) that you have working? I have done some troubleshooting and cannot solve the first issue. EDIT: |
I've got a setup where I have a LoRa Neighbor 1 (N1) and no neighbor 2 and no LoRa sensor yet. The N1 node doesn't reliably receive LoRa time periodically from the LoRa repeater that should be sending out the time. I'm not sure whether the issue is the transmitting repeater side or the receiving neighbor side yet. I'll have to do more testing on this. I'll add in some LoRa pings and add a LoRa sensor to determine what is happening.
All of my LoRa nodes are using a Wemos ESP32 with the Hallard LoRa shield.
Thanks, I'll look for this and see if this is part of the problem. |
@timmbogner, just wanted to let you know I'm continuing to work on this feature. I don't know for sure when I'll have the whole thing done but I'm hoping to be able to spend several hours a week on it. I'm not really sure where I left off but I have time going from MQTT node -> Serial -> ESP-NOW -> LoRa -> LoRa - all of those gateways so I need to test the sensor and gateway code. Anyways, just wanted to let you know. |
Greetings @aviateur17, that sounds awesome! If I recall correctly, the ESP-NOW implementation was perfect, but there was a glitch in the LoRa portion. One thing I'd propose is that we start with a fresh pull request. This one is kind of a lot to go through, although mostly due to my own blunders. An important upcoming change is going to be functionally major. I'm going to extend the 'data' portion of the DataReading structure to 5 bytes, and use CBOR to allow the data to be boolean, integer, binary, or float. This will only extend the DataReading to 8 bytes, which I think is pretty reasonable. There shouldn't be too many external changes for users. I will add new iterations of loadFDRS() for the different datatypes and then everything else will be handled internally. What do you think? SystemPackets will stay the same, so I'm not sure if it will affect your work here on timekeeping. I have a rough sketch worked out with a lot of the experimentation done using this TinyCBOR library. I'll make it a priority to make some progress on it and post something today or tomorrow. Thanks! |
@timmbogner , agreed on posting a new pull request. When I'm ready or close to ready and done with testing I'll post a new PR against the updated main branch. Regarding the change in the structure of the DataReadings, in the LoRa code I'm using the length to differentiate between valid packets and invalid packets and then going in and dissecting what is actual data and what are the headers so once you have that change we'll need to go into that LoRa code and modify those parts. Shouldn't be too difficult unless the size of the data is variable in nature. |
Cool! Nope, the data will now be a constant 5 bytes, so it'll look like this (probably): typedef struct __attribute__((packed)) DataReading {
uint8_t[5] d;
uint16_t id;
uint8_t t;
} DataReading; I'm relying on the fact that (in most cases) |
Closing old PR request. See PR #193 |
You can reject if you want to add this to a different branch in order to wait to include in the main release branch. This adds code in Gateway for NTP Time server queries via WiFi. Compiled in Arduino 1.8.19 with and without USE_WIFI defined.