Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pro8MHz board does join but does not transfer messages #17

Closed
SensorsIot opened this issue Jul 22, 2021 · 17 comments
Closed

Pro8MHz board does join but does not transfer messages #17

SensorsIot opened this issue Jul 22, 2021 · 17 comments
Labels
question Further information is requested

Comments

@SensorsIot
Copy link

I wanted to use a LoRa Radio Node board with an 328 chip and OTA on the V3 stack. It uses the old LMIC library and joins, but does not transfer messages. Serial stops after:

LMIC-node

Device-id: pro-mini
LMIC library: Classic [Deprecated]
Activation: OTAA
Interval: 60 seconds

Clock Error: 30000 ppm (1965)
000000000817: Event: EV_JOINING

000000000882: doWork job started

Unfortunately, the memory is too small to use the new MCCI library. Was anybody successful to use an "Arduino-like" board on the V3 stack?

@lnlp
Copy link
Owner

lnlp commented Jul 22, 2021

Hi @SensorsIot,

Pro8MHz board does join but does not transfer messages

I see the EV_JOINING event in your output but not a corresponding EV_JOINED. This means that the node is not yet joined.
Joins can fail if DIO1 is not correctly mapped/connected but that does not explain your serial output stopping.

I wanted to use a LoRa Radio Node board with an 328 chip and OTA on the V3 stack. It uses the old LMIC library and joins, but does not transfer messages. Serial stops after: ... doWork job started

What 'LoRa Radio Node board with an 328 chip' exactly did you use?

You appear to have selected the pro8mhzatmega328 ('pro-mini') boardid in LMIC-node.
This is based on the Arduino Pro-Mini 3.3V 8 MHz. Wiring of the LoRa radio needs to exactly match the connections as described in the corresponding BSF file (bsf_pro8mhzatmega328.h).
If your ATmega328 based board is different, e.g. the LoRa radio is already hard-wired on the PCB then you will probably need to adapt the (lmic) pin mappings in the BSF to match the connections on the board.

Did you:

  • make any changes to LMIC-node other than the configuration steps described in the Instructions chapter in README.md?
  • verify if pin mappings in the BSF and the actual hardware connections are correct and match?
  • check if the node actually hangs or only serial output stopped?
  • check Live data in the TTS V3 console for join requests/accepts and uplinks?
    what does it show?

Was anybody successful to use an "Arduino-like" board on the V3 stack?

Yes, I have LMIC-node working on TTN V3 (TTS V3 CE) with both Arduino Pro-Mini (clone) and BSFrance LoRa32u4 II.
Due to limited memory LMIC-node has to use the older (matthijskooijman) Classic LMIC library for these MCU's.

In addition I also sucessfully tested the Pro-Mini with MCCI LMIC using its included ttn-otaa.ino example.
The results of both tests with the Pro-Mini are shown below.

FYI: LMIC-node was successfully tested with all its supported boards on both TTN V2 and TTN V3 (TTS V3 CE).
Except for 'Heltec Wireless Stick' (not tested by me due lack of the hardware and no feedback yet from others).

For TTN V3 OTAA device settings I used:
LoRaWAN version: MAC V1.0.3
Regional Parameters version: PHY V1.0.3 REV A
Frequency plan: EU863-870 MHz (SF9 for RX2 - recommended)
(and for AppEUI: all 00's)
MAC V1.0.3 may not be the best match for Classic LMIC but it is for MCCI LMIC. For testing LMIC-node on all supported boards I use only a single lorawan-keys.h file with two sets of device keys, one for OTAA and one for ABP (so for the testing, in the console only two generic devices are defined, one for OTAA and one for ABP).

It works fine (including downlinks) for my pro-mini(s). Below the serial output:
(I have enabled external display as well, so I can check output on both serial and display.)

LMIC-node

Device-id:     pro-mini  
LMIC library:  Classic [Deprecated]
Activation:    OTAA
Interval:      20 seconds

Clock Error:   30000 ppm (1965)
000000032827:  Event: EV_JOINING 

000000036598:  doWork job started
000000407266:  Event: EV_JOINED  

000000411115:  doWork job started
000000414311:  Input data collected
               COUNTER value: 1    
000000416032:  Packet queued
000000811667:  Event: EV_TXCOMPLETE
               Up: 1,  Down: 0

000001661121:  doWork job started
000001664315:  Input data collected
               COUNTER value: 2    
000001666037:  Packet queued
000002061759:  Event: EV_TXCOMPLETE
               Up: 2,  Down: 0

000002911129:  doWork job started
000002914322:  Input data collected
               COUNTER value: 3    
000002916045:  Packet queued
000003311959:  Event: EV_TXCOMPLETE
               Up: 3,  Down: 0

000004161137:  doWork job started
000004164330:  Input data collected
               COUNTER value: 4    
000004166054:  Packet queued
000004561793:  Event: EV_TXCOMPLETE
               Up: 4,  Down: 0

000005411143:  doWork job started
000005414338:  Input data collected
               COUNTER value: 5    
000005416061:  Packet queued
000005811807:  Event: EV_TXCOMPLETE
               Up: 5,  Down: 0

000006661152:  doWork job started
000006664344:  Input data collected
               COUNTER value: 6    
000006666066:  Packet queued
000006989141:  Event: EV_TXCOMPLETE
               Up: 6,  Down: 2
               Downlink received
               RSSI: -86 dBm,  SNR: 10.0 dB
               Port: 100
               Length: 1
               Data: C0
               Reset cmd received
000006999511:  Counter reset

000007911158:  doWork job started
000007914351:  Input data collected
               COUNTER value: 1    
000007916075:  Packet queued
000008311822:  Event: EV_TXCOMPLETE
               Up: 7,  Down: 2

Here with display disabled. Timing of the first events is now more similar to the timing in your output (I use a shorter doWork interval for testing though):

LMIC-node

Device-id:     pro-mini  
LMIC library:  Classic [Deprecated]
Activation:    OTAA
Interval:      20 seconds        

Clock Error:   30000 ppm (1965)  
000000000777:  Event: EV_JOINING 

000000000841:  doWork job started
000000358469:  Event: EV_JOINED  

000000358541:  doWork job started
000000361737:  Input data collected
               COUNTER value: 1    
000000361838:  Packet queued       
000000753500:  Event: EV_TXCOMPLETE
               Up: 1,  Down: 0     

000001608547:  doWork job started
000001611739:  Input data collected
               COUNTER value: 2    
000001611845:  Packet queued       
000002003504:  Event: EV_TXCOMPLETE
               Up: 2,  Down: 0

Unfortunately, the memory is too small to use the new MCCI library.

Yes, for using MCCI LMIC with LMIC-node the memory on ATmega328 and ATmega32u4 is just too small. But you can run MCCI LMIC with its included ttn-otaa.ino example (and other LoRaWAN applications if kept small enough) on these MCUs.

Don't forget to add setting clock error to 30000 ppm (1965) in ttn-otaa.ino or else it won't join.
See below example extract for setup():

    // LMIC init
    os_init();
    // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();

    #define LMIC_CLOCK_ERROR_PPM 30000
    #if LMIC_CLOCK_ERROR_PPM > 4000
        // Allow clock error percentage to be > 0.4%
        #define LMIC_ENABLE_arbitrary_clock_error 1
    #endif    
    uint32_t clockError = (LMIC_CLOCK_ERROR_PPM / 100) * (MAX_CLOCK_ERROR / 100) / 100;
    LMIC_setClockError(clockError);

After compilation PlatformIO shows the following memory usage:

RAM:   [========  ]  78.4% (used 1605 bytes from 2048 bytes)
Flash: [========= ]  93.5% (used 28716 bytes from 30720 bytes)

Output from ttn-otaa.ino (uplinks are visible in V3 console Live data):

Starting
Packet queued   
2650: EV_JOINING
377143: EV_TXSTART
698263: EV_JOINED
netid: 19
devaddr: 260BAF26
AppSKey: 13-C1-77-1B-81-F4-6D-5D-19-76-03-41-F5-D6-D8-5A
NwkSKey: CD-AF-72-BE-45-6D-48-DA-38-D2-18-17-76-01-D6-91
699360: EV_TXSTART
1081085: EV_TXCOMPLETE (includes waiting for RX windows)
2331632: EV_TXSTART
Packet queued      
2713362: EV_TXCOMPLETE (includes waiting for RX windows)
3963814: EV_TXSTART
Packet queued      
4345544: EV_TXCOMPLETE (includes waiting for RX windows)
5595995: EV_TXSTART
Packet queued      
5977725: EV_TXCOMPLETE (includes waiting for RX windows)
7228177: EV_TXSTART
Packet queued      
7609905: EV_TXCOMPLETE (includes waiting for RX windows)

@lnlp lnlp added the question Further information is requested label Jul 22, 2021
@al1fch
Copy link

al1fch commented Jul 23, 2021

Hi @Sensorslot and @lnlp !!

This thread helped me to get a Pro Mini work on TTS V3 stack (tested OK with OTAA)
For the test I use the recent MCCI LMCIC V4 library and ttn-otaa.ino exemple from here
Adding a DIO1 connection was necessary (DIO1 was unconnected before for ABP mode, TinyLora Library, TTN V2, single chanel
Clock error set to to 30000 ppm (this thred pointed me to that point !!)

Flash : 80%
RAM : 74%
fot ttn-otaa exemple, as is, without user's code

So Mega 328(pro mini) and TTS V3 are OK, I can confirm with your help !!

LMIC.node tested OK on a Ttgo LoRa Oled V1 borad for new OTAA or ABP V3 devices
Very easy to configure and test !!

Best Regards

@SensorsIot
Copy link
Author

Thank you for both answers. I tried now again with the 328 chip as well as with TTGO boards and discovered strange things: Sometimes the payload is transferred ok with both boards (using this sketch, 328 with the old library, and the TTGO with the new library). First, I tried with OTAA. Often, I saw that the join request was accepted by TTNV3 but my node did not transmit the payload. I have to say: I want to use deep sleep and for the moment I often reset my board to simulate this effect. Unfortunately, I did not find a deep-sleep example that does not re-join every time. Such an example would be extremely important because most LoRa sensors use deep sleep.
So I moved to ABP. This used to work ok with V2. Unfortunately, I get a lot of times "Uplink not scheduled because TxRx pending". This, BTW, appears now also when I use OTAA. Very strange.

@lnlp
Copy link
Owner

lnlp commented Jul 26, 2021

using this sketch, 328 with the old library, and the TTGO with the new library

With this sketch you mean LMIC-node?

I tried now again with the 328 chip as well as with TTGO boards

Again, what 328 board exactly?
Which TTGO boards?

Previous questions:

What 'LoRa Radio Node board with an 328 chip' exactly did you use?

Did you:

  • make any changes to LMIC-node other than the configuration steps described in the Instructions chapter in README.md?
  • verify if pin mappings in the BSF and the actual hardware connections are correct and match?
  • check if the node actually hangs or only serial output stopped?
  • check Live data in the TTS V3 console for join requests/accepts and uplinks?
    what does it show?

You seem not willing to provide details asked for. Without details, exact type of board, console data etc. it will be very difficult to try help determine what the possible cause of your problem is.

I want to use deep sleep and for the moment I often reset my board to simulate this effect. Unfortunately, I did not find a deep-sleep example that does not re-join every time. Such an example would be extremely important because most LoRa sensors use deep sleep.

Deep sleep implementation and behavior dependends on the type of microcontroller and often also dependends on the rest of the hardware on the (development) board. As LMIC-node is a relatively basic cross-platform example application for showing how to use the LMIC library, adding support for deep sleep for different families of microcontrollers would be a huge project in its own. Also the (MCCI) LMIC library currently does not have builtin (generic) support to save and restore LMIC state that could be used for (e.g. ESP32) deep sleep. Therefore deep sleep is not supported in LMIC-node.

I often reset my board to simulate this effect

I guess these simulations mainly target behavior of ESP32 chips which do not maintain their memory/state during deep sleep.

Unfortunately, I did not find a deep-sleep example that does not re-join every time.

Possibly the ttn-esp32 library mentioned in below topic contains an example and can provide help for LMIC with ESP32 deep sleep: https://www.thethingsnetwork.org/forum/t/overview-of-lorawan-libraries-howto/24692

So I moved to ABP. This used to work ok with V2.

Please note that ABP on V3 is not exactly the same as on V2. It is more strict and requires additional device settings to be specified when adding the device in the TTS console (or via CLI tools). In the console, part of these required parameters must be manually added; they are not automatically set by default.
For more information see topics in the V2 to V3 Migration category and the How to Migrate ABP Devices from V2 to V3 guide on The Things Network forum.


For now (and without additional details) your problems appear not to be caused by an issue in LMIC-node.

@al1fch
Copy link

al1fch commented Jul 26, 2021

Unfortunately, I did not find a deep-sleep example that does not re-join every time.
This one seems OK, tested : No re-join, ESP32 RTCRAM is used to save what is to be saved .
(uses MCCI LoraWan LMIC V4)
tested for a new device registerd in TTS V3,

8mA during deep-sleep with this exemple as is on a TTGO Oled LoRa V1
https://jackgruber.github.io/2020-04-13-ESP32-DeepSleep-and-LoraWAN-OTAA-join/
https://github.com/JackGruber/ESP32-LMIC-DeepSleep-example/

@lnlp lnlp closed this as completed Jul 27, 2021
@SensorsIot
Copy link
Author

You were right. I found the problem (it took quite a while and is the reason for my late answer): DIO1 was connected to a different pin (from an old project). Now OTAA works with the TTGO board.
Thank you for the lint to the files with deep sleep. I will try to add this knowledge to my project.
I did not migrate my devices, I created new ones (I only have a few) to avoid any migration problems.

@lnlp
Copy link
Owner

lnlp commented Jul 28, 2021

Thanks for your feedback.

I did not migrate my devices, I created new ones (I only have a few) to avoid any migration problems.

The link to the "How to migrate ABP Devices..." contains information that may still be useful for you.
It contains information that is currently difficult to find (or not yet available) in the official documentation.
Especially the extra parameters that need to be explicitly specified in Network layer | Advanced MAC settings when adding a new ABP device (which is not done by default) e.g. explicitly adding Factory preset frequencies.

@SensorsIot
Copy link
Author

And now I got it also working on the pro mini 3.3v board. Thank you, lnlp: The sketch really needed your addition

#define LMIC_CLOCK_ERROR_PPM 30000
#if LMIC_CLOCK_ERROR_PPM > 4000
// Allow clock error percentage to be > 0.4%
#define LMIC_ENABLE_arbitrary_clock_error 1
#endif
uint32_t clockError = (LMIC_CLOCK_ERROR_PPM / 100) * (MAX_CLOCK_ERROR / 100) / 100;
LMIC_setClockError(clockError);

Unfortunately, I do not know what it is for:-( But it worked also with my TTGO board. What is the reason it is not included in the example file?

I created an example that stores the LMIC variables to EEPROM (I switch the board completely off and it starts only once a day).
Your links helped a lot. Thank you again. Now I have the needed examples for my video. Next is the migration of my gateway to V3...

@TD-er
Copy link

TD-er commented Jul 29, 2021

@SensorsIot
The clock error is to make sure the receive window isn't missed.
It requires quite an accurate timing to have the radio on on the right moment to receive the downlink messages.
So it is some trade-off between the lowest energy usage and making sure you don't miss downlink messages.

Maybe LMIC also uses it on a lower level to allow for a higher tolerance in tracing the chirp frequency hopping?
If so, then it might also affect the actual sensitivity.
here is some very short description of it.

Edit:
Found it referenced in the MCCI LMIC library documentation

@lnlp
Copy link
Owner

lnlp commented Jul 29, 2021

I created an example that stores the LMIC variables to EEPROM (I switch the board completely off and it starts only once a day).
Your links helped a lot. Thank you again. Now I have the needed examples for my video. Next is the migration of my gateway to V3...

@SensorsIot
Thank you. Good to hear you have it working now.
Don't forget to mention LMIC-node in your video. 🙂

What is the reason it is not included in the example file?

If with "the example file" you mean ttn-oota.ino included with the MCCI LMIC library:

  1. That question would best be asked in the MCCI LMIC Github repository.

  2. Setting ClockError is exceptional and should normally not be done. It loosens LMIC timing (makes timing less strict, less conforming to LoRaWAN specs) which normally has negative effects. It is needed for some slower 8-bit AVR MCUs but should not be used for anything else (unless practice has shown it is needed).

  3. The examples included with the MCCI LMIC library are less comprehensive than LMIC-node (see below excerpt from MCCI LMIC README.md). Different from LMIC-node they do not support a wide range of boards out of the box without having to make hardware dependent changes to the code.
    LMIC-node includes board-specific support for ClockError because it supports some boards that need it.

Excerpt from MCCI LMIC library README.md:
The examples in this library (apart from the compliance sketch) are somewhat primitive. A very complete cross-platform Arduino application based on the LMIC has been published by Leonel Lopes Parente (@lnlp) as LMIC-node.


I do not know what it is for ... But it worked also with my TTGO board.

DO NOT set ClockError for boards that do not need it (e.g. TTGO boards) as it will only make things worse, not better.

For all boards supported by LMIC-node you can check if ClockError is used (set) or not by checking the board's Board Support File (BSF). The README.md states to check BSF files for important information about a board.
Much efforts and time went into writing and testing LMIC-node so that it is simple and effortless for others to use (with a supported board), out of the box. Determining if setting ClockError (and which value) for a board is needed was part of those efforts.

Excerpt from the pro8mhzatmega328 BSF:

// LMIC Clock Error
// This is only needed for slower 8-bit MCUs (e.g. 8MHz ATmega328 and ATmega32u4).
// Value is defined in parts per million (of MAX_CLOCK_ERROR).
// Value 30000 was determined empirically.
#ifndef LMIC_CLOCK_ERROR_PPM
    #define LMIC_CLOCK_ERROR_PPM 30000
#endif   

Excerpt from the ttgo_lora32_v2 BSF:

// LMIC Clock Error
// This is only needed for slower 8-bit MCUs (e.g. 8MHz ATmega328 and ATmega32u4).
// Value is defined in parts per million (of MAX_CLOCK_ERROR).
// #ifndef LMIC_CLOCK_ERROR_PPM
//     #define LMIC_CLOCK_ERROR_PPM 0
// #endif   

@ALL
I've given some tips to make MCCI LMIC work with 8-bit AVR MCU's.
However, let's not forget that this repository is about LMIC-node and while LMIC-node uses the MCCI LMIC library, this repository is not the place to discuss the internals of the MCCI LMIC library and its included examples.

@SensorsIot
Copy link
Author

Thank you for your explanations. Now I understand better. I like your code and it runs well on ESPs. A lot of people still used 8 bit CPUs for their projects because the ESPs use quite some power. This is why I searched for a solution to use the newest LMIC library and V3. I learned that I have to recommend the OTAA example for this situation, which is ok with me.
I am glad you explained things which are not easily found and maybe it will help others to save time. Now, this thread is closed ;-)

@lnlp
Copy link
Owner

lnlp commented Jul 29, 2021

because the ESPs use quite some power

It's a pity that the ESP32 is often blamed for being power hungry, while this is true when run at full speed and using Wifi/BT.
When lowering clock speed to 10 MHz and not using Wifi/BT it is possible to get 8 mA power consumption during operation (assumed you do not let it crunch numbers or mine bitcoins).

For deep sleep it should be possible to go as low as 10 to 20 uA. But unfortunately most (Chinese) ESP32 development boards are not designed for efficient battery powered operation using deep sleep (but that you already knew 🙂).

While this does still not match popular ARM chips, it is already way better than "using quite some power".

Maybe some case for one of your videos?: A bare ESP32 module with 4 MB flash without PSRAM, running at 10 MHz, not using Wifi/BT, with SX1276/RFM95 LoRa module using the MCCI LMIC library.

BTW, did you notice the ESP32 low-power topic?

@SensorsIot
Copy link
Author

I agree with your analysis that the board often is the problem and not the chip. Maybe it would be an idea to do a test for the best LoRa board. Contenders would then also be the Pi Pico, an 328p, and maybe an STM Blue or Blackpill and an ESP32.
I created a Mailbox notifier that is completely switched off and started with a magnetic switch. This is why I needed a sketch that does not join all the time... And I was able to use a not-so-good board.
I still think you should include something for deep sleep/restart and OTAA because I do not know of many nodes running all the time.

@TD-er
Copy link

TD-er commented Jul 29, 2021

Not sure if the MCCI LMIC library allows access to the OTAA keys etc.
If it does and allows to restore them, then it would be a great idea to support it in this library (I also would like it for ESPEasy ;) )
Currrently I'm only aware of the RN2483 which allows to save/restore OTAA activation.

@lnlp
Copy link
Owner

lnlp commented Jul 29, 2021

I still think you should include something for deep sleep/restart and OTAA because I do not know of many nodes running all the time.

I think many people would like that.
However, as said before doing that cross-platform for all supported MCU families would be a huge project on its own.
Also MCCI LMIC currently does not provide any generic hooks for saving and restoring LMIC state. This will make a custom implementation more brittle and possibly non-future-proof (and hence maintenance intensive).

My goal for LMIC-node was to make using LMIC simple and accessible to a wide audience, while preventing them from running into invisible walls and having to go through steep learning curves first. Based on the feedback LMIC-node seems to succeed in that goal quite well.

@SensorsIot
Copy link
Author

For the moment I only saved the LMIC structure in EEPROM and reload it after boot (another project saves it in RTC memory for deep sleep). For the moment this works. But I only have a few hours of experience. I also saw a parameter that joins automatically from time to time. I plan to investigate this matter a little more.

@lnlp
Copy link
Owner

lnlp commented Jul 29, 2021

Not sure if the MCCI LMIC library allows access to the OTAA keys etc.

It already reads and stores all OTAA and ABP related LoRaWAN keys so that should not be an issue.

it would be a great idea to support it in this library

This functionality is on the LMIC Development Plans list but no further details have been made available yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants