STM32F1 update hang + SWBB support

[osman-aktepe]

Hi,

First of all thanks for your effort.

I am trying to connect an stm32f1 to esp-01 by using ThroughSerial. Once i accomplished to communicate but after that somehow i changed the code and lost my connection and can not achieve again. Whatever, the one of the problem is the code below hangs at bus.update() for a while later like 20 seconds. I changed lots of combinations and parameters.

I assume that, even if i can not have connection, it should work (I am not quite sure)

Please can you tell me what am i doing wrong.

Here is the code:
`
#define USE_RANDOM_SEED_PB0 true
//#define PJON_INCLUDE_ASYNC_ACK true
#define PJON_INCLUDE_TS
//#define TS_RESPONSE_TIME_OUT 10000
//#define TS_BYTE_TIME_OUT 50000
//#define TS_MAX_ATTEMPTS 5

#define PJON_PACKET_MAX_LENGTH 128 // Make the buffer big enough
#define PJON_MAX_PACKETS 1 // Reduce number of packets not to empty memory
//#define PJON_INCLUDE_PACKET_ID true

//#define portModeRegister(pin) (&digitalPinToBitMask(ETHERNET_SHIELD_SPI_CS))
#include <PJON.h>
// bus(selected device id)
PJON bus(45);

int inByte; // Byte read from Serial1
unsigned long lastMilis = 0;

uint32_t latency = 100000;

void setup() {
Serial1.begin(115200);
Serial.begin(115200);

//bus.set_crc_32(true);
//bus.set_packet_id(true);
//bus.set_synchronous_acknowledge(false);
//bus.set_asynchronous_acknowledge(true);
bus.strategy.set_serial(&Serial1);
//bus.set_synchronous_acknowledge(true);
bus.begin();
bus.set_error(error_handler);
};

void error_handler(uint8_t code, uint16_t data, void *custom_pointer) {
Serial.print("Error: ");
Serial.println(code);
Serial.println(data);
}

void loop() {
//delay(100);
const char *content = String("|AOBOC1:25000&|AOBOC2:24000&|AOBOC3:23000&|AOBOC4:22000&").c_str();

if(millis() - lastMilis > 1000) {
if(!bus.update()) {
Serial.println(bus.send(44, content, strlen(content)));
Serial.println("Content sent");
} else {
Serial.println("Content can not sent");
}

lastMilis = millis();

}
bus.update();

};

`

[gioblu]

Ciao @osman-aktepe try disabling the synchronous acknowledgment on both sides.
I have experimentally determined that the acknowledgment procedure is not feasible for uses where the hardware serial data-link is involved. Will follow the update to READMEs and examples to more clearly state this.

[osman-aktepe]

Ciao @gioblu the problem is i tried asynchronous acknowledgment again now and i am still facing the problem, i jus decomment these:

#define PJON_INCLUDE_ASYNC_ACK true

bus.set_synchronous_acknowledge(false);
bus.set_asynchronous_acknowledge(true);

are not these enough?

[gioblu]

Ciao @osman-aktepe could I ask you to give a try disabling both acknowledgment schemes and see what happens? It works fine with this configuration?

[osman-aktepe]

Ciao @gioblu First of all thanks for quick responses 👍 I tried what you say and it could stay like for 10 seconds, if i only set this bus.set_synchronous_acknowledge(false); it could go for like 60 seconds. if i set bus.set_asynchronous_acknowledge(true); it could go for like 20 seconds. I thought that something drain its memory but after that i added freeRam and there was no change until hang and as you can see i do not know what happened after hang.

If i add delay, it can last longer. I have changed lots of times but i got no chance. Do you have any other suggestions. By the way, i am using mapple mini and i have severals and i tried for all of them.

[gioblu]

Ciao @osman-aktepe I am sorry my answers could have been faster, but I am receiving a flood of requests in the last days, still not clear the source of this spike :)

Your tests were a bidirectional exchange or just sending from a device to another?
Maple to Maple or Maple to Arduino?

Thank you for your support :)

[gioblu]

@osman-aktepe another thing could I ask you to verify if the same condition happens with no acknowledge or synchronous using just send_packet instead of calling send and update?

[osman-aktepe]

@gioblu basically my intended usage is sending from mapple minis to esp and if i can achieve bidirectional but the crucial direction is from multiple maple mini to esp. By the way i tried send_packet and it sends approximately 120 packets and hangs again (with delay(100)).

I am insisting to use your library because it really helps to handle data and error, far better instead of i2c :)

[osman-aktepe]

@gioblu and here is the another test, I changed it to SWBB, with only send_packet it does not hang but with update and send it hangs again which is quite interesting. 🤕

[gioblu]

That is strange, @osman-aktepe I agree.
STM32F1 support is quite early, added initially by @Matheus-Garbelini with ThroughLoRa strategy and tests are demonstrating that the behavior is solid with this data-link.

Consider that SoftwareBitBang does not support for now STM32F1, so there is for sure something that may be worked out. I have never personally tested SoftwareBitBang strategy with this mcu. There may be some aspect of the network or data-link layer implementation that is not liked by the st chip.

If you have some time could you try to add some debug prints and see which is the last function that is called before hanging?

In any case, I have a couple of chinese blue pill boards laying around recently arrived from there, will solder the pins and give them a try.

[osman-aktepe]

@gioblu I found the problem :) in strategies.handle_collision the code calling
PJON_DELAY_MICROSECONDS(PJON_RANDOM(SWBB_COLLISION_DELAY));

which is

#ifndef PJON_RANDOM
#define PJON_RANDOM random
#endif

the problem is when it returns 0 it hangs for

#ifndef PJON_DELAY_MICROSECONDS
#define PJON_DELAY_MICROSECONDS delayMicroseconds
#endif

I think this one does not accept 0 from stm32, i am not sure for other architectures. I changed it as in the code that if it is 0 give as 1, and it seems okay :). I am thinking to change PJON_RANDOM macro to handle this but i am not sure about affecting other parts or changing behavior. What do you suggest?

[gioblu]

Ciao @osman-aktepe thank you so much for your support and effort.
Yes, it seems you really quickly found the solution, my compliments :)

It probably may be safer to conditionally define in the Arduino interface (before the default version):
https://github.com/gioblu/PJON/blob/master/src/interfaces/ARDUINO/PJON_ARDUINO_Interface.h#L69
a version of PJON_RANDOM dedicated to STM32F1 looking for its related defined constant containing the change you did to avoid parameter value = 0.

I have tested back Arduino compatible devices and delayMicroseconds(0); or random(0); do not freeze the device. Infact in 7 years of testing I have never before seen a hang using SoftwareBitBang, it had to be something related to STM32F1. For this reason it may really be safer to define a version of PJON_RANDOM for this mcu not to influence procedure on mcus where this issue does not occur.

Thank you again.

[gioblu]

Ciao @osman-aktepe here the conditional fix I was speaking about, to be added here:
https://github.com/gioblu/PJON/blob/master/src/interfaces/ARDUINO/PJON_ARDUINO_Interface.h#L69

  #ifdef __STM32F1__
    #ifndef PJON_RANDOM
      #define PJON_RANDOM(R) random(R ? R : R + 1)
    #endif
  #else
    #ifndef PJON_RANDOM
      #define PJON_RANDOM random
    #endif
  #endif

Could you give it a try?
Thank you again for your support.

[Matheus-Garbelini]

@osman-aktepe what's the arduino board support package you are using for the STM32F1?
Also It's the commom Stm32F103C8T6 part or a different?

[osman-aktepe]

Ciao @gioblu I could not test because i had a job to finish but you can be sure that i will test it as soon as possible and i will let you know. To be honest, i had the intention to write the macro and share with you. Thanks for the macro :) I will check for other delays, if it is only the problematic one, maybe we can write another macro for just delayMicroseconds one.

Hi @Matheus-Garbelini, I am using ordinary maple mini which has STM32F103CBT6.
Also the hardware files i used here: https://github.com/rogerclarkmelbourne/Arduino_STM32. If you need anything else, do not hesitate to ask.

I checked the delayMicroseconds in the hardware files which directly executes assembly code. Probably it is a bug of this microchip, to be honest i have no idea, i did not check the hardware level details.

Thanks for your support.

[osman-aktepe]

And also I forgot to mention that, maple mini with this hardware files does not recognize a pin as A0 which is used in PJON.h for random seed. I added a code like this but I have no idea for other STM32 micro controllers and i am adding that flag as true in my code. You have tested more devices i am only using ESP-01 and Maple Mini. I do not know maybe it is a problem for only maple mini but still i wanted to share this too.

 #if(USE_RANDOM_SEED_PA0)
      uint8_t random_seed = PA0;
    #else
      uint8_t random_seed = A0;
    #endif

[gioblu]

Ciao @osman-aktepe thank you very much, absolutely that has to be fixed, maybe before Arduino.h inclusion in the ARDUINO interface:

  #ifdef __STM32F1__
    #ifdef PA0
      #define A0 PA0
    #endif
  #endif

Another solution could obviously be to avoid the automatic random seed setup and let users call set_random_seed on the target architecture as required.

[gioblu]

Ciao @osman-aktepe, thank you again for your support, I have referenced a couple of commits to fix the issues you have rised. I have tried to compile for generic module / blue pill using the Arduino IDE and it is working fine. Could you give a try to master?

Thank you again.

[Matheus-Garbelini]

@osman-aktepe So I've forced manually tried delayMicroseconds(0); in some sketch for STM32F103C8T6 and in fact it freezes. As in my application, I'm not using random delay nor random seed for it, I didn't found this problem. I'd strongly suggest you open an issue in the rogerclark board support github so he can fix this issue. It's funny to think it didn't get noticed before.

Alternatively, if you want to fix the core files, go to Arduino_STM32/STM32F1/cores/maple/wirish_time.cpp

and edit the function delayMicroseconds:
void delayMicroseconds(uint32 us) {
delay_us(us);
}

Libmaple also reports that the maximum duration for this is (2^32 - 1) / 12 microseconds, or less than 6 minutes, that explains that with the currently assembly code implemented, there's an underflow that leaves the comparation register in it's maximum value.

[gioblu]

Ciao @osman-aktepe thank you very much for your support to the project and for your detailed report.
I will open an issue immediately in the rogerclark's repo.

As soon as the bug will be fixed I will remove 2969841

Thank you very much again :)

[osman-aktepe]

Hi @gioblu sorry for late response, i am on a vacation. I have found an opportunity to test, here is my foundings:

  #ifdef __STM32F1__
    //#ifdef PA0
      #define A0 PA0
    //#endif
  #endif

It worked like this, other change has no effect but after changing the code you changed in maple mini, it worked so it may mean that your other code has no effect.

Now it does not hang but still i can not communicate each other :( Do you have any other recommendation or do you have any working setup like mine? Also i am planning to communicate one esp with several stm32f1, serial is really appropriate for this setup?

[gioblu]

Ciao @osman-aktepe thank you for your response.
Strange, you are not able to get communication running at all?
If you want to use the ESP as the "master" connected to the internet, maybe the use of SoftwareBitBang strategy using one of the supported devices could be a better choice for a less complex, cheaper and more reliable setup.

I will run some tests with the bluepills.

[osman-aktepe]

Ciao @gioblu my electronic knowledge is not so good but i have some hesitations for SWBB because i am planning to connect 100 stm32f with one esp and they will send data to the esp and the esp will send them to the server. The cable line will be probably 50-60 meters and SWBB has higher error ratio which means it can have trouble because i am planning to send each data in every 1 or 2 seconds from stm's. Did you have any experience with SWBB for a scenario like this? ( As I know serial is more reliable but SWBB has advantages like easy connection)

[gioblu]

@osman-aktepe how you are planning to connect 100 stm32f using serial? You mean using RS485?
How you determined that SWBB has an higher error ratio?

I am not sure Serial is more reliable than SWBB, also Serial is generally a point-to-point dual channel connection so it is not done to connect more than a couple of devices toghether.

[gioblu]

@osman-aktepe whatever is the strategy you select, my suggestion considering the numbers and the duty cycles you are speaking about is to branch the network, so if a problem occurs, only a section is affected, and also doing so you are adding parallel collision domains that would effectively multiply your network's overall available bandwidth.

                    _________
        ___________| ESP8266 |___ SWBB 2.11kB/s 
       |           |_________|
 ______|_           _________
| SERVER |_________| ESP8266 |___ SWBB 2.11kB/s
|________|         |_________|
       |            _________
       |___________| ESP8266 |___ SWBB 2.11kB/s
                   |_________|
                                  Overall bandwidth: 6.33kB/s
                                  Collision domains: 3

[osman-aktepe]

@gioblu You are right about the connection problem with serial, currently i am just focused on to communicate one with another. I2c ports as i know have hardware support (especially repeating) can i use pjon over i2c? (I know i2c is a bus too but it is too primitive :) )

Also i could not undestand what you mentioned about? How can i achieve this?

Also during sending data to server my esp will be busy for like 100-200 ms, but i assume that they will re-send again but the duration is too short according to 100-200 ms so these data will be lost right?

Thanks for your help :)

[osman-aktepe]

@gioblu Also it is like as you draw but one esp should handle 100 stm because there are around 100 esp :) there will be 12500 stm's .If i try to increase esp count, the problem i will face access point connection and noise.

[gioblu]

Ciao @osman-aktepe i2c is designed to connect ICs in circuit boards, not to connect hundreds of devices for hundreds of meters, transporting the clock line around, is particularly susceptible to interference, so I would exclude that.

What I mentioned is to just use more than one ESP connected to the internet that sends data to the server that are connected to 3 separate SWBB buses. See graph included in the earlier answer.

Yes, a PJON packet transmission retries 20 times in more than 3 seconds before returning an error, so although some bandwidth is lost the packet is delivered properly also if ESP is dead for a second.

[gioblu]

@osman-aktepe 12500 stm's??? What you are building a smart city? heheh

[osman-aktepe]

Kinda :) I am gonna cover a whole factory. So what i focused on is to communicate around 100 stms with esp, we have tested i2c for this cases and it works like expected (it can work for like 100 meters). SWBB has 2.11 kb/s bandwith at most? I do not wanna use can bus :) i can check swbb for 60 meters too but bandwitdh is crucial too because i need at least 3.2kb/s. but if it works i may decrease the cycle which requires around 1.1 kb/s which i would not prefer.

[gioblu]

@osman-aktepe using SWBB and an Arduino Duemilanove/Uno/Nano/Micro/Mega/Leonardo + Ethernet shield as masters and Duemilanove/Uno/Nano/Micro/Mega/Leonardo as slave you can use MODE 2: 19.417kBd or 2.42kB/s

If you use only Arduino Duemilanove/Uno/Nano you may run at MODE 3: 27.027kBd or 3.378kB/s

[osman-aktepe]

Okay i figured it out i calculated wrong i need 12.8 kb/s :( can i use multiple pin to increase bandwidth.

[gioblu]

@osman-aktepe you need 12.8kbits/s or 12.8kBytes/s ?

[osman-aktepe]

12.8 kbytes/second which is raw i do not count packets other fields

[gioblu]

@osman-aktepe no way for now to use SWBB if that is the requirement.
More probably could have sense to use serial over RS485 with the desired baudrate.

[gioblu]

@osman-aktepe see also : https://en.wikipedia.org/wiki/I%C2%B2C

Because of these limits (address management, bus configuration, potential faults, speed), few I²C bus segments have even a dozen devices. It is common for systems to have several such segments.

[osman-aktepe]

Hi @gioblu, I found a way to decrease the required bandwidth, I will test for long cables like 20 meters :) which will be quite interesting. I thought to use the can bus also; will be exhausting and makes the design more complicated. I know i2c's problems which I insist to use PJON but I have a question, what makes Uno works on Mode3 but stm32 or esp8266 on Mode1?

[gioblu]

Ciao @osman-aktepe I need some time to work out the ESP8266 MODE 2 and 3 timing if that is possible, still haven't dedicated time on that.

[osman-aktepe]

Hi @gioblu I am about to get mad :( SWBB is not working either, once somehow i achieved to make it work (ThroughSerial), after that i used different stms and esps, nothing changes, I am always getting 101. What can I do ?

By the way, if you give me documentation to write mode 2 or 3 I can help, at least I can do whatever I can.

[gioblu]

Ciao @osman-aktepe consider that stms are still not supported by SWBB.
See https://github.com/gioblu/PJON/wiki/Interfacing

[osman-aktepe]

Hi @gioblu I am sorry, I believe that I am bothering you a lot.
I have tested lots of different parameters and this seems the most stable one.

/* STM32F1 ---------------------------------------------------------------- */
#if defined(__STM32F1__)
  #if SWBB_MODE == 1
    #if F_CPU == 72000000L
      #define SWBB_BIT_WIDTH   44
      #define SWBB_BIT_SPACER 112
      #define SWBB_ACCEPTANCE  40
      #define SWBB_READ_DELAY  -10
    #endif
  #endif
#endif

I am mostly getting these values:

---------------------
Packet Overhead: 10B - Total: 580B
Bandwidth: 1740B/s
Data throughput: 1160B/s
Packets sent: 58
Mistakes (error found with CRC): 0
Fail (no acknowledge from receiver): 2
Busy (Channel is busy or affected by interference): 0
Accuracy: 100.00 %
---------------------

Does it look okay? Is not bandwidth a little bit low?

[gioblu]

Beautiful @osman-aktepe my compliments!!! Have you used trial and error ??
Would you open a pull request to record your contribution?

[osman-aktepe]

If you mean facing problems by saying "Have you used trial and error" yes I had, especially connection problems, after using pull down resistor, even if I remove and reattach the wire, it could connect properly. If I can optimize it more and you guide, I can work on different parameters. If it seems okay for you, I can create pull request.

[gioblu]

CIao @osman-aktepe thank you very much for your support.
I would have solved for you the compatibility issue, but I am trying to coupe with other issues and requests, thank you for dedicating time on that.

There are 3 examples that can be used to assess the quality of the timing you have found:

1. PacketSeparationTest checks that the frame initializer is detected as expected with no false positives
2. NetworkAnalysis used to assess the quality of communication on transmitter side
3. SpeedTest used to assess the quality of communication on the receiver side

If all tests are fine, you can feel free to open a pull request 👍

[gioblu]

Ciao @osman-aktepe thank you very much, I have added your timing as I was able to replicate your results. I will as soon as possible dedicate some time to effectively tuned those values for higher performance and reliability.

[gioblu]

Will keep attention to roger's releases, closing for now, feel free to re-open if required.
Thank you again!