Local bus going through walls / outdoors

[drtrigon]

Now as I have my minimal bus up and running (linux/raspi - LUDP - Uno+ETHshield - SWBB - devices) I would like to extend it to outdoor sensors. What setup would you suggest to go through walls (without drilling holes) or windows? I can imagine these setups:
A) SWBB - AS+LED - window - AS+LED - SWBB
B) SWBB - LoRa/radio - window/wall - LoRa/radio - SWBB
C) ?
for A+B I would like to use 2 tunnelers. The LoRa radio setup would be preferred as that would allow also to add devices to the LoRa bus (between the 2 tunnelers) and I do not need to setup LED line-of-sight communication. If LoRa is the best choice, I would like to use a Dragino LG01-S (Uno/Yun combo) for the SWBB - LoRa part, so also direct LUDP - LoRa tunneler is a possibility...

What would be your advice?

[gioblu]

ciao @drtrigon for sure LoRa is a choice, but if the requirement is a point to point connection, maybe HC-12 it can be cheaper and simpler. It can be used with SoftwareSerial and it has a range of around 150 meters without line of sight. If "paranoid" and you dont want to broadcast your data over a relatively wide range, you may also use AS along with a couple of laser diodes, although it is for sure a solution that would require some more tinkering and collimation, with line of sight it can work for hundreds of meters.

Obviously it is required to take extreme care working with laser specially when collimating paths, always wear protective glasses and if installing a definitive setup be sure its path cannot intersect human or other living creatures eyes' nearby. The use of infrared spectrum does not reduce the risk but just keeps the emitted light invisible.

In not recent tests I got more than 100m range out of a 10mW laser diode.

[gioblu]

@drtrigon I do not have personal experience with Yun and Dragino. Who has more experience with LoRa is @Matheus-Garbelini who is the creator of the ThroughLoRa strategy. For sure I would choose from the list of supported devices if possible.

[gioblu]

@drtrigon another thing, if you use a shared medium, be sure to use PJON in shared mode, in order to avoid potential collisions with others' devices nearby.

[Matheus-Garbelini]

@drtrigon Interesting, application. As @gioblu suggested, there are other options for point to point communication, but as your application has more than one wall/window sections (B option),
I strongly suggest you use Loras so you can avoid broadcast interference between the bridge points.
In order to do that you need to use a different spreading factor for each of the transmitter and receiver pair of the bridge sections:

LoRa.strategy.setSpreadingFactor(7)

and in the other segment:

LoRa.strategy.setSpreadingFactor(8)

This ensures that each PJON message frame is transmitted in an orthogonal way, avoiding broadcast issues.

Be aware that LoRa support in pjon is new so you need to tweak some timeout default values as discussed ni #220 (You'd need to modify 2 definitions).

As you already have a Lora Gateway at your disposal and it uses an SX1278 internally, the current PJON strategy appear to be compatible with your device. Looking through the software support for this gateway, it uses Radiohead rf95 library to communicate with the Lora, it means that the sx1278 is attached to the default SPI pins of the Arduino UNO that is inside Dragino LG01-S.
In short, you can run a simple through Lora sketch and it should work right away.

Tell us about your tests an we'd be happy to add to the documentation your device as supported. Thanks.

[drtrigon]

@gioblu: Thanks for your fast reply! I don't know the HC-12 radios and a quick view into google showed me no local distributor so I'll keep looking, but for now it does not seem like a quick alternative. Laser diodes are something I did consider too at some point. Do you have experience? How well does the beam need to be aligned to the receiving diode?
Another question is do you have an idea/experience how good the performance is when putting 3 tunnels in a row (LUDP-SWWP, SWBB-?, ?-SWBB) ? I mean do you expect it to work at all? ;)
I also wondered if it is possible to setup a "3-way tunneler", e.g. LUDP - SWBB - LoRa on a single Arduino Uno?

[drtrigon]

@Matheus-Garbelini: Thanks for your post and verifying the specs with the manual! That sounds great! I actually already wrote the SWBB - LoRa tunneler and the plan was to use it twice [1]. The thing is - I was not able to progress to that point, as the SWBB bus communication between my Uno+ETH (LUDP-SWBB tunneler [2]) and the Dragino LG01 SWBB part did not work properly. It did on a test setup with Nano (TS-SWBB tunneler [3]) so I currently have no idea (may be the bidirectional level shifter [4] causes issues) ... in any case I have to fix that first ... so at this point I first wanted to verify that my concept can work theoretically at least.

[1] https://github.com/drtrigon/sketchbook/blob/master/OWPJON/ARDUINO/Local/SoftwareBitBang/Tunneler/BlinkingSwitch_SWBB-TL_DRAFT/BlinkingSwitch_SWBB-TL_DRAFT.ino
[2] https://github.com/drtrigon/sketchbook/blob/master/OWPJON/ARDUINO/Local/SoftwareBitBang/Tunneler/BlinkingSwitch/BlinkingSwitch.ino
[3] https://github.com/drtrigon/sketchbook/blob/master/OWPJON/ARDUINO/Local/SoftwareBitBang/Tunneler/BlinkingSwitch_SWBB-TS/BlinkingSwitch_SWBB-TS.ino
[4] https://cdn-shop.adafruit.com/datasheets/txb0108.pdf

[gioblu]

I have experimented a lot with light pulses communication and laser communication using PJDLS, that is quite effective. Depending on the beam diameter at the receiver side a certain deviation in the path can be present without issues. So using a lens that has an extremely wide beam at a short distance (let's say a meter) the deviation can be big (a lot of degrees of error) although the maximum range will be obviously limited. In the opposite situation, with a small beam, you will have a longer range but a smaller acceptable deviation. The acceptable deviation is slightly less than the beam radius at receiver side. The beam diameter grows with the distance between transmitter and receiver. Knowing the beam diameter then with simple geometry is possible to determine at a given distance which is the maximum acceptable deviation for a system of this sort. Consider that the smaller is the beam the more dangerous the system is for eyes :)

[drtrigon]

Good news! I got most of it working! For the 2nd TL-SWBB tunnel, the time I had was to short - will follow. LoRa works! (I have that stuff - for laserdiodes I would have to wait several weeks.)
What is the recommended way for SWBB bus 3.3-5v level conversion? The bidirect. shifter seems not to behave well. Currently I use a voltage divider 68 ohms and 3.6v zener diode. That works on a 5v bus with 2 devices, but I expect it to cause issues on a bigger bus with longer cables?
Now I have to test a bit more and consider all your hints...

[drtrigon]

@drtrigon another thing, if you use a shared medium, be sure to use PJON in shared mode, in order
to avoid potential collisions with others' devices nearby.

@gioblu: How can I enable the "PJON shared mode" ? https://github.com/gioblu/PJON/wiki/Configuration does not seem to work for LoRa tunnel with PJONAny?

[gioblu]

Ciao @drtrigon my stupidity to leave outdated pages in the wiki, that will be deleted now.
Where is the link to that page?

This is the configuration docs (that are also linked in the wiki menu): https://github.com/gioblu/PJON/blob/master/documentation/configuration.md
there you find the info about shared mode and here:
https://www.pjon.org/get-bus-id.php
you can request a unique bus id.

Sorry for the inconvenience :)

[gioblu]

@drtrigon forgot to say previously that, considering the stability of PJDL using also unconventional conductors (human body, fruits and vegetables ecc.) for communication, if conductive parts of the house/fixtures that are not grounded are present they could be considered as a potential path for a PJDL bus if the necessary caution is taken to protect the IO pins.

[gioblu]

Ciao @drtrigon about the level shifter, what about this design:

              Schottky diode
          |---[___|_]---|
3v3 IO ---|             |---  5v BUS
          |---\/\/\/\---|
              resistor 

so 3v3 from 3v3 device can pass through the path with less resistance (the diode) and conserve its voltage - the diode voltage drop across its ends (using a schottky the drop can be negligible). In the opposite case the 5v would pass through the resistor and its voltage can be brought down to 3v3 selecting the required resistance value.

[gioblu]

@drtrigon here a simulation with circuitjs:
http://www.falstad.com/circuit/circuitjs.html?cct=$+1+0.000005+12.050203812241895+88+5+43%0Av+112+384+112+256+0+0+40+5+0+0+0.5%0Ad+496+80+112+80+1+0.875%0Aw+112+80+112+144+0%0Aw+496+80+496+144+0%0Ap+496+144+592+144+1+0%0Aw+256+384+496+384+0%0Ap+112+144+0+144+1+0%0Ar+496+144+496+384+0+1000000%0Ar+112+144+496+144+0+510000%0Av+176+384+176+256+0+0+40+3.3+0+0+0.5%0Aw+112+384+176+384+0%0As+256+384+176+384+0+1+false%0Aw+176+256+256+256+0%0Aw+496+144+256+320+0%0As+256+320+256+256+0+0+false%0Ar+64+224+64+352+0+1000000%0Aw+64+352+112+384+0%0As+112+144+112+256+0+1+false%0As+112+144+64+224+0+0+false%0A

Closing the active switches (clicking on them) and activating the others you can simulate the opposite case. It seems it is required to have:

• 2 pull down resistors of 1M
• 1 "voltage reducer" resistor of 510k
• 1 diode with a forward voltage of 0.875mv

[drtrigon]

@gioblu: Thanks for all your answers! I try to answer properly:

Thanks for the info in https://github.com/gioblu/PJON/blob/master/documentation/configuration.md about shared mode - will consider that. However I am a bit confused by "Medium" column in the table; why is it radio for OverSampling but serial port for ThroughLoRa? That's radio too (actually the medium is "air" ;). Then I am also missing the more exotic ones you mentioned "unconventional conductors (human body, fruits and vegetables ecc.)". Because this is one of the main reasons why I love PJON meanwhile. Additionally I would strongly suggest to mention that security measures to protect human body might be needed for several of them (e.g. laserdiodes, etc.).

I used this circuit (somehow inspired by V-USB but is not the same):

3v3 IO  ---o---\/\/\/\---  5v IO
           |   resistor (68...4k7)
          ---              
          | |  3v6 Zener (protect 3v3)
          ---              
           |
          GND

It works but has at least these disadvantages:

• knee of zener known not to be very sharp, so 3v6 are not precise
• a device on the 5v bus needs to be able to drive (5v - 3.6v) / 68 ~ 20mA for a high
• (cut-off frequency might be low due to big capacity)

Because of the 2nd item I plan to setup a "3 way" tunneler (LUDP, SWBB, SWBB) that gives me 2 SWBB pins (on same bus). That way I could put all 5v divices on 1 pin and the 3.3v on a separate, isolating the 2 parts from each other.
Your first circuit looks interesting, but the later post confuses me. So I also setup a simulation http://www.partsim.com/simulator/#148244 and do not understand how it should work. However I did the same with my proposal http://www.partsim.com/simulator/#148247 and there I learned that using a smaller current (bigger resistor, e.g. 4k7) is actually an advantage as it reduces the cut-off voltage. So actually my 2nd concern about drawing to much current is solved.

[drtrigon]

I should may be also mention that I got the system working: LUDP - SWBB - LoRa - SWBB where on the two SWBB buses my devices are connected. I also setup a device that can connect to DS2438 and DS18B20 1wire devices in order to be able to use the old sensors with my new network also. Very nice!

Some issues still remain, e.g. sometimes the first message does somehow get lost or the answer is related to an request done earlier.

[gioblu]

Ciao @drtrigon thank you for reporting the inconsistencies in configuration docs, will follow commits to solve those issues. I agree with you must follow also a remark about safety.

The working principle of the circuit I was suggesting to try is to use a Schottky diode to reduce the amount of voltage loss (if compared to a normal diode) when electrical impulses pass through it making a high pulse from 3v3 side high enough (3/3.1v) for the 5v side to detect it as a 1 also if there is a considerable voltage drop in between. On the other side a single resistor reduces the voltage of the 5v pulses to 3v3 while the Schottky blocks the flow in that direction.

My compliments for having a network that complex in operation already :)

[gioblu]

@drtrigon see this simulation (press RUN to see it live):
http://www.falstad.com/circuit/circuitjs.html?cct=$+1+0.000005+12.050203812241895+88+5+43%0Ad+496+80+112+80+1+0.875%0Aw+112+80+112+144+0%0Aw+496+80+496+144+0%0Ap+496+144+592+144+1+0%0Ap+112+144+0+144+1+0%0Ar+112+144+496+144+0+510000%0Av+496+368+496+240+0+0+40+3.3+0+0+0.5%0Aw+112+368+496+368+0%0Ar+112+192+112+320+0+1000000%0Aw+112+320+112+368+0%0Aw+112+144+112+192+0%0Aw+496+240+496+144+0%0A
Where 3v3 passes through the Schottky and on "5v" side is 3.09v

Here the contrary:
http://www.falstad.com/circuit/circuitjs.html?cct=$+1+0.000005+12.050203812241895+88+5+43%0Av+112+384+112+256+0+0+40+5+0+0+0.5%0Ad+496+80+112+80+1+0.875%0Aw+112+80+112+144+0%0Aw+496+80+496+144+0%0Ap+496+144+592+144+1+0%0Aw+256+384+496+384+0%0Ap+112+144+0+144+1+0%0Ar+496+144+496+384+0+1000000%0Ar+112+144+496+144+0+510000%0Aw+112+384+176+384+0%0Aw+112+144+112+256+0%0Aw+176+384+256+384+0%0A
Where the resistor reduces the 5v impulse to 3v3.

Obviously depending on the drain of the IO pin in real life the optimal resistors value may vary.
And also, that was only a proposed experiment for having a compact and simple way to patch voltage levels making use of the cool properties of the Schottky :)

[drtrigon]

Hmmm, thanks for the simulations I had a look also at mines and replaced the Schottky with an other type, now I can see how this is supposed to work. I have the impression that the resistor needs to have quite the right value as the level (3v3) depends on the divider 1M / (1M + 510k).

My PJON network is very nice - I enjoy playing with it. But it will take some more time to get it working properly as I use solar cells for power supply and thus the used Yun is a bad choice (very power hungry - need to be replaced by an Uno).
I do have minor issues that sometimes the network is not very responsive (through 3 tunnels) what would be the correct way to debug that? Is there a value similar to TS_RESPONSE_TIME_OUT to set? Where would I have to set them in the two end-point devices or in all tunnels in between also?
Does it make sense to setup and run https://github.com/gioblu/PJON/tree/master/examples/ARDUINO/Local/SoftwareBitBang/NetworkAnalysis ?

[drtrigon]

@gioblu: Thanks a lot for the wiki updates and fixes! You added a few nice hints I did not think about yet!

Just tried to use the code from https://github.com/gioblu/PJON/wiki/Error-handling and found that the constant names are not correct anymore:

• CONNECTION_LOST is now PJON_CONNECTION_LOST
• PACKETS_BUFFER_FULL is now PJON_PACKETS_BUFFER_FULL
• CONTENT_TOO_LONG is now PJON_CONTENT_TOO_LONG

[drtrigon]

As mentioned earlier sometimes the first message does somehow get lost or the answer is related to an request done earlier. Other issues like delay resp. not replying on first message also exist mainly on the LoRa tunnel part. The SWBB-TL tunnel reports this:

Packet buffer is full, has now a length of 3
Possible wrong bus configuration!
higher MAX_PACKETS in PJON.h if necessary.
Packet buffer is full, has now a length of 3
Possible wrong bus configuration!
higher MAX_PACKETS in PJON.h if necessary.
Connection with device ID 0 is lost.
Connection with device ID 1 is lost.
Connection with device ID 0 is lost.

(the devices on the bus have ids 42, 44, 45 and 46 - 0 and 1 do not exist)

[gioblu]

Ciao @drtrigon thank you for reporting the outdated wiki pages. I have cleaned up removing all the outdated pages.

Could you describe me more in detail what you are sending and in which section this problem appears?

[gioblu]

@drtrigon about possible serial output related to stuff happening before the reset, it may happen in general if serial data was incoming before reset. Probably data still in one of the buffers. I would say that is an issue of the Arduino IDE.

[drtrigon]

@gioblu: Let me first write down the hardware setup:

linux (0) --[ethernet]-- LUDP/SWBB tunnel (1) --[SWBB (A)]-- SWBB/LoRa tunnel (2) --[LoRa (B)]-- SWBB/LoRa tunnel (3) --[SWBB (C)]-- ...

so that debug output was coming from (2). As I derived all the tunnels from BlinkingRGBSwitch they contain PJON_MAX_PACKETS 3 which I removed now (should increase to default of 5). My "brute-force" solution would be to add a watchdog reset whenever PJON_PACKETS_BUFFER_FULL occurs but that does not feel correct... ;)

Forgot the what:

• linux code sends a single byte (the command)
• devices answers either with string or casted float, e.g. (after unpacking):
• first 1 command to device 44 (on SWBB A), then 7 commands to 42 (on SWBB C)
  (1 line below is equivalent to 1 answer to a command)

owp:dg:v1                        <-- device 44@A
owp:dg:1w:v1                     <-- device 42@C (everything below is 42 also)
38, 0, 0, 215, 1, 0, 0, 77
0, 0, 0, 0, 0, 0, 0, 0
22.625,
0.029999999329447746,
4.510000228881836,
nan,

...but sometimes is screws the returned values (on SWBB C mainly), e.g.:

owp:dg:v1
owp:dg:v1
nan
38, 0, 0, 215, 1, 0, 0, 77
28.3125,
0.05999999865889549,
nan
nan,

not only that there are wrong values (comming from other devices), there are also correct values at the wrong place (e.g. comming 1 call later). Somehow looks like a buffering issue - but I am not sure what would be better for PJON_MAX_PACKETS ... 3 or 5 or switch it off completely with 0?

(Watchdog reset of (2) and (3) helps the whole system to recover faster - within seconds to minutes.)

May be I should also add that my program on the linux side does NOT run continuously. It starts, sends a command (1 byte usually) and waits for an answer, after that it closes. If no answer arrives it re-tries to send. So it is possible that some messages keep hanging in some buffer on the bus. Adding a short period of receiving "old" messages before sending seems to improve the situation. Is there a command to find and /or drop "old" messages from the bus when connecting to it?

[drtrigon]

@gioblu: Am I right in assuming that Connection with device ID 0 is lost. refers to the first entry in

PJONInteractiveRouter<PJONVirtualBusRouter<PJONSwitch>> router(2, (PJONAny*[2])
{
  &bus1, &bus2
});

so this is bus1 and in that case StrategyLink<SoftwareBitBang> link1; so the SWBB part?

Is there any statement possible whether Connection ... lost. is more related to hardware (e.g. level shifter, pull-up/downs, etc.) or software issues (like buffer size, etc.)?

[fredilarsen]

@drtrigon I searched for the "Connection with device ID" text, and found it only in the SendAndReceive examples, not in any router/switch-related code. I therefore assume that the errors are coming from SendAndReceive device1 or device2, not a switch.

I do not see how it can print that message if the SendAndReceive example is trying to send to device id 44 or 45, but I do not think this message is routing related. Can you share the actual sketch writing these messages, assuming it is a modified example?

Are you running Device1(Linux, UDP) <-> LUDP-SWBB Switch <-> 2 SWBB-LORA Switches <-> Device2(Arduino, SWBB)?
If so, can you try to simplify by connecting Device2 directly to the LUDP-SWBB Switch and see if the behavior is the same?

[fredilarsen]

Comments to a couple of questions (quoted):
Another question is do you have an idea/experience how good the performance is when putting 3 tunnels in a row (LUDP-SWWP, SWBB-?, ?-SWBB) ? I mean do you expect it to work at all? ;) I also wondered if it is possible to setup a "3-way tunneler", e.g. LUDP - SWBB - LoRa on a single Arduino Uno?

1. The switches (at least in local mode) start in "hub mode", sending duplicates of each incoming packet to all attached buses except the source bus. When a packet (often a reply) from the target device has been received, the switch will skip duplicating packets because it knows where the device is. This reduces bus traffic and increases speed. So multiple switches in a row is expected to work, slowly at first, but faster as the switches learn which devices are where. Obviously the latency will increase by each switch layer, but speed should not be severely degraded unless sender is waiting for replies.
2. A 3-way (or more) switch is designed to be possible, but not tested. Obviously the maximum packet size and send queue length determines if this is possible with the memory of an Uno. Each PJON object uses (1+PJON_MAX_PACKETS)*PJON_PACKET_MAX_LENGTH bytes of RAM for the buffers. Aside from RAM limitations, the listening time on the SWBB bus will be reduced when polling several buses, so speed will be reduced.

Regarding the use of LoRa and different spreading factors for multiple LoRa switch pairs, given the long range of LoRa, would it not be possible to avoid this by having one SWBB-LoRa switch indoor, and a common LoRa bus for all outdoor SWBB-LoRa switches? This would reduce the number of devices and wireless complexity. @Matheus-Garbelini and @gioblu, what do you think about this?

[gioblu]

CIao @drtrigon sorry for the delay, I suspect you are seeing the effect of the notebook not being able to discharge induced interference not having a direct link to ground. To solve the issue in SWBB bus, reduce the resistivity of the pull-down resistor, or place it if not present.

[drtrigon]

I dont think so (or I don't understand) - the notebook is not connected (physically) to the SWBB part. The notebook is connected via wifi only. I really think it is a timing issue related to udp socket respective ethernet hardware. When I remove the code block to make it "non-blocking" essentially all BUSY and connection lost are gone. So to do it properly I increased #define LUDP_RESPONSE_TIMEOUT (uint32_t) 1000000 and that helps a lot. Additionally to get rid of possible old (hanging) messages, I added a

bus.update();
bus.receive(1000);

before bus.send. And the last point was to have a more or less free lora frequency (with few blocked times). Forget about my other speculations the last few posts. It is still not fully free of hickups and misstakes, but it is A LOT better now.

Question: Does LUDP_RESPONSE_TIMEOUT also influence the LUDP-SWBB switches? I don't have the impression but I am wondering whether I have to recompile and update the code/firmware in the switch also?

[drtrigon]

I did a very naiive test using ping to my router, here the results:

battery powered:

$ ping -c 10 192.168.11.1
PING 192.168.11.1 (192.168.11.1) 56(84) bytes of data.
64 bytes from 192.168.11.1: icmp_seq=1 ttl=64 time=1.35 ms
64 bytes from 192.168.11.1: icmp_seq=2 ttl=64 time=2.31 ms
64 bytes from 192.168.11.1: icmp_seq=3 ttl=64 time=1.35 ms
64 bytes from 192.168.11.1: icmp_seq=4 ttl=64 time=1.84 ms
64 bytes from 192.168.11.1: icmp_seq=5 ttl=64 time=1.30 ms
64 bytes from 192.168.11.1: icmp_seq=6 ttl=64 time=2.28 ms
64 bytes from 192.168.11.1: icmp_seq=7 ttl=64 time=1.18 ms
64 bytes from 192.168.11.1: icmp_seq=8 ttl=64 time=1.36 ms
64 bytes from 192.168.11.1: icmp_seq=9 ttl=64 time=1.40 ms
64 bytes from 192.168.11.1: icmp_seq=10 ttl=64 time=1.13 ms

--- 192.168.11.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9014ms
rtt min/avg/max/mdev = 1.130/1.555/2.312/0.413 ms

average ping time: 1.555 ms / minimal ping time: 1.130 ms

from power supply:

$ ping -c 10 192.168.11.1
PING 192.168.11.1 (192.168.11.1) 56(84) bytes of data.
64 bytes from 192.168.11.1: icmp_seq=1 ttl=64 time=1.02 ms
64 bytes from 192.168.11.1: icmp_seq=2 ttl=64 time=0.858 ms
64 bytes from 192.168.11.1: icmp_seq=3 ttl=64 time=1.48 ms
64 bytes from 192.168.11.1: icmp_seq=4 ttl=64 time=0.901 ms
64 bytes from 192.168.11.1: icmp_seq=5 ttl=64 time=1.03 ms
64 bytes from 192.168.11.1: icmp_seq=6 ttl=64 time=1.24 ms
64 bytes from 192.168.11.1: icmp_seq=7 ttl=64 time=0.908 ms
64 bytes from 192.168.11.1: icmp_seq=8 ttl=64 time=0.973 ms
64 bytes from 192.168.11.1: icmp_seq=9 ttl=64 time=1.25 ms
64 bytes from 192.168.11.1: icmp_seq=10 ttl=64 time=1.56 ms

--- 192.168.11.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9007ms
rtt min/avg/max/mdev = 0.858/1.124/1.565/0.239 ms

average ping time: 1.124 ms / minimal ping time: 0.858 ms

As you can see on battery the ping time is always >1ms. As it seems this is quite relevant for my setup and even more for the devices behind the lora channels/3 switches. This would really explain a lot of the stuff I have seen and reported here. It is not 1 single cause but a combination of:

• network delay/latency (especially when on battery powered wifi)
• a lot of switches in row (here 3)
• possibility of blocked lora channels (increasing delays)

all that leads to lot of repetitions on the bus and that again to BUSY or even connection lost (and these to completely messed up returned values in my case).

Here some info that might be relevant: https://www.ctrl.blog/entry/linux-wifi-dpm-latency and https://www.howtogeek.com/217463/wi-fi-vs.-ethernet-how-much-better-is-a-wired-connection/

Also switches on ethernet will increase the latency in order of ms, see https://www.edn.com/Pdf/ViewPdf?contentItemId=4135948.

[fredilarsen]

Now that you have LUDP and SWBB buses under control, I think I would thave tried an alternative to LoRa just to have tested alternatives. What about HC12 or the very simple ASK modules? Reduced sensitivity may reduce problems?

In my earliest setups I used a Nordic Semiconductors NRF24 transceiver which worked well, except for strange temporary periods of a few hours with no communication. I never found a reason for these, as they came and passed with no action from my side. Most probably external interference, potentially from a neighbor. Later I was lazy and dug down a long wire instead of solving the wireless problem :-)

[drtrigon]

@fredilarsen: I agree I was thinking about alternatives, 2 things came to my mind:

• (as you mentioned yourself) another kind of radio has similar issues - external interferences - I guess that the probability is lower because of the range - but basically the issues stay the same
• did not find a good supplier for a good module (I did look for the HC-12 when you first mention them)
• (may be not a reason but a note; I like LoRa for beeing low power and having the possibly for long range)

The reason why I did not continue this route (for now):

• as I can see PJON already handles packet collisions / blocked channels quite well - what I would love to see is something like a "SLOW" or "ROBUST" mode with lower baud rates and bigger timeout for rough environments? e.g. like "WSJT's communication modes can be divided into fast and slow modes" or the Dallas/Maxim 1wire that has an overdrive mode.
• about alternatives to LoRa; has anybody experience in capacitive coupling between 2 wires sticked with tape to a window (crosstalk)? any idea?

[fredilarsen]

Then there is the alternative already mentioned by @gioblu, using AS with normal or IR LEDs. Just putting a couple of LEDs face to face should do it. Low power usage and no chance of anybody picking up or interfering with the traffic.

Perhaps @gioblu has done experiments with capacitve coupling, But I suppose the wires will act as antennas and be more susceptible to electromagnetic interference, just like the radios.

It is on my agenda to try out the HC12 and LoRas at some point. I tried the green very cheap ASK modules without much luck, then bought some blue ones that should work better, but in the meantime I have gone wired, so wireless lost priority.

[drtrigon]

The AnalogSampling ... I could talk for days about it (I fell in love with it when I first read https://www.researchgate.net/publication/221568729_Very_Low-Cost_Sensing_and_Communication_Using_Bidirectional_LEDs?_sg=r-MhohoC0ihWf5g5jnBHao8JNqV8qDvtWdMB82RV-eAN4Q2GThLUM4gKbKWp-wSdY0nTkxuhig years ago and sweared to use it myself someday ;) ... the points holding me back currently are:

• low speed (0.640-1.530 kB/s)
• interferences from nearby ir remote controls - may be even from intense sunlight (so again the same issue of external interferences - if PJON would have an (optional) mode that is safe against it that would be cool and I think also a valuable feature)
• no possibility to include sensors like e.g. ID 43 Local bus going through walls / outdoors #222 (comment) directly

... nevertheless I am currently looking into ordering some ir leds ;) ... any recommendation?

I configured my outdoor network now to be the productive one even though it does not work perfectly yet. That way I can easily get more experience with it, test some parameters and last but not least the error and recovery behaviour of PJON is in any case better than that of owfs. Meanwhile I had 2 freezes of the SWBB-lora switches and am considering to use an 8s watchdog there.

@fredilarsen: Any update regarding FIFO LUDP-StdIO or wireless setup are very welcome and will make me very happy! So I look forward to hear from you! ;)

[gioblu]

Ciao @drtrigon that paper is a bible I agree, although it does not describe the AnalogSampling procedure. They apply there the reverse bias technique (charge inversely the LED, sample voltage discharge time that is proportional to incident light) also popularized by the Arduino LED sensor example:
https://playground.arduino.cc/Learning/LEDSensor

Years of experiments have practically determined that for communication is better to use forward bias, instead for detecting distances, without a signal amplification stage, it may be more sensitive reverse bias, at the cost of a lower sampling frequency.

I have chosen to use LEDs as photodiodes in forward bias (or as photo-cells/solar cells) to be able to potentially obtain the highest baudrate and in the worst case need an op-amp.

The biggest difference is that we use a passive analog reading (from where comes the name AnalogSampling) instead of an active procedure (reverse biasing the LED then swap the pin state and determine the duration of a digital impulse).

• The forward bias option requires just 1 ADC and a ground connection
• The reverse bias option requires 2 digital IO

For the reasons described above AnalogSampling was designed as it is, and I would say that the paper describes a less "direct" and more convoluted way to obtain the same result (the discharge time it is strictly related to and is direct consequence of the photo-voltaic effect AS uses in the first place).

See this issue here: #228

[gioblu]

Ciao @drtrigon sorry, about LED suggestions, use 5mm infrared LEDs, this:

is L-53SF4C https://www.rapidonline.com/pdf/55-9204_v1.pdf it works nicely, tested up to mode 3

or you can buy a bunch and evaluate their performance, see how to do that here:
https://github.com/gioblu/PJON/tree/master/src/strategies/AnalogSampling

[gioblu]

@drtrigon

Meanwhile I had 2 freezes of the SWBB-lora switches and am considering to use an 8s watchdog there.

this is bad, probably third-party lora library, should be investigated further, looking into it.

Thank you very much for reporting this :)

[fredilarsen]

@drtrigon I see there are some unanswered questions in your posts above, relating to LUDP.

First, if the PC goes on battery, you may still change the WiFi behavior in power savings settings, and ask the network card to stay on high performance while on battery. Probably it just wakes up now and then when on max battery saving.

The LUDP_RESPONSE_TIMEOUT is used also in a switch. We should change this timeout to be overridable in the strategy. Have you done this, or should I do it?

As a third comment , I have been struggling with UDP based networking on the W5100 based cards. It seems there is a bug in the W5100 chip that makes it lock up at random times when it tries to send when a reception starts. I am experimenting to see if it can be compensated for by changes to the Arduino network library. Otherwise we will have to use the watchdog timer to do a reset when it locks up. In my setup a lockup appears between 6 hours to 10 days after startup. Incredible to see such behavior in a product sold in millions.
arduino-libraries/Ethernet#17

[drtrigon]

The LUDP_RESPONSE_TIMEOUT is used also in a switch. We should change this timeout to be overridable in the strategy. Have you done this, or should I do it?

No I did not do it, I just did this: https://github.com/drtrigon/sketchbook/blob/master/libraries/PJON-master/src/strategies/LocalUDP/LocalUDP.h#L32
I think a simple #ifndef before that should do it, but as I was not sure on how proper this was to tackle my issue I did it as draft to test. Would be great if you could do it (as you have commit access to the repo)!

Regarding the UDP switch issue, I am currently experimenting a bit with watchdog on my SWBB-lora switches: https://github.com/drtrigon/sketchbook/blob/master/OWPJON/ARDUINO/Local/SoftwareBitBang/Tunneler/BlinkingSwitch_SWBB-TL/BlinkingSwitch_SWBB-TL.ino#L179
The watchdog jumps in in 2 different situations:

• either if the execution freezes completely and loop() does not get executed anymore
• or if the last time sendnotification_function(..) was called is more than 3mins ago, as I poll my devices once per minute

The W5100 chip lock up; what is the effect of this? A freeze or a delay in packet processing? I suffer more from freezes in the SWBB-lora switch. May be my system was just not running stable and long enough to uncover the UDP switch flaws yet.

[fredilarsen]

The UDP bug in the W5100 makes the while loop in the sendUDP function in socket.cpp in the Arduino Ethernet library loop forever when it happens. So the switch is just hanging on this function forever.

It happens when an incoming packet arrives while sending, so the status is set to RECV instead of SEND_OK, and it waits forever for sending to complete. So I guess it will not happen in a strict master-slave based setup where a packet will not be received while trying to send. But I have two master-slave setups running at the same time on the same LUDP bus for testing, and also my ModuleInterface library supports exceptions from the master-slave principle, allowing in my case a movement sensor to be broadcast immediately without waiting for the regular request from master. This will heighten the probability of this happening as well.

I am testing modifications to the Ethernet library, but with days of waiting between each attempt it is slow going. Currently trying to detect the eternal loop with a loop counter, closing the socket in 2 layers, and trying to open a new one and continue. Hoping the network shield and the Arduino does not actually need to be rebooted.

I have not tried the watchdog timer yet, to see if the reboot also reboots the network shield enough to get it working again. This is the least preferred solution for me.
I also ordered a couple of W5500 to see if they are bug-free.

[mrfawy]

Hi @fredilarsen ,

You mentioned ..

In my earliest setups I used a Nordic Semiconductors NRF24 transceiver which worked well

May I ask how you were able to do that with PJON? .. my understanding is that ThroughSerial accepts a serial (or software serial ) which is OK for HC-12, but NRFs has SPI interface ..

apologies if the question is naive .. I'm still a newbie who just discovered PJON! .. Thanks a lot

[drtrigon]

@mrfawy: I guess he is referring to "OverSampling" strategy.

@gioblu: Just came across "If using OverSampling, the asynchronous acknowledgement is suggested as acknowledgement mechanism because includes in the packet's meta-info a packet id, avoiding duplicated receptions." Can you elaborate whether this is useful for lora too? (You recommended to me to use it for ThroughSerial some time back...) Is this id not included by default? Is it also compatible wit attiny85 (code size etc)?

[fredilarsen]

@mrfawy I used the NRF24 before I switched to PJON. I used a more primitive serialization and routing at that point. Surely a PJON Strategy could be made for the NRF, but it would have to be on top of SPI.

@drtrigon There is a PJON_INCLUDE_ASYNC_ACK preprocessor definition that must be set to include support for async ack. It will allocate memory to keep a list of recent packets, but you can reduce the size of this list by setting PJON_MAX_RECENT_PACKET_IDS (for example to 2 instead of 10). Try setting these defines and see if it fits along with your own code on a tiny. It may get crowded.

[gioblu]

Ciao @drtrigon if you are looking for packet uniqueness, and so avoid the duplicate reception of packets you could include just the packet id see docs here: https://github.com/gioblu/PJON/blob/master/documentation/configuration.md#extended-configuration

I would suggest you to start from adding the packet id, the asynchronous acknowledgment can be useful to have transmission certainty in a multi-hop path

[mrfawy]

@fredilarsen @drtrigon thanks a lot for clarification... yeah I guess that would be great to implement an over sampling strategy for NRF24s over SPI ..

[drtrigon]

@gioblu: Quick test on tiny85 shows about +500 Bytes to end up with about 6 kBytes (70-75% of space) this is not to much of free space for other code but works theoretically. Asynchronous acknowledgment adds another about 100 Bytes.

[gioblu]

Ciao @drtrigon thank you very much for the test,
you can reduce the memory footprint using optional defines see:
https://github.com/gioblu/PJON/wiki/ATtiny-interfacing

#define PJON_MAX_PACKETS 0 // Spare memory with an empty packet buffer
#define PJON_PACKET_MAX_LENGTH 25 // Maximum packet length of 25B - overhead

Avoid using the packet handler, or the use of update or send functions, use only send_packet or send_packet_blocking along with receive instead. Consider defining PJON_MAX_PACKETS with a value of 0 to free memory from the unused packet buffer and define PJON_PACKET_MAX_LENGTH with a value fitting your application requirements.

The LEDAR program:
https://github.com/gioblu/PJON/blob/master/devices/sensors/LEDAR/software/LEDAR/LEDAR.ino
that contains proximity detection, configuration exchange and EEPROM handling compiles like this:

Sketch uses 5622 bytes (68%) of program storage space. Maximum is 8192 bytes.
Global variables use 161 bytes of dynamic memory.

Async ack does not work on ATtiny (because the packet handler requires too much ram).

[drtrigon]

@gioblu: Here is the code I use https://github.com/drtrigon/sketchbook/blob/master/OWPJON/ATTINY/Local/SoftwareBitBang/DeviceGeneric/DeviceGeneric.ino#L56 ... Do I assume correct that I cannot mix async and sync ack? So if I want to use tiny85 on my bus I cannot use async ack at all?

Async ack is not supported for ATtiny (because the packet handler requires too much ram). For that reason is optionally activated via define.

What does that mean? Even though I activate it it will not work? Or does it mean it is not activated by default?

Going back to my very initial question ... from my side we can clean up this issue as I got a lot of advices regarding on how to setup a solution for an isolated connection going through a wall or a window - thanks! Other issues raised here that need further work:

• updates to lora strategy, eg. timing etc.
• examinating lora third-party library for issues causing a freeze of the tunnel or other issues
• investigate on general (LUDP) network latency issues
• investigate on how many packets per second is passing through the switch in each direction in normal operation
• have a StdIO strategy so that we could have a ready-made LUDP-StdIO switch for Linux/Windows/RPI/Mac
• LUDP_RESPONSE_TIMEOUT change this timeout to be overridable in the strategy
• "SLOW" or "ROBUST" mode with lower baud rates and bigger timeout for rough environments (e.g. like "WSJT's communication modes can be divided into fast and slow modes")
• have a lora sniffer to check link quality, eg. https://github.com/drtrigon/sketchbook/blob/master/Uno_Serial_LoRa_Sniffer/Uno_Serial_LoRa_Sniffer.ino
• wiki page about wireless/isolated setups in general

did I forget anything...?

[gioblu]

Beautiful, thank you very much @drtrigon for your valuable and passionate support to the project :)
I will quickly eat something and will give feedback about the points you have exposed in the checklist 👍

[gioblu]

Ciao @drtrigon :

• LoRa timing can be optimized, I agree, @Matheus-Garbelini may look into that when he has free time
• LoRa library hangs on which device? Also this could be useful info for @Matheus-Garbelini
• I think @fredilarsen have already worked out the stdIO strategy if I am not mistaken :)
• I agree that slower modes must be provided, in the last 2 days I have worked out a new single wire protocol that is slower but designed to be more reliable and less susceptible to timing differences if compared to PJDL, that may be the solution to this point
• I Will work out as soon as possible the wireless/isolated wiki page

[drtrigon]

LoRa library hangs on which device? Also this could be useful info for @Matheus-Garbelini

Most hints I have so far indicate that it seems to be Sw B more often than Sw C. May be also a send receive clash when a alien lora transmission comes in? Don't know yet...

I need #242 for more debug info - I need to find the proper condition for the watchdog resp. more details on what happens; which bus stops sending? what error does get triggered? does it freeze completely? etc...