v3.0.0b3 - WebUI does not respond anymore

[tp1de]

I upgraded from v3.0.0b1 to v3.0.0b3 yesterday.
Now after approx. one day of operation the webui does not respond anymore. (empty white page - no error displayed)
MQTT is still working as well as the telnet session.
The command interface took approx. 5 secs to respond - that's not the normal bevavior:

ems-esp:/system$ show
Uptime: 000+22:40:46.701
SDK version: v3.3.4-432-g7a85334d8
CPU frequency: 240 MHz
Free heap: 94296 bytes

WiFi: Connected
SSID: xxxxxx
BSSID: DC:39:6F:BF:DC:ED
RSSI: -79 dBm (42 %)
MAC address: F0:08:D1:D7:E7:70
Hostname: ems-esp
IPv4 address: 192.168.178.71/255.255.255.0
IPv4 gateway: 192.168.178.1
IPv4 nameserver: 192.168.178.1

Ethernet: disconnected

Syslog: disabled

[tp1de]

After a restart from command interface (su) the webui is available again ...

[proddy]

I'll monitor and keep an eye out. Next time could you capture the # available memory? This is the usual culprit when things start getting slow. Perhaps there is a memory leak somewhere. Also did you have this problem with earlier builds?

[tp1de]

Next time could you capture the # available memory?

Yes I will do, but how shall I capture the # available memory? It's not Free heap .... or?
I had this behavior some 2-3 months ago before using ESP32.

[proddy]

yes it's free heap. grab it either from the MQTT hearbeat or via the webUI

[tp1de]

see above ... Free heap was during webui freeze: 94296 bytes - after restart (now) 95.076 bytes

[proddy]

oops, missed that. I'll monitor here. very strange

[tp1de]

My webui started to slow down again .... > 10 secs to respond and no memory problems.
I would like to build based on latest source code for esp32 by myself.
If I download sourcode zip-file it is based on esp8266 and not esp32.

Could you please provide the correct download link ..... Thanks

[proddy]

take the branch called esp32_dev

[tp1de]

take the branch called esp32_dev

I have done so.
But by just selecting code - download - zip it looks for me like that I am getting the esp8266 version .
Or is this a wrong interpretation ...

[proddy]

weird, if you select the branch from the pull-down list and download the zip it will take the branch,

[tp1de]

@proddy
Hi, I might have an idea what happened with the webui not beeing accessible.
I just tried to connect to webui without success. Telnet console was availabe but quite slow. Mqtt was working.

I then the looked at my mesh network and recognized that the ems-esp was now connected to my main router quite fare away with very poor signal and speed. When I restarted the gateway connected to the next mesh accesspoint (in 50cm distance) and everything works fine. Could it be that after a while wlan connection is somehow changed? I understood that there has been changes on network access since v3.xx.
(This is new ... I never had this before and the mesh network hasn't changed)

[proddy]

that would explain it what you're seeing. I see from the first post the signal strength was 42% which is quite low.

The ESP will automatically reconnect the WiFi if it loses a connection. The best way to spot this is to use Syslog and scan the log files for the re-connect messages.

Another way is to look at the MQTT Heartbeat for the rssi (wifi signal strength) and see if this is dropping. For example, Home Assistant will keep a history or alternatively push this information into Grafana.

In later v3 versions, I added Ethernet code, but that shouldn't affect the Wifi behavior.

I don't know why your network is reconnecting and sending the ESP to another router node. I've had similar problems at home with the ESP32 which is literally 1 meter from my Unifi AC Pro wireless Access Point and EMS-ESP would still always connect to one of my other Access Points on another floor in my house. I could never figure out why. I guess I'll google it...

[tp1de]

In v3.0.0b1 I haven't had this reconnection issue and in the v2.xx versions neither.
SInce I installed the b3 bin-file this reconnection after approx. 12 hours started to happen.

Any changes done on wlan since b1?
Anyhow I will track the rssi in my database from now on - actually it is 100.

[proddy]

The biggest change I could think of is the ESP32 libraries that got upgraded to 3.1.0 and uses the latest Arduino core libraries. That was pushed 2 weeks ago. See https://github.com/platformio/platform-espressif32/releases/tag/v3.1.0. You could try downgrading? Replace with platform = espressif32@3.0.0 in the platformio.ini for example

[tp1de]

Since I still can't manage to download the right source code for esp32 v3.0.0.b3 I took my own compiled b1 from 16.2.2021.
I recognized:

1. I can't manage for the moment that the esp-ems connect to the local mesh accesspoint - b1 / b3 connects always to the main cable router quite far away.
2. ... but b1 signal quality is better then with b3 (20 Mbit vs. 1-2 Mbit) and rssi now between 50 and 60 compared to 34-42 before.
3. The networkscan in Webui shows in my cases the different accesspoints with same ssid and different signal strength. It does not work to select an entry with the highest signal strength.
4. When I restartet for the first time (b3) it got connected to local accespoint with rssi 100. After 3-4 minutes it reconnected to main router with rssi between 34-44.
5. In the graph the last part is the b1 version
   

[tp1de]

I changed my WLAN Mesh configuration. Now I am using the same wlan channel for a ssid. Before I had automatic channel search activated and the access points selected different channels for the same ssid depending on location.

Now with fixed (same) wlan channel for all accesspoints it seems to work. My raspberry Pi's had no problem selecting the right (strongest) accesspoint, but it seems to gave problems with the esp32 when wlan channels where different for same ssid.
For the moment it looks like working .... but I will observe.

[tp1de]

I just installed then bin from v3.0.0b5 and I got ems-esp connecting again to the main router with low wlan signal level ...

[proddy]

a good test would be to compile 3.0.0b5 yourself with platform = espressif32@3.0.0 and then platform = espressif32 and comparing the results to see if the problem really is with the newer ESP32 Arduino 1.0.5 library

[proddy]

if you add the -DEMSESP_WIFI_TWEAK to your pio.local.ini it will change the WiFi Tx signal strength and also switch off the modem sleep mode (which stops listening to beacons). This may help with connecting to mesh nodes.

[tp1de]

I would like to test. But still I can't download the latest ESP32 dev branch source code from github.
@proddy Could you please advice how to do it.
I still get ESP8266 assets (#729) for latest v3

[proddy]

I spent like 3hrs yesterday trying to fix emsesp/EMS-ESP#729. For some reason GitHub is using the wrong branch for the source code assets.

How are you building the firmware, using VSC? If so make sure you have git installed and clone the repository, then choose heads/esp32_dev branch. If you're using the command line just use the git CLI to pull the remote branch

[tp1de]

Sorry but I am not an IT-guy but rather a retired electrical engineer .... with home automization as a hobby.
So I am not too literate with Github ... sorry.

Yes I am using VSC but just the easy way: 1. Dowload zip-file with source, expand into local directory and open the folder structure with VSC .... then doing some test like with @MichaelDvP and compiling and testing the new binary.

I do have Git installed but up to yet I have not understood how to use it.
Could you explain what steps I have to do?

[proddy]

Could you explain what steps I have to do?

sure, no problem. I'll write down some steps tonight. Are you Windows, Linux or OSX?

[tp1de]

WIN10 - Thanks ....

[proddy]

ok, if you VSC and Git installed, open VSC, click F1, type Git:Clone, type in https://github.com/proddy/EMS-ESP.git. Choose a directory and it should download the source code (it may ask you for a username/password). Then in VSC pick the heads/esp32_dev branch from the bottom left like:

[MichaelDvP]

For some reason GitHub is using the wrong branch for the source code assets.

It's the same if i push to my github. But if i rename the branch, the source assets are ok.

[tp1de]

@proddy
Thanks for your help .... I just needed to start command interface manually (F1 was not working) .
This was quite easy

[tp1de]

Just a question before starting testing:

I would like to avoid re-entering wlan and other settings.
In platformio.ini there are lines:
extra_configs =
factory_settings.ini
pio_local.ini

Can I take the factory_settings.ini out to keep my settings when uploading the bin-file after compilation?

[proddy]

you can just keep the platformio.ini as it is. The settings are stored in the ESP's filesystem and are persisted so when you re-flash it's only updating the application code.

[tp1de]

@proddy
Are you sure with v3?
Up to yet I needed to set-up manually.
to v3.0.0b1 was clear for me since new filesystem coming from v2
to v3.0.0b3 again with factory defaults
to v3.0.0b5 again with factory defaults

[proddy]

yes, absolutely. I do many updates each day. Have you changed any settings in platformio? Are you using VSC to upload or via the command line?

[tp1de]

No, up to yet I used the bin-files from github. Just b1 I compiled myself.
Uploads I do by webui from ems-esp by selecting the compiled binary file.

[tp1de]

@proddy yes you where right.
The compiled new version from yesterday kept the settings and only changed code. (like in v2 before)

I restarted all components on my Mesh Network yesterday (Router and all Mesh Repeaters) and since then the ems-esp connects to the nearest Mesh Repeaters (rssi 100).
I will observe if this is stable and not doing any of the proposed changes on wifi until I see any reconnection to the main router again..

[tp1de]

@proddy I think we can close this issue.

[proddy]

before we do, did you change anything? Like use the -D compiler build flag?

[tp1de]

before we do, did you change anything? Like use the -D compiler build flag?

I just selected the build option for esp32