[ESP32] Request for option to deinitialize Wi-Fi

[beckerzito]

Build environment: macOS, Windows, or Linux
Moddable SDK version: OS220805b
Target device: ESP32 platforms

Description
Whenever the WiFi module is initialized, the power consumption increases a lot. Currently, the implementation makes any WiFi API to call the initWiFi function, which initializes the WiFi module. (e.g.: Disconnect, Scan, set mode, get mode, Connect ).
Having said that, it's possible to conclude that if the firmware connects to the router and at some moment it forces a disconnection, the power consumption doesn't return to the original state, but instead keeps very high.

Example to Reproduce

import WiFi from "wifi";
import Timer from "timer";

WiFi.mode = 1; // station mode

Timer.set(() => {
	WiFi.disconnect();
}, 10000);

Steps to Reproduce

1. Build and install the app using this build command: mcconfig -d -m
2. Measure power consumption before the WiFi.disconnect() is called (before 10s)
3. Measure power consumption after the WiFi.disconnect() is called (after 10s)

Expected behavior
The power consumption shouldn't change when calling any API that doesn't actively use WiFi communication, like Scan or Connect.

Images
Baseline power consumption (before WiFi module is initialized) - around 196mW

After WiFi module is initialized - around 565mW

Other information
With some research, it seems that what triggers the high power consumption is the API esp_wifi_start(). (espressif/esp-idf#8324).
This API is currently used in the WiFi implementation of the function initWiFi. That means when any API is called, the WiFi module is kept in a "STARTED" state, in which the station is enabled and keeps consuming.

According to Espressif documentation, the API esp_wifi_stop should be called to turn off the station and reduce the consumption (https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_wifi.html#_CPPv413esp_wifi_stopv)

Besides that, currently, the WiFi destructor doesn't implement a proper deinitialization phase, which is described here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/wifi.html#wi-fi-deinit-phase

I did some tests by applying the following patch patch.diff.zip, just improving the wifi states during initialization, and forcing the stop API to be called on the disconnect. I could see that after the disconnection, the power consumption decreased significantly... It didn't go back to the original levels because I didn't implement the entire DEINIT strategy, even though I think we should!

Maybe we have to implement two different things:

• A way to stop the station. Not sure if after each "disconnect", or a specific API to stop it.
• A way to properly deinitialize the module on the destructor.

[phoddie]

Thank you for the report.

Always shutting down Wi-Fi on disconnect doesn't seem correct. There are valid scenarios where the project disconnects from the current access point but still continues to use Wi-Fi in some way (scan, connect, etc).

I think I would prefer an explicit way to "turn off Wi-Fi" instead. That happens to more closely matches the underlying APIs in the ESP-IDF.

[beckerzito]

Hi @phoddie I agree. Makes total sense.
Also, I don't think that we should just have the "stop" but also a proper way to destruct the wifi, by calling also the wifi_deinit. That might be also deactivating some clock or other peripheral that also consumes power.

The only thing is that, for the STOP... would we create a new API on the wifi.js? What about the other platforms, how would they implement that?

Thanks!

[phoddie]

@beckerzito – We seem to be using different words to talk about the same thing?

• "Turn off Wi-Fi"
• "stop"
• "destruct the wifi"

Is there some subtle difference between these or are they the same?

The only thing is that, for the STOP... would we create a new API on the wifi.js? What about the other platforms, how would they implement that?

Yes, introducing a new behavior will require either a new API or an enhancement to an existing API. I'm not sure yet. It could be as simple as WiFi.deinitialize() or it could be a new mode wifi.mode = (deinitialize. Or something else.

Each platform would implement the behavior as best they can. That's always the case. ;) In some cases, that might be to do nothing. The advantage of an approach like WiFi.deinitialize() is that the calling code can check to see if it is present, to know if it is supported. Platforms that don't support it would simply not provide the function at all.

[beckerzito]

I was using two different words "stop" and "destruct" to refer to the two different APIs we have from Espressif:

"stop" = "turn off WiFi" that would just call "esp_wifi_stop".
"destruct the wifi" that would call "esp_wifi_stop and esp_wifi_deinit"

But just rethinking, I'm not sure if there is a use case to just "stop" the wifi. If it's needed to turn off the wifi, then let's destroy it all.

I think your approach makes sense.

[phoddie]

Thanks for the clarification. I missed the distinction between stop and deinit. For current purposes, that doesn't seem necessary to raise to the JavaScript API level.

[tve]

Whichever way you decide, please keep in mind that there is also the AP mode whose state is completely independent of the STA state (obviously they use the same HW). There is already some unfortunate coupling in the API (when starting AP mode one has to specify whether STA should be on/off),
Finally, there are several power saving states to consider in the bigger picture, although they are probably independent of power on/off control.

[phoddie]

AP and STA mode both depend on Wi-Fi being active. A static method on the WiFi class (default export of module "wifi") would be independent of the operating mode.

There is already some unfortunate coupling in the API (when starting AP mode one has to specify whether STA should be on/off),

That's as much a reflection of the underlying host APIs as anything, if I recall correctly.

Finally, there are several power saving states to consider in the bigger picture, although they are probably independent of power on/off control.

Yes, seems distinct. FWIW – when I've experimented with these in the past, it was not clear that they did much. Maybe that has improved. These also tend to be pretty specific to a silicon vendor, so they may not be appropriate for a portable API whereas completely disabling Wi-Fi should be pretty general.

[tve]

The low power modes actually do a lot but they don't instantly turn the esp32 into a battery-friendly device. I did spend quite some work understanding the details a few years back (https://blog.voneicken.com/projects/low-power-wifi-intro/), haven't looked into it recently...

[phoddie]

Great stuff. Thanks for sharing. To support power modes on Wi-Fi. I'd definitely start with APIs specific to ESP32. I suppose they could be patched onto WiFi but they shouldn't try to be device independent.

[beckerzito]

Thanks for the support team!

@phoddie Just to align the expectation. Are you going to implement this proposal to the module?
Thanks

[phoddie]

@beckerzito – yes, we'll try implementing.

[phoddie]

@beckerzito – I did some experimentation here today. The API I started with is to make this another operating mode. That fits reasonably well. It looks like this:

WiFi.mode = WiFi.Mode.off;

The WiFi.Mode "enum" is new. It is nicer than using numbers. And it provides a way to do feature detection since a device that doesn't support "off" mode, for example, could just leave out that constant from the enum.

That's all straightforward. The deinitializaion process is more complicated.

The Espressif docs and examples aren't entirely consistent about reinitialization. I have something that seems to work. But, it is imperfect.

1. A test that just enables and disables Wi-Fi -- never making a connection -- works perfectly.
2. A test that enables Wi-Fi, connects to an AP, and then disables Wi-Fi -- again, never making a connection - mostly works perfectly. It seems to leak small amount a memory on each init/deinit pair. The memory lost is inconsistent, and on some cycles it actually increases. So, maybe not a big deal for real world uses, but not optimal.
3. A test that enables Wi-Fi, Connects to an AP, makes an HTTP request, waits for the request to complete, and then disables Wi-Fi - generally leaks a bit more memory. But, worse, if the HTTP socket is open when deinit happens, the system will crash.The problem seems to be that deinit causes lwip to deallocate the socket. But, it doesn't let the sockets know about that, so when they eventually try to deallocate themselves, it is a double dispose which crashes.

Of course, (3) is the closest to the real-world. Requiring the app to close all sockets (or crash!) seems bad. We could track the number of active sockets (instrumentation already does) and throw if deinit is called when that is non-zero.

Anyway... going to take a break from this and take a look in another day or two with fresh eyes,

[tve]

@phoddie : are you seeing the same issue if you set mode=0, i.e. just turn wifi off instead of deinit? Maybe you have to first turn it off and then deinit?

[phoddie]

Interesting thought. That moves the crash around, but ultimately things still go bad. I also tried setting to mode 0 and then waiting a few hundred milliseconds before deiniitializing.

I'm reasonably confident I can implement the correct behavior in our runtime. The problem is that the correct is not clear.... teasing that out of the ESP-IDF seems challenging. The docs seem straightforward but reality is more complex.

[tve]

(I had to read up on the esp-idf docs...)

• Are you calling esp_wifi_stop() before esp_wifi_deinit()?
• Do you have a wifi notifier running and getting a disconnect event when you call esp_wifi_stop?
• What I find the most confusing is guessing what the interaction between esp_wifi_start/esp_wifi_init/... and esp_wifi_set_mode is. The latter can be called before and after esp_wifi_start and it must do different things...

Edit: Oh, great, esp_wifi_set_mode is in a binary blob, so can't easily peek at what they're doing...

[phoddie]

(I had to read up on the esp-idf docs...)

Fun times.

Are you calling esp_wifi_stop() before esp_wifi_deinit()?

Yes.

Do you have a wifi notifier running and getting a disconnect event when you call esp_wifi_stop?

Yes. But, I unregister that too. If it is unregistered before calling stop, then obviously it doesn't get called.

Maybe I should wait to be notified before proceeding with deinit?

So much guesswork....

[tve]

Long thread which may or may not be useful, it's for esp-idf v4.4: espressif/esp-idf#8702
Has a link to an interesting gist: https://gist.github.com/david-cermak/612487bc96df8a4dd56610d97b20dc3d
I can't quite tell whether the gist over-complicates things or not.
But maybe some of this provides pointers to something useful, like the calls to remove netif stuff. But is that really necessary? Ugh.

[phoddie]

Thanks for the references. The thread and gist are very consistent with my experience. The calls used for initialization and deinitialization are basically the same and in the same order. I suppose that's reassuring.

Where I get into trouble is where sockets are involved. The discussion is pretty clear that those should be shut down before deinitializing the lower networking layers that they depend on. I did that, but it doesn't seem to be enough. The example in the gist does use a socket for SNTP - though it is UDP, not TCP. And, curiously, the gist example waits until after deinitialization is complete to free SNTP's UDP socket

I'll run some more tests based on the gist, to see if something makes a difference.

[phoddie]

It looks like it doesn't crash if a UDP socket is left open over deinitialization. Definitely not the case for TCP sockets though.

If the TCP sockets are closed, it doesn't seem to crash. Memory use is not perfectly stable, but it looks like it could be close across a number of init/deinit cycles.

Watching in xsbug, you can see the deinitialize free up about 30 KB of RAM, so it is definitely doing something.

[phoddie]

Got an ESP32 dev board connected to a 5V power supply where I can measure the power draw. The swing between Wi-Fi enabled and disabled is about 77 mA. That proves that the deinitialization is doing something, which is encouraging. 🎉

Memory use is reasonably stable. Just changing the mode without connecting to Wi-Fi appears to be perfectly stable - no memory is lost. Connecting to Wi-Fi after enabling Wi-Fi station mode seems to leak a few bytes (between 8 and 40), though sometimes some of those lost bytes are recovered on a connection. Making an HTTP request after connecting doesn't seem to change the memory lost/gained. So, the memory instability seems to be related to connecting to a Wi-Fi access point.

[phoddie]

OK. Got this into reasonable shape to commit. It should be available some time next week.

Deinitialize of Wi-Fi is just:

WiFi.mode = WiFi.Mode.off;

WiFi.Mode is a new object, basically an enumeration. It also has WiFi.Mode.station and such. It can be used for feature checking. Only supported modes are listed, so you can do a feature check:

if ("off" in WiFi.Mode)
   ;

ESP32 and Raspberry Pi Pico implement off mode for deinitialize.

There is one detail to be careful about. The ESP-IDF really doesn't like it if you deinitialize Wi-Fi while there is an active TCP socket (UDP seems safe, but I'm skeptical). It will eventually crash. To help with that, in debug and instrumented builds, if there are active sockets when deinitializing, the Wi-Fi module traces a warning to the console with the number of active sockets:

WARNING: close all sockets before setting Wi-Fi mode to off. 3 active sockets.

[phoddie]

This is now live as part of Moddable SDK 4.5. Please give it a try!

Note that the changes led to several Wi-Fi unit test failures. They were largely conflicts between connection and scanning (because ESP-IDF doesn't support scan while connecting). Those were something of a challenge to track down, but have been resolved.

[louisvangeldrop]

Is it still needed to put the Wifi in the "off" mode, for the best deepsleep option, even if I don't use Wifi, but only Bluetooth for a temperature/humidity sensor.

[phoddie]

If your project never initializes Wi-Fi, there is no need to deinitialize it. Wi-Fi is automatically initialized on first use, not before. If you never connect to an access point or perform a scan, it should be off.

[louisvangeldrop]

I am not using Wifi, so the Wifi-radio will be off in deepsleep mode.

[phoddie]

That sounds right. This new deinitialize state is for projects (a) don't need Wi-Fi enabled always and (b) want to optimize energy use. Most projects won't need to use it.

[phoddie]

Closing as this has ben resolved for some time.