Work around non-OTA auth issue in ESP8266

[airdrummingfool]

Hello,
There's a bug between PlatformIO and esptool that causes an error when an auth password is set while doing a plugged-in (non-OTA) upload.

The issue is that no matter the upload method, if there is an OTA password set PlatformIO sends --auth=[password] in upload_flags, which esptool cannot handle (only espota understands the --auth parameter). There's more discussion here: https://community.platformio.org/t/upload-error-255-esp8266-upload-flags-for-auth-break-usb-serial-upload/774

The fix is to delete the OTA password when doing non-OTA uploads, and add it back in when you want to do an OTA upload. This makes switching between password-protected OTA uploads and standard uploads a pain, because you have to reset the password each time you switch between the two.

Unfortunately, according to that link the PlatformIO people say it is an esptool bug, and I doubt the esptool people would be any more inclined to fix it.

To be clear, this bug has nothing to do with Deviot. I certainly understand if you think this is out of scope, but I figured Deviot could work around the error by skipping the self.authOTA() call (in PlatformioCLI.py near line 329) IFF the upload is not an OTA upload.

I tried to work up a PR to demonstrate, but I'm not sure of a foolproof way to detect whether or not it is an OTA upload. The only thing I can think of is checking the serial port to see if it is a path (/something/like/this) vs an IP address or network host - but I'm not sure how to do that reliably across all platforms. Do you have any suggestions? Is this something you'd be willing to incorporate into your code, or is this completely out of scope?

Thanks for this great plugin!

[gepd]

I do remember to be dealing with this issue, I didn't known it was a esptool bug.

I am currently rewriting the core of the plugin. As I don't have much time to do the test by myself, can you try to set the password as blank or 0 (cero)?

I've inspect the code and I remember to be added some fix in the code to that issue (as you can see here)

So every time you let the password as blank or write a cero, it will remove the upload_flags from the platformio.ini

[airdrummingfool]

Yes, setting the password to blank works fine, and that's what I'm doing to get around the issue currently. What I was hoping to add is some kind of bypass that also removes upload_flags from platformio.ini if the upload isn't going OTA. That way it would bypass the bug that occurs when trying to upload via a physical serial port with --auth set as an upload flag (the upload always fails if --auth is set for a non-OTA upload).

[gepd]

I see, try this;

Add self.authOTA() after Line 205 and tell me if that fix the problem

[airdrummingfool]

I don't think we want to add another self.authOTA() call - that just makes it more likely that the --auth flag will be there.

Do you know a way to check if the upload is OTA or physical serial cable? My suggestion is something like this (replacing line 328):

if isOTAUploadPort(PORT):
    self.authOTA()

The problem is, I don't know how to implement isOTAUploadPort(). Can I just do something like this?

def isOTAUploadPort(port):
    return (search(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", port) is not None || search(r"^[A-Za-z0-9-.]*$", port) is not None)

The first regex is an IP address check directly from your code line 183
The second regex is a basic (not super strict) check for valid hostnames. I'm not sure, but it seems like all physical serial ports have / in them (possibly \ for Windows? I don't have a Windows box to test with). If that assumption is correct, something like my code might work.

EDIT: it looks like Windows might use COM# port designators... if so, could just make the regex reject anything that starts with COM or has a / in it...?

[gepd]

That should be the behavior adding, self.authOTA().

In Line 183 the code checks if the port selected is an IP or not, so when you select a serial port it adds a 0 to the preferences file.

authOTA check if the password is 0 and removes the flag from platformio.ini

EDIT: I had the same problem checking the serial ports, that is why I only check if it's an IP, if it's not I assume it will be a serial (physical)

EDIT2: I know it's a messy code, the new one is more cleaner and will be easy to implement/improve new features

[airdrummingfool]

My apologies, I did not take the time to understand what the code was doing.

I added self.authOTA() below line 205, but unfortunately it did not work. I still get Error 255 when I attempt to upload via serial cable when an OTA password is set. Once I remove the OTA password in Deviot settings, the upload works fine (same behavior as before).

Small sidenote: enabling "verbose" mode does not work on the master branch - it looks like there are a lot of issue fixes that are only on the develop branch. Should I be using the master or the develop branch to test?

[gepd]

Use develop is true, there is many fixes in that branch. I'm not developing the new version in that branch so you can use it without problem

I think the fix is not working because I missed something when I check the port

the line 183 should be like this:

if(search(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", PORT) is not None and "esp" in MCU.lower()):

Please replace that line and try again

[airdrummingfool]

Thank you for your response and sorry for the delay. I'm now working off of develop.

I changed the line as you instructed and tried again, but it didn't make a difference - I still get Error 255.

I tried to do some debug printing, and while I'm admittedly unfamiliar with the code, it appears that some parts may be indented more than they should. For example, the section that starts # check auth in esp is indented so that it is part of the if(PORT == '' and C['CALLBACK'] is 'upload'): block, i.e. that entire block won't run unless PORT =='' (which isn't true).

It also seems like perhaps the # check if auth is required block should be outside the

if(search(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$", PORT) is not None and "esp" in MCU.lower()):

check, so I de-indented that as well.

This fixed the auth problem for hard-wired connections, but seemed to break other things (including auth for OTA uploads). It looks like the problem I have now is that mdns = Serial.listMdnsServices() returns a list with an empty service, so auth is never marked as necessary.

Since I'm unfamiliar with the code, my changes could be very wrong. But perhaps this may help you in some way.

Thanks!

[gepd]

I have release an alpha version of Deviot 2, but sadly it doesn't have this feature yet, however I hope to implement it this week, I'll let you know as soon as it's ready.

[gepd]

Hey @airdrummingfool I've pushed some changes today with a first implementation of OTA, but I only test it on windows because I couldn't run my esp in the virtual machine

Can you try it and tell me if it's working for you? the device should appear like always in the serial ports menu.

The steps to install Deviot 2 are here