fw-server: add ip option #2759

gtortone · 2018-05-17T16:05:12Z

Hi,
a new option (-i | --ip) is available in order to specify IP address to bind (instead of network interface)

Regards,
Gennaro

arendst · 2018-05-17T16:13:36Z

It works! On windows only with the -i option.

.\fw-server.py -i 192.168.2.1
 * Serving Flask app "fw-server" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://192.168.2.1:5000/ (Press CTRL+C to quit)

The -d option still returns strange message:

.\fw-server.py -d {AC6EA539-40B4-4921-A3BD-E081354D06CC}
Usage: fw-server {-d | -i} arg

fw-server.py: error: no such option: -o

Any way. Let's decide to use the -i option on windows only.

gtortone · 2018-05-17T16:18:18Z

I agree !!

gtortone · 2018-05-17T18:18:18Z

@arendst is there a wiki page related to OTA updates from web server ? I could add there some info on this solution...

arendst · 2018-05-17T19:43:19Z

I suggest a new entry under Upload tools. A dedicated document for this solution.

5.14.0a * Add feature information to Status 4 * Add tools folder with python script decode-status.py for decoding some status fields like SetOption and Features * Add python script fw-server.py in tools folder to create a simple OTA server (#2759) * Fix vars on rules (#2769) * Fix bug in KNX menu (#2770)

fw-server: add ip option

5.14.0a * Add feature information to Status 4 * Add tools folder with python script decode-status.py for decoding some status fields like SetOption and Features * Add python script fw-server.py in tools folder to create a simple OTA server (arendst#2759) * Fix vars on rules (arendst#2769) * Fix bug in KNX menu (arendst#2770)

localhost61 · 2018-12-03T23:33:57Z

Hi,
I managed to host fw-server on a headless RPi3 running OpenHAB but was missing the way to run it as a service. After some research I achieved this solution:

With Samba as root I copied .\tools\fw_server folder from Tasmota release to /srv/tasmota on RPi3
Then: # chmod +x /srv/tasmota/fw_server/fw-server.py
In /etc/systemd/system I created a new file tasmota.service that contains :

[Unit]
Description=Local OTA server for Tasmota
Wants=network.target
After=network.target multi-user.target
 [Service]
Type=idle
ExecStart=/usr/bin/python /srv/tasmota/fw_server/fw-server.py -d wlan0
Restart=on-abort
 [Install]
WantedBy=multi-user.target

Then:

# chmod 644 /etc/systemd/system/tasmota.service
# systemctl daemon-reload
# systemctl enable tasmota.service
# systemctl start tasmota.service

So, it works, and # systemctl status tasmota.service reports latest request to the service.
But I'm not skilled, is it the best way to go? A wiki howto would have been welcome.

ascillato · 2018-12-04T00:03:57Z

Great! Thanks for share this. Could you please add this to the wiki? It is very useful. 👍

localhost61 · 2018-12-04T01:49:40Z

Sorry @ascillato I was wrong, the wiki will wait! Firefox on Windows can download from fw-server as a service, but the sonoff devices only reboot, firmware is not updated. A downloader coded in Java I use on Windows neither work in this mode (file not found) when it works with the CLI.

gtortone · 2018-12-04T08:01:18Z

Hi @localhost61,
if you run fw-server by absolute pathname (e.g. /srv/tasmota/fw_server/fw-server.py) from an external directory, line 81:

if os.path.exists("fw/" + str(filename)):

try to open 'fw/' directory from current directory....
it should be useful to add a parameter to specify the directory that contains firmware files;

in the meanwhile you can modify by hand line 81 to include your full pathname (/srv/tasmota/fw_server/fw)

localhost61 · 2018-12-05T02:02:36Z

Thank's @gtortone that's it, of course !
In that context, I think I have no other choice than running the script with absolute pathname.
So I did your changes on line 81 (and 82 too) and it works like a charm.
Wiki soon ;-)

I guess that you are better in Linux than I am. What do you think of my tasmota.service content? About which targets? What is "Wanted", "Required"?
I'm pretty sure it would work with less settings.

gtortone · 2018-12-05T04:33:16Z

Hi @localhost61,
ok, I will add option to specify a firmware directory...

regarding "WantedBy" and "RequiredBy" they are options to specify dependencies between services;
you script is correct because you require network is up before start fw-server service. In your case you want your service is included in multi-user.target that is the old "runlevel 3" where the operating system is configured with network services....

if you want to know more details you can read "systemd" guide:
https://www.freedesktop.org/wiki/Software/systemd/

gtortone · 2018-12-17T09:56:31Z

Thank's @gtortone that's it, of course !
In that context, I think I have no other choice than running the script with absolute pathname.
So I did your changes on line 81 (and 82 too) and it works like a charm.

Hi,
PR #4651

localhost61 · 2018-12-17T19:35:39Z

Thanks @gtortone ! Good job ! 👍
So I can leave fw-server.py untouched out of v6.4.0.1 release
and just edit my file /etc/systemd/system/tasmota.service with :

ExecStart=/usr/bin/python /srv/tasmota/fw_server/fw-server.py -d wlan0 -f /srv/tasmota/fw_server/fw/

I use it now and it works fine in conjunction with the following setup in platformio.ini:

; *** Upload file to OTA server using SCP
upload_port                = pi@raspberrypi:/srv/tasmota/fw_server/fw
extra_scripts              = pio/strip-floats.py, pio/sftp-uploader.py

gtortone · 2018-12-17T19:48:09Z

Hi @localhost61,
you are welcome!

if you want to use fw/ (default) directory you can omit - f option...

localhost61 · 2018-12-17T19:57:59Z

@gtortone, you don't remember ? You did this because services are started from root path and not from the python script directory, one need full qualified path for the script AND for the binary files directory. ;-)

gtortone · 2018-12-17T20:07:57Z

yes, I remember, but now this bug is solved! Try and let me know :-)

localhost61 · 2018-12-17T20:09:15Z

You are right, it works without too.
ExecStart=/usr/bin/python /srv/tasmota/fw_server/fw-server.py -d wlan0
is enought, no option -f required.

localhost61 · 2018-12-18T02:12:50Z

Wiki updated
:-)

localhost61 · 2019-07-20T23:12:55Z

@gtortone
Those days I ran into an issue with sftp-uploader.py after upgrading PIO_IDE to v1.80 and it remained after reinstall PIO_IDE v1.71.
It appears that UPLOAD_PORT is no more included in PIO ARGUMENTS so that line 6: b64decode(ARGUMENTS.get("UPLOAD_PORT")) evaluates to nothing and triggers en error after the dependency graph completion when running an upload task.

Out of the two flags ISATTY and PIOVERBOSE, all the ARGUMENT variables are now included in env and projenv so that deciphering with b64decode is no more a necessity.
Here is what works for me now, can you confirm the issue on your side?

sftp-uploader.py

#  access to global and project construction environment
#Import("env","projenv")
#from base64 import b64decode
Import("env")

#  Dump construction environments (for debug purpose)
#print("\nenv.Dump()=\n------------\n%s\n\nprojenv.Dump()=\n------------\n%s\n" % (env.Dump(),projenv.Dump()))
#print ("\nARGUMENTS=\n------------\n%s\n\n" % ARGUMENTS)
#print ("\nPIOENV=\n------------\n%s\n\n" % b64decode(ARGUMENTS.get("PIOENV")))
#print ("\nUPLOADER=%s, UPLOAD_PORT=%s, PIOENV=%s\n" % (env['UPLOADER'], env['UPLOAD_PORT'], env['PIOENV']))

env.Replace(UPLOADER="scp")
env.Replace(UPLOADERFLAGS="")
env.Replace(UPLOADCMD='$UPLOADER $SOURCES "$UPLOAD_PORT/${PIOENV}.bin"')

#print ("\nUPLOADCMD=%s\n" % env['UPLOADCMD'])

I don't use http-uploader.py but I imagine it needs same treatment:

# from base64 import b64decode
Import("env")

env.Replace(UPLOADER="pio\espupload.py")
env.Replace(UPLOADERFLAGS="")
# env.Replace(UPLOADCMD="$UPLOADER -u " + b64decode(ARGUMENTS.get("UPLOAD_PORT")) + " -f $SOURCES")
env.Replace(UPLOADCMD="$UPLOADER -u " + "$UPLOAD_PORT" + " -f $SOURCES")
'''
env.Replace(UPLOADCMD="pio\espupload.py -f $SOURCES") # Windows
env.Replace(UPLOADCMD="pio/espupload.py -f $SOURCES") # Linux

'''

arendst · 2019-07-21T06:47:01Z

You're right. I had to remove the b64decode from http_uploader too.

Will change soon. Thx for testing.

localhost61 · 2019-07-21T07:09:44Z

Thanks to confirm Theo but I'm not only testing, it's my way to use Tasmota, I was locked and had to find a solution ;-).
Did it arise with PlatformIO IDE 1.8? It's funny it remained when I downgraded to v1.71.

Fix pio 4.0.0 related issues (#2759)

arendst · 2019-07-21T10:26:46Z

It's pio core version 4.0.0 related. We had to change a parameter in platformio.ini too #6073

If downgraded pls check the platformio version with pip list. It probably stayed at 4.0.0

localhost61 · 2019-07-21T11:39:40Z

You're right, I'm using Win8.1 and downgrading the IDE didn't change the core version which is located for me in "%userprofile%.platformio\penv\Scripts". I think the IDE didn't use pip to install the core, it doesn't appear when I enter pip list in the PIO terminal. I say that because I have several versions of Python installed (and use the py.exe utility to use the one or the other), but PIO installed it's own (third) version. What a mess! The same with Git that is included in Cygwin that I use essentially for ssh and that includes it's own version that is different from Git for windows that is better suited for PlatformIO.
Anyway, I don't need anymore to stay in v1.7.1 and as the core is 4.0 in can begin to optimize my platformio.ini ;-)

We had to change a parameter in platformio.ini too #6073

Yes, I saw that but I don't feel concerned with old ESP core v2.30

gtortone · 2019-07-21T14:46:57Z

@gtortone
Those days I ran into an issue with sftp-uploader.py after upgrading PIO_IDE to v1.80 and it remained after reinstall PIO_IDE v1.71.
It appears that UPLOAD_PORT is no more included in PIO ARGUMENTS so that line 6: b64decode(ARGUMENTS.get("UPLOAD_PORT")) evaluates to nothing and triggers en error after the dependency graph completion when running an upload task.

Out of the two flags ISATTY and PIOVERBOSE, all the ARGUMENT variables are now included in env and projenv so that deciphering with b64decode is no more a necessity.
Here is what works for me now, can you confirm the issue on your side?

sftp-uploader.py
#  access to global and project construction environment
#Import("env","projenv")
#from base64 import b64decode
Import("env")

#  Dump construction environments (for debug purpose)
#print("\nenv.Dump()=\n------------\n%s\n\nprojenv.Dump()=\n------------\n%s\n" % (env.Dump(),projenv.Dump()))
#print ("\nARGUMENTS=\n------------\n%s\n\n" % ARGUMENTS)
#print ("\nPIOENV=\n------------\n%s\n\n" % b64decode(ARGUMENTS.get("PIOENV")))
#print ("\nUPLOADER=%s, UPLOAD_PORT=%s, PIOENV=%s\n" % (env['UPLOADER'], env['UPLOAD_PORT'], env['PIOENV']))

env.Replace(UPLOADER="scp")
env.Replace(UPLOADERFLAGS="")
env.Replace(UPLOADCMD='$UPLOADER $SOURCES "$UPLOAD_PORT/${PIOENV}.bin"')

#print ("\nUPLOADCMD=%s\n" % env['UPLOADCMD'])
I don't use http-uploader.py but I imagine it needs same treatment:
# from base64 import b64decode
Import("env")

env.Replace(UPLOADER="pio\espupload.py")
env.Replace(UPLOADERFLAGS="")
# env.Replace(UPLOADCMD="$UPLOADER -u " + b64decode(ARGUMENTS.get("UPLOAD_PORT")) + " -f $SOURCES")
env.Replace(UPLOADCMD="$UPLOADER -u " + "$UPLOAD_PORT" + " -f $SOURCES")
'''
env.Replace(UPLOADCMD="pio\espupload.py -f $SOURCES") # Windows
env.Replace(UPLOADCMD="pio/espupload.py -f $SOURCES") # Linux

'''

Hi, this commit is related to fw-server... while you are describing a problem
for sftp-uploader.py....

localhost61 · 2019-07-21T16:08:04Z

Yes @gtortone, I know that but it's closed and as I believe you are the author/maintainer of those scripts it seemed to me the best way to reach you and get your attention.

gtortone · 2019-07-21T16:25:12Z

@localhost61 I'm mantainer of fw-server not pio scripts... but I see that Theo is already working on you issue!

localhost61 · 2019-10-06T08:40:34Z

For several weeks I couldn't upgrade my devices through OTA. I thought I introduced a bug in my previous build that broke the mechanism... without suspecting the OTA server.
Today I finally tried to download the served files from my browser and found it didn't work neither.

I checked the status of fw-server in a terminal session and found that the wlan0 IP was wrong, certainly due to a previous power failure.

Wiki updated: After a power failure, local HTTP OTA server may become unresponsive

fw-server: add ip option

0356a90

gtortone mentioned this pull request May 17, 2018

Eastron SDM630 energy meter - new sensor #2735

Closed

arendst merged commit edf9a5c into arendst:development May 17, 2018

curzon01 pushed a commit to curzon01/Tasmota that referenced this pull request Sep 7, 2018

Merge pull request arendst#2759 from gtortone/development

db0b0e8

fw-server: add ip option

arendst added a commit that referenced this pull request Jul 21, 2019

Fix pio 4.0.0 related issues

3a3e5a9

Fix pio 4.0.0 related issues (#2759)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fw-server: add ip option #2759

fw-server: add ip option #2759

gtortone commented May 17, 2018

arendst commented May 17, 2018 •

edited

gtortone commented May 17, 2018

gtortone commented May 17, 2018

arendst commented May 17, 2018

localhost61 commented Dec 3, 2018

ascillato commented Dec 4, 2018

localhost61 commented Dec 4, 2018

gtortone commented Dec 4, 2018 •

edited

localhost61 commented Dec 5, 2018 •

edited

gtortone commented Dec 5, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018 •

edited

localhost61 commented Dec 18, 2018

localhost61 commented Jul 20, 2019

arendst commented Jul 21, 2019

localhost61 commented Jul 21, 2019 •

edited

arendst commented Jul 21, 2019

localhost61 commented Jul 21, 2019 •

edited

gtortone commented Jul 21, 2019

localhost61 commented Jul 21, 2019

gtortone commented Jul 21, 2019

localhost61 commented Oct 6, 2019

fw-server: add ip option #2759

fw-server: add ip option #2759

Conversation

gtortone commented May 17, 2018

arendst commented May 17, 2018 • edited

gtortone commented May 17, 2018

gtortone commented May 17, 2018

arendst commented May 17, 2018

localhost61 commented Dec 3, 2018

ascillato commented Dec 4, 2018

localhost61 commented Dec 4, 2018

gtortone commented Dec 4, 2018 • edited

localhost61 commented Dec 5, 2018 • edited

gtortone commented Dec 5, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018

gtortone commented Dec 17, 2018

localhost61 commented Dec 17, 2018 • edited

localhost61 commented Dec 18, 2018

localhost61 commented Jul 20, 2019

arendst commented Jul 21, 2019

localhost61 commented Jul 21, 2019 • edited

arendst commented Jul 21, 2019

localhost61 commented Jul 21, 2019 • edited

gtortone commented Jul 21, 2019

localhost61 commented Jul 21, 2019

gtortone commented Jul 21, 2019

localhost61 commented Oct 6, 2019

arendst commented May 17, 2018 •

edited

gtortone commented Dec 4, 2018 •

edited

localhost61 commented Dec 5, 2018 •

edited

localhost61 commented Dec 17, 2018 •

edited

localhost61 commented Jul 21, 2019 •

edited

localhost61 commented Jul 21, 2019 •

edited