Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added desription of Arduino IDE OTA process to documentation #984

Merged
merged 9 commits into from Nov 10, 2015

Conversation

@krzychb
Copy link
Contributor

@krzychb krzychb commented Nov 8, 2015

Following http://www.esp8266.com/viewtopic.php?p=33235#p33235
Keep up development of this great platform 馃槃 馃憤

@mangelajo
Copy link
Contributor

@mangelajo mangelajo commented Nov 10, 2015

You now have the ArduinoOTA lib that encapsulates this, no need to use the mDNS examples.

@mangelajo
Copy link
Contributor

@mangelajo mangelajo commented Nov 10, 2015

good work, by the way!, documentation is always awesome

@igrr igrr merged commit 421206f into esp8266:master Nov 10, 2015
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@krzychb
Copy link
Contributor Author

@krzychb krzychb commented Nov 11, 2015

@ mangelajo - this is good point to use simpler functionality encapsulated in 鈥淎rduinoOTA 鈥 library.

I am looking forward to include it in wiki and propose to do it once new json release is published with this library build in (I assume this is your plan).

The other way around is to change instruction from 鈥淚nstalling with Boards Manager鈥 to 鈥淯sing git version鈥. Loading 鈥淎rduinoOTA鈥 seems to me not enough if I use staging release and "OTALeds.ino" example. I see compiler complaining about missing "MD5Builder.h" that is part of the core on git.

Seeing repetitive questions asked about OTA on esp8266.com forum I prepared this instruction having beginners in mind. My impression is that adding extra complexity to it would discourage people rather than attract. Extended number of steps == more chances to fail and even more questions.

By making it simpler and referring to proven code we should reduce the same questions asked over and over. As result you will be less distracted and could do what I believe you like doing that is adding new exciting functionality to this platform 馃槃

What do you think?

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

Guys,
just forked the repo to add bargraph to espota.py, then tried what's the documentation for OTA says but seems the sketch described in it https://github.com/esp8266/Arduino/blob/master/doc/ota_updates/ota_updates.md
is not there, when I try
"Start Arduino IDE and load sketch DNS_SD_Arduino_OTA.ino available under File > Examples > ESP8266mDNS"

Looking the repo sktech, it's not there ?https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266mDNS/examples
Did I missed something ?

@igrr
Copy link
Member

@igrr igrr commented Nov 11, 2015

@hallard there were two parallel pull requests: this one and another one from @me-no-dev related to OTA improvements. DNS_SD_Arduino_OTA.ino was gone as part of the second one:
a8976a0.
Just need to update docs to use ArduinoOTA library and its examples.

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

@igrr
ok thanks, no problem, will just get this one, test my change in espota.py and do a pull request ;-)

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

@hallard what are you adding to espota.py?

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

@me-no-dev
just answered there in the meantime you were asking the question ;-)
a8976a0

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

to be able to have something like that, I always like bargraph ;-)
image

@igrr
Copy link
Member

@igrr igrr commented Nov 11, 2015

Does this work fine when output is piped by Arduino to its console? I had issues when i used \r to generate progress indicator.

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

no, needs to print to stderr for it to show :)

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

and goes miltiline :)

python /Users/ficeto/Desktop/ESP8266/Arduino-Main/build/macosx/work/Arduino.app/Contents/Java/hardware/esp8266com/esp8266/tools/espota.py -i 192.168.254.182 -p 8266 --auth= -f /var/folders/b_/2gy4g72944359d2vj4htbtwm0000gn/T/build6c445534217ff306d2c48161a213eacf.tmp/ESP_FSBrowser.ino.bin 

Uploading: [                                                            ] 0% 
Uploading: [                                                            ] 0% 
Uploading: [=                                                           ] 1% 
Uploading: [=                                                           ] 1% 
Uploading: [=                                                           ] 2% 
Uploading: [==                                                          ] 2% 
Uploading: [==                                                          ] 3% 
Uploading: [==                                                          ] 3% 
Uploading: [===                                                         ] 4% 
Uploading: [===                                                         ] 4% 
Uploading: [===                                                         ] 5% 
Uploading: [====                                                        ] 5% 
Uploading: [====                                                        ] 6% 
Uploading: [====                                                        ] 7% 
Uploading: [=====                                                       ] 7% 
Uploading: [=====                                                       ] 8% 
Uploading: [=====                                                       ] 8% 
Uploading: [======                                                      ] 9% 
Uploading: [======                                                      ] 9% 
Uploading: [======                                                      ] 10% 
Uploading: [======                                                      ] 10% 
Uploading: [=======                                                     ] 11% 
Uploading: [=======                                                     ] 11% 
Uploading: [=======                                                     ] 12% 
Uploading: [========                                                    ] 12% 
Uploading: [========                                                    ] 13% 
Uploading: [========                                                    ] 14% 
Uploading: [=========                                                   ] 14% 
Uploading: [=========                                                   ] 15% 
Uploading: [=========                                                   ] 15% 
Uploading: [==========                                                  ] 16% 
Uploading: [==========                                                  ] 16% 
Uploading: [==========                                                  ] 17% 
Uploading: [===========                                                 ] 17% 
Uploading: [===========                                                 ] 18% 
Uploading: [===========                                                 ] 18% 
Uploading: [============                                                ] 19% 
Uploading: [============                                                ] 20% 
Uploading: [============                                                ] 20% 
Uploading: [=============                                               ] 21% 
Uploading: [=============                                               ] 21% 
Uploading: [=============                                               ] 22% 
Uploading: [==============                                              ] 22% 
Uploading: [==============                                              ] 23% 
Uploading: [==============                                              ] 23% 
Uploading: [===============                                             ] 24% 
Uploading: [===============                                             ] 24% 
Uploading: [===============                                             ] 25% 
Uploading: [================                                            ] 25% 
Uploading: [================                                            ] 26% 
Uploading: [================                                            ] 27% 
Uploading: [=================                                           ] 27% 
Uploading: [=================                                           ] 28% 
Uploading: [=================                                           ] 28% 
Uploading: [==================                                          ] 29% 
Uploading: [==================                                          ] 29% 
Uploading: [==================                                          ] 30% 
Uploading: [===================                                         ] 30% 
Uploading: [===================                                         ] 31% 
Uploading: [===================                                         ] 31% 
Uploading: [===================                                         ] 32% 
Uploading: [====================                                        ] 33% 
Uploading: [====================                                        ] 33% 
Uploading: [====================                                        ] 34% 
Uploading: [=====================                                       ] 34% 
Uploading: [=====================                                       ] 35% 
Uploading: [=====================                                       ] 35% 
Uploading: [======================                                      ] 36% 
Uploading: [======================                                      ] 36% 
Uploading: [======================                                      ] 37% 
Uploading: [=======================                                     ] 37% 
Uploading: [=======================                                     ] 38% 
Uploading: [=======================                                     ] 38% 
Uploading: [========================                                    ] 39% 
Uploading: [========================                                    ] 40% 
Uploading: [========================                                    ] 40% 
Uploading: [=========================                                   ] 41% 
Uploading: [=========================                                   ] 41% 
Uploading: [=========================                                   ] 42% 
Uploading: [==========================                                  ] 42% 
Uploading: [==========================                                  ] 43% 
Uploading: [==========================                                  ] 43% 
Uploading: [===========================                                 ] 44% 
Uploading: [===========================                                 ] 44% 
Uploading: [===========================                                 ] 45% 
Uploading: [============================                                ] 45% 
Uploading: [============================                                ] 46% 
Uploading: [============================                                ] 47% 
Uploading: [=============================                               ] 47% 
Uploading: [=============================                               ] 48% 
Uploading: [=============================                               ] 48% 
Uploading: [==============================                              ] 49% 
Uploading: [==============================                              ] 49% 
Uploading: [==============================                              ] 50% 
Uploading: [===============================                             ] 50% 
Uploading: [===============================                             ] 51% 
Uploading: [===============================                             ] 51% 
Uploading: [===============================                             ] 52% 
Uploading: [================================                            ] 53% 
Uploading: [================================                            ] 53% 
Uploading: [================================                            ] 54% 
Uploading: [=================================                           ] 54% 
Uploading: [=================================                           ] 55% 
Uploading: [=================================                           ] 55% 
Uploading: [==================================                          ] 56% 
Uploading: [==================================                          ] 56% 
Uploading: [==================================                          ] 57% 
Uploading: [===================================                         ] 57% 
Uploading: [===================================                         ] 58% 
Uploading: [===================================                         ] 58% 
Uploading: [====================================                        ] 59% 
Uploading: [====================================                        ] 60% 
Uploading: [====================================                        ] 60% 
Uploading: [=====================================                       ] 61% 
Uploading: [=====================================                       ] 61% 
Uploading: [=====================================                       ] 62% 
Uploading: [======================================                      ] 62% 
Uploading: [======================================                      ] 63% 
Uploading: [======================================                      ] 63% 
Uploading: [=======================================                     ] 64% 
Uploading: [=======================================                     ] 64% 
Uploading: [=======================================                     ] 65% 
Uploading: [========================================                    ] 66% 
Uploading: [========================================                    ] 66% 
Uploading: [========================================                    ] 67% 
Uploading: [=========================================                   ] 67% 
Uploading: [=========================================                   ] 68% 
Uploading: [=========================================                   ] 68% 
Uploading: [==========================================                  ] 69% 
Uploading: [==========================================                  ] 69% 
Uploading: [==========================================                  ] 70% 
Uploading: [===========================================                 ] 70% 
Uploading: [===========================================                 ] 71% 
Uploading: [===========================================                 ] 71% 
Uploading: [============================================                ] 72% 
Uploading: [============================================                ] 73% 
Uploading: [============================================                ] 73% 
Uploading: [============================================                ] 74% 
Uploading: [=============================================               ] 74% 
Uploading: [=============================================               ] 75% 
Uploading: [=============================================               ] 75% 
Uploading: [==============================================              ] 76% 
Uploading: [==============================================              ] 76% 
Uploading: [==============================================              ] 77% 
Uploading: [===============================================             ] 77% 
Uploading: [===============================================             ] 78% 
Uploading: [===============================================             ] 79% 
Uploading: [================================================            ] 79% 
Uploading: [================================================            ] 80% 
Uploading: [================================================            ] 80% 
Uploading: [=================================================           ] 81% 
Uploading: [=================================================           ] 81% 
Uploading: [=================================================           ] 82% 
Uploading: [==================================================          ] 82% 
Uploading: [==================================================          ] 83% 
Uploading: [==================================================          ] 83% 
Uploading: [===================================================         ] 84% 
Uploading: [===================================================         ] 84% 
Uploading: [===================================================         ] 85% 
Uploading: [====================================================        ] 86% 
Uploading: [====================================================        ] 86% 
Uploading: [====================================================        ] 87% 
Uploading: [=====================================================       ] 87% 
Uploading: [=====================================================       ] 88% 
Uploading: [=====================================================       ] 88% 
Uploading: [======================================================      ] 89% 
Uploading: [======================================================      ] 89% 
Uploading: [======================================================      ] 90% 
Uploading: [=======================================================     ] 90% 
Uploading: [=======================================================     ] 91% 
Uploading: [=======================================================     ] 91% 
Uploading: [========================================================    ] 92% 
Uploading: [========================================================    ] 93% 
Uploading: [========================================================    ] 93% 
Uploading: [========================================================    ] 94% 
Uploading: [=========================================================   ] 94% 
Uploading: [=========================================================   ] 95% 
Uploading: [=========================================================   ] 95% 
Uploading: [==========================================================  ] 96% 
Uploading: [==========================================================  ] 96% 
Uploading: [==========================================================  ] 97% 
Uploading: [=========================================================== ] 97% 
Uploading: [=========================================================== ] 98% 
Uploading: [=========================================================== ] 99% 
Uploading: [============================================================] 99% 
Uploading: [============================================================] 100% Done...

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

maybe add it as an option?

@igrr
Copy link
Member

@igrr igrr commented Nov 11, 2015

Maybe just remove percentage indicator? This way we won't need to use \r
and output will be single line.

On Wed, Nov 11, 2015 at 6:39 PM, Me No Dev notifications@github.com wrote:

maybe add it as an option?


Reply to this email directly or view it on GitHub
#984 (comment).

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

that's strange it was working fine on my env some months ago, need to check this

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

on my computer under Win10 cmd and cygwin works fine without new lines, I need to test under IDE as soon as I can see my device in network board list
image

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

here is the pull with the optional progress: https://github.com/esp8266/Arduino/pull/1003/files
use -r or --progress to see it

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

@me-no-dev
I installed this Arduino build, but unfortunately I still don't see ESP8266 device in network port name
Device just updated with OTA,

  • serial COM6 see the log and display ip address, http server started (sketch running)
  • bonjour browser see the service and name declared WifInfo_fcb401
  • cmd prompt can ping device by name WifInfo_fcb401.local
  • and Arduino IDE just see serial port !!

But don't worry, under windows seeing network device using Arduino IDE has been a pain from beginning, You can't imagine how much time I lost trying to get this working, I tough this time with 1.6.6 and build 1.6.7 will correct, but still not.

image

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

Are you using the latest ESP8266 core from git? The changes necessary for it all to work were merged yesterday. There is an updated BasicOTA example in the ArduinoOTA library that has all new code needed/available.

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 11, 2015

my bad :) visible in the bonjour browser :) no clue then...

@hallard
Copy link
Contributor

@hallard hallard commented Nov 11, 2015

Don't worry I'm doing on command line for months now, already had this issue with dragino and arduino yun, Of course tried with Windows Firewall disabled ;-)
I'm just telling you I can't test newline on espota.py progress bar under win10 and arduino IDE ;-(

@krzychb
Copy link
Contributor Author

@krzychb krzychb commented Nov 12, 2015

@hallard 鈥 your documentation of issue with network port is excellent. Thank you for investing your time and providing such level of details.

I have exactly the same issue and was about to document it as I am preparing an update of documentation to cover new ArduinoOTA library.

My testing was done on 11.11.2015 late afternoon CET and I made fresh installation of Arduino IDE 1.6.5-r5 for Windows and fresh "git clone https://github.com/esp8266/Arduino.git esp8266". I tried both 鈥淏asicOTA.ino鈥 as well as 鈥淥TALeds.ino鈥 provided among examples. I have exactly the same results as you had 鈥 the module successfully joins the network but the OTA network port does not show up in IDE.

Could you, please, point me to your work around / what command line you are using to make the ArduinoOTA work for Windows?

Krzysztof

@hallard
Copy link
Contributor

@hallard hallard commented Nov 12, 2015

@krzychb
yeah, no pb, I will do it by the end of the day (French time) I'm in a hurry now ;-)

By the way, I wanted to tell that OTA now rocks !!! I've been updated my device la least 100 times yesterday without any lost, even if I had browser requesting the device in the meantime (before this was causing trouble)
Nice work to all of this team ;-)

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

now imagine my disappointment after spending so much time on OTA, that Windows refuses to see the devices... and there is nothing I can do about it

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

Any chance this can be caused by Antivirus software?

@hallard
Copy link
Contributor

@hallard hallard commented Nov 12, 2015

I've just got MS Windows Defender, that's all
Don't worry @me-no-dev some windows users report this to be working on Arduino Forum, it was working sometimes on my old Windows 8.1 but this is the 1st try I test on windows 10

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

found fine on both 8.1 and XP that I have around. Will look for v10 download to give it a shot.
Defender could also be causing this if it's thinking that the mDNS is doing something wrong opening a multicast port

@hallard
Copy link
Contributor

@hallard hallard commented Nov 12, 2015

  • Installed Apple Bonjour Service
  • Disabled Windows Defender
  • Disabled Windows firewall

same thing :-( :-(

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

Oh... a hunch... maybe remove Bonjour? It could be clashing since they use the same port

@hallard
Copy link
Contributor

@hallard hallard commented Nov 12, 2015

I've just installed it to give a try (it was not before last hour), but I have an idea, let me check something

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

Default clean install:
screen shot 2015-11-12 at 15 37 19

@krzychb
Copy link
Contributor Author

@krzychb krzychb commented Nov 12, 2015

@me-no-dev 鈥 don鈥檛 worry 鈥 basing on responses so far I am certain that you and other guys there are a step apart from finding the reason of this issue or a workaround 馃槃

Looking through new ArduionOTA code it believe it is very versatile and I can鈥檛 wait trying out all the functions you have implemented.

To supplement information provided I am using Windows 7. The OTA port shows up when I am using the legacy 鈥淒NS_SD_Arduino_OTA.ino鈥 sketch.

Krzysztof

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

@krzychb thanks for the good words :)
You can grab the windows build from above and download the latest core as zip from the repo and since the build is portable, you can have it on a side and try the new OTA :)
I have to say, I can't wait for the Arduino team to merge so we can finally have a fully working OTA. Flashing is a breeze now and the result is not only consistent, but I really have not had a failed OTA since the changes :)

@sticilface
Copy link
Contributor

@sticilface sticilface commented Nov 12, 2015

I agree the new OTA, seems a lot faster and I've not had a single fail. Good work @me-no-dev

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

to be honest with you all, the change that made OTA work consistently was replacing yield() with delay(1).
The issue manifested itself in a really strange way... the connection hangs and as soon as the computer closes it as a result of the timeout, the data that was supposed to come from the ESP is sent with the first FIN packet. I tried a few things and looked much deeper, then as a last resort tried with some delay instead. And whoa !!!
So keep in mind when you do network projects and have issues, to try something that simple and see if things improve.

@igrr
Copy link
Member

@igrr igrr commented Nov 12, 2015

As mentioned by @EUA in #1005 this might have been related to light sleep mode (which is the default one).

I had an issue with network and light sleep in one of my sketches, where I was sending telemetry from a UAV (plane) to ground station. When I disabled light sleep, the number of dropped UDP packets went down by two orders of magnitude.

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

unrelated: what range have you tested on the UAV?

@sticilface
Copy link
Contributor

@sticilface sticilface commented Nov 12, 2015

That is interesting, I might have a play with that.
I was / have been trying to implement self/downloading firmware and files for SPIFFS, and adding delay, but not yield fixed it for HTTP only downloads. I thought the code was looping too fast and the buffer being emptied to quickly, so a delay give it time to accumulate. might be total BS though.

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

based on how TCP works, one packet with data and one response with data look like this:

// PC -----> DATA ----> ESP
// PC <----- ACK <----- ESP //needs yield
// PC <- RESPONSE <- ESP //needs another yield
// PC -----> ACK ----> ESP //might also need yield

so having lots of data will lead to not enough empty loops/yields to be ale to transmit and receive all packets
If on other hand the situation is like @igrr is saying, the wifi is napping too much and needing more yields to wake up and do it's thing

@krzychb
Copy link
Contributor Author

@krzychb krzychb commented Nov 12, 2015

Basing on previous posts I did the following steps:

  1. Uninstalled Java from my PC
  2. Uninstalled Arduino IDE 1.6.5-r5
  3. Installed Arduino IDE 1.6.6
  4. Checked for network OTA port - it was there!!!
  5. Installed latest esp8266 cores from Github
  6. Checked for network port - it was gone 馃槥
  7. Tried to replicate this sequence 鈥 the port did not show up again.

So I am back to step zero and keep looking for clues 馃槃
BTW - this is all on Windows 7 PC

@me-no-dev 鈥 I wonder if you tried to load esp module from your Windows set up and if the network port was not gone at some certain point?

Krzysztof

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

@krzychb Yes I did. I installed python and did OTA. The device showed as soon as I opened the IDE and pulled the ports menu.

@me-no-dev
Copy link
Collaborator

@me-no-dev me-no-dev commented Nov 12, 2015

Also tested on Windows 8.1 and Windows XP with success as well

@igrr
Copy link
Member

@igrr igrr commented Nov 12, 2015

@krzychb i'm concerned about your step 3. I think you need to get one of the builds mentioned by @me-no-dev.
The reason is that these IDE builds do not attempt to check port 8266 for being reachable over TCP.

Older version of OTA sample works because it also initializes a TCP server on port 8266, and IDE reachability check passes.

@krzychb
Copy link
Contributor Author

@krzychb krzychb commented Nov 12, 2015

@igrr 鈥 every thing is hunky-dory now!

capture

I will leave this special screen on my PC overnight it took so much sweat and tear 馃槃

I was indeed intrigued where you got Arduino 1.6.7 from and did not catch that you coin own by yourself. You guys are awesome 馃憤

Krzysztof

@tzapu
Copy link
Contributor

@tzapu tzapu commented Nov 18, 2015

i can't get the port to show up in os x either
#1039

@aminbehravan
Copy link

@aminbehravan aminbehravan commented Jun 9, 2017

http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-4107-BUILD-421-windows.zip
404 error not found. it cant be download.
actually all links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants
You can鈥檛 perform that action at this time.