Upload & Monitor Task selects wrong environment

[TD-er]

Steps to reproduce:

• Open arduino-blink example from esp8266 platform
• Run "Upload & Monitor" Task in Project Tasks for board "A"
• Run "Upload & Monitor" Task in Project Tasks for board "B" (BUG!! task for board "A" will be restarted)

[valeros]

Hi @TD-er ! The MMU configuration is handled by this code. PIO_FRAMEWORK_ARDUINO_MMU_CACHE16_IRAM48_SECHEAP_SHARED macro adds the following defines to the compile command:

-D MMU_IRAM_SIZE=0xC000
-D MMU_ICACHE_SIZE=0x4000
-D MMU_IRAM_HEAP

It's a pretty straightforward configuration, so TBH I'm not sure why it might be unreliable. Are there any changes between two consecutive compilations?

[TD-er]

I got the idea the building process on its self might be flaky.
It doesn't matter if I use the PIO define or the ones you mention (with the -DPIO_FRAMEWORK_ARDUINO_MMU_CUSTOM or else you get lots of complaints about redefines)

Just some observations:

• Make a build -> OK, 2nd heap is present
• Do some tests, decide things don't work out, thus revert to previous state of code
• Make a build -> sometimes OK, sometimes 2nd heap is not present.

N.B. I don't build other PIO envs inbetween and don't touch the PIO ini files.

What does happen every now and then -now that I think of it- is that my PIO project tasks tree gets folded with an infinite waiting state (like what you have when unfolding it for the first time, showing "Loading...") unless you close the item and open it again.

I mean like pressing here:

This does seem to happen more often when you toggle between tasks like "build", "clean" and "Upload and Monitor".
Could this also affect the build process?

[valeros]

Could this also affect the build process?

Highly unlikely. That task tree simply runs the pio project data command on your project to retrieve the build information. So if you manually run pio project data --json-output command in the IDE terminal you will see all the build flags and targets available (including PIO_FRAMEWORK_ARDUINO_MMU_CACHE16_IRAM48_SECHEAP_SHARED).

In any case, your freshly edited files will trigger linker to reassemble the final elf image and there might something wrong with autogenerated linker scripts.

Anyway, could you please prepare a minimal example so I can reproduce it on my machine?

[TD-er]

Well a "minimal" example is quite hard as I don't have a clue here what triggers it.
N.B. working on the code for this PR: letscontrolit/ESPEasy#3680

By the way, the IDE just suddenly "folded" the project tasks as I described, which happens every now and then.
So as you said, not likely to be causing this problem, but more to indicate what happens as I mentioned it.

Edit2:
Just pushed the latest changes to this PR.
I now changed the 'beta' builds (Arduino staging code) to using the 2.2.2 SDK as 3.0.0 SDK does cause lots of WDT reboots not leaving any clue where the ESP may hang.
It does not seem to be related to changing heap/cache ratios as it also causes WDT crashes when using the default ratios.

[TD-er]

I do have the feeling (I know, lots of 'feeling', not yet hard facts) that it may somehow be related to this issue:
esp8266/Arduino#8169

The reason I got the feeling it may be related is that I'm having a number of issues when running the current staging code of esp8266/Arduino, regardless if it is using SDK3 or SDK222 and also with or without experimenting with the 2nd heap.

What I'm seeing (so leave out the idea of the MMU defines) is that the ESP may crash on seemingly unrelated code.
Code that was not changed in a long time.
It does look like code called with flash strings does appear more often on the decoded stack trace, but not always.
Also the WDT resets do occur quite often.

So this does seem like there is something fishy in the way how code is being linked. The order in which it is linked may lead to working or crashing build.
For example I now had it crash with the stack trace in the linked issue. Did a complete clean of the .pio directory, the packages, cache and some other folder in the .platformio directory and did a clean rebuild. Now it is running fine.
With the crashing build it was a rebuild with only a minor change in 1 file and the build before was based on a clean build just like the one running on my node.

I hope this is enough defense to give my 'gut feeling' some credits ;)

Edit:
N.B. I was just thinking... if it is indeed related to how things are being linked, then I am not sure it is even possible to make a 'minimal' example to reproduce it.

[TD-er]

I have another idea of what may be happening here...

In VS code, you have on the right the various PIO tasks once done so you can switch between them.

What I now noticed was that I was building a build before that did not have the 2nd heap stuff defined for the PIO env.
So I cleared all build files etc. and selected from another PIO env. to "upload and monitor". (an env which should have it defined)
It starts building, uploading and again no 2nd heap in that build.
Then I noticed it built the wrong environment even though I am 100% sure I clicked the right one.

Clicked this one:

But since a previous task with "Upload and Monitor" from another env was still present in this list in the lower right corner of VS code, it built that one instead.

So it looks like the re-use of tasks is picking the wrong environment.

[TD-er]

Ah proof!

What is pressed:

What is executed:

And what has been flashed:

Build Time:⋄	Jun 25 2021 17:26:28
Binary Filename:⋄	ESP_Easy_mega_20210625_test_B_beta_ESP8266_4M1M

[TD-er]

And it is very well reproducible.
At least when I "go up" in the project tasks.

Had "custom_beta_ESP8266_4M1M" via Upload and Monitor and then selected "custom_ESP8266_4M1M" Upload and monitor, but it selected the previous one, as long as the task was still present in the lower right corner of VS code.

After closing those tasks there, the actual clicked task was executed.

[valeros]

So it seems the issue has nothing to do with the PIO_FRAMEWORK_ARDUINO_MMU options as it's rather a PlatformIO for VSCode plugin bug. Does it make sense to move this issue to the https://github.com/platformio/platformio-vscode-ide repository?

[TD-er]

Yep, I guess so.

[novirium]

I'm hitting a similar issue, though I've managed to get some more details - in my case, the environment passed when running the "Upload" or "Upload and Monitor" task is not updated if a monitor started by "Upload and Monitor" is still running.

Say I have two environments set up in my platformio.ini file, and am happily using the task list in VSCode to build and upload both of them. As soon as I have a monitor running (for instance, by running the "Upload and Monitor" task) for one of them, the environment gets frozen for any future "upload" or "upload and monitor" tasks". Running either task will run it for the environment the monitor was running for.

Stopping the monitor allows things to function correctly.

[novirium]

Further investigation has revealed this is due to the "autoCloseSerialMonitor" feature. When the monitor is resumed, this is done by restoring the task that opened the monitor. If that task was itself an "Upload and Monitor" task, then restoring it will clobber the task you just tried to run - you can actually see this in list of task terminals (if you tried to do an "Upload" it will have failed, and the old Upload and Monitor will have restarted - whether it's the right environment or not).

Worse, here the new "Upload and Monitor" task is completely ignored in favour of restoring the old one - even if args like the environment are different.

Fundamentally, I'm not sure it's ever really okay to be trying to restore an "Upload and Monitor" task - as it'll try and re-upload via that, rather than the "Upload" or "Upload and Monitor" task you've just selected...

For now, the main workaround is to just close any active monitor before uploading if you are using multiple environments. The risk here is that people will get stung expecting to have uploaded one environment, when it's actually re-uploaded the old one.

As far as fixing this, I'm also not really sure what the desired behavior here would actually be. Unless there is a way of breaking out the "monitor" task such that it is always independent of which combination task called it? Then it would be able to be restored without the side effects of re-running the entire task that first needed the monitor (like Upload and Monitor).

[TD-er]

For now, the main workaround is to just close any active monitor before uploading if you are using multiple environments. The risk here is that people will get stung expecting to have uploaded one environment, when it's actually re-uploaded the old one.

That I can confirm from experience as it happens to me all the time.

[reivaxy]

This is happening to me all the time as well.
I clicked 3 times in a row on the Upload and Monitor in one task. the first two builds were done on the taks I built before that, and the third on the selected task.
Didn't close terminal or monitor between the 3 clicks.

But it happens almost each time I switch from one task to another.

[Jason2866]

@ivankravets Is there a possibility to use added targets from a platform like platform-espressif32 in a custom toolbar with the current active env?

So in this example platformio run --target uploadfs --environment tasmota32-lvgl

[nicklasb]

@ivankravets Hi!
Tested this and found some problems:
My boards:

1. "Upload and monitor" creates a new terminal and it now uses the right target, which is good.
2. However, if you try with yet another target, the second terminal closes and the new takes its place.
3. If I click "Upload and monitor" on the first first target, both are closed and only the last one runs.

2-3 are problematic, as I want to monitor several targets on different boards.

Native platform:
4. The nativeplatform now seems to have gotten a "Upload and monitor" choice, which is nice as it runs it which required a run config before.
5. However, that platform is just an operating system process, so there is nothing to upload to and monitor. Instead, for some reason, the monitor instead connects to what seems to be the last thing I monitored before, which is one of my boards.

Edit: Actually, my Multi-platform-Multi-board demo project could be used to test this, even if there are no boards attached, one should be able to see what PlatformIO tries to do:
https://github.com/RobustoFramework/Multi-platform-Multi-board

[ivankravets]

@Jason2866, yes, just use the name of the task which you see in the VSCode Menu: Terminal > Run Task > PlatformIO

        "text": "$(check)",
        "tooltip": "PlatformIO: Upload Filesystem Image (tasmota32-lvgl)",
        "commands": [
          {
            "id": "workbench.action.tasks.runTask",
            "args": "PlatformIO: Upload Filesystem Image (tasmota32-lvgl)"
          }
        ]
      

[Jason2866]

@ivankravets Thx for the answer, is there a variable holding the current env which can be used instead of a hard coded value here?

[ivankravets]

@Jason2866, good idea. That we can remove "custom commands", such as platformio-ide.uploadAndMonitor.

Could I ask yo to file a feature request? We will implement it in 3.0.1 release. It will allow users to use native VSCode's workbench.action.tasks.runTask and PlatformIO commands.

PlatformIO: Upload Filesystem Image (${currentEnvironment})

[ivankravets]

Could you re-test with PlatformIO IDE 3.0.1-beta.3 for VSCode?

[nicklasb]

@ivankravets Nice!
Works much better now, if I click on "Upload and monitor" on the different boards, it will correctly re-use the correct one.

The only kink is that "Native" still starts a connection to one of my boards (the first one in platformio.ini), and that will later likely block the upload of "Upload and monitor" as it hogs that port.
Not difficult to work around, but annoying to have to think remember.

In its current state,perhaps the Native platforms' "Upload and monitor" should have a "Run"-task instead to separate it properly? Unless there is some upload support for Native to for example som RaspberryPy or something?

[mryall-mawson]

Thanks for fixing this. I tried 3.0.1-beta.3 in my project with two devices/environments and the build targets now use the correct environment and open or reuse the right terminal windows.

I've just started using PlatformIO this week, so was great to see this fixed. It makes it really simple to deploy and monitor on two different devices now - kudos!

[ivankravets]

The only kink is that "Native" still starts a connection to one of my boards (the first one in platformio.ini), and that will later likely block the upload of "Upload and monitor" as it hogs that port.
Not difficult to work around, but annoying to have to think remember.

@nicklasb, could you share here the platformio.ini contents to reproduce this issue?

[nicklasb]

@nicklasb, could you share here the platformio.ini contents to reproduce this issue?
@ivankravets
Sure! Here is the project (linked to it before, but realized that was an edit).

You should be able to run at least Native out of the box, the project is sort of a start-of-point so it should work:
https://github.com/RobustoFramework/Multi-platform-Multi-board

[ivankravets]

The PlatformIO IDE 3.0.1 (pre-release version of 3.1.0) is published to the registry.

@nicklasb, it contains the fix for the "native" platform. Please re-test:

1. Navigate in VSCode to the Extensions activity bar
2. Click on the PlatformIO IDE
3. Switch to the PRE-RELEASE version.

Does it work now?

[nicklasb]

@ivankravets
Strange, it doesn't seem to be possible to choose the pre-release (3.0.1) anywhere.
I don't presume that the setting "use Development PIO Core" is relevant as I gather this is not a PIO Core change, and that doesn't affect my choices.
Is there something more I should do?

[Jason2866]

@nicklasb In VSC extensions switch to Pre-Release Version

[nicklasb]

@ivankravets Interestingly, I don't see that button. Is that because I run the .vsx?

[Jason2866]

@nicklasb "update" to 3.0.0 and after this you can change to prerelease

[nicklasb]

Ok, now I have managed to switch, but the problem persists, regrettably.

Note the log below, after [SUCCESS] it tries to connect to /dev/cu.usbmodem142401 | 9600 8-N-1 which causes all the garbage data as that the wrong speed for that device.
Which device it tries to contact seems to be the last one used, which might indicate that the command that does it have no parameters or something.

I don't want to carry on, but could you not reproduce using this project? It should try to connect to some (hopefully) non-existing usb-port on your host.

Linking .pio/build/native/program
Configuring upload protocol...
Executing .pio/build/native/program
I (6) Test_Tag: Before delay 1001
I (1006) Test_Tag: After delay 1002
================================================================ [SUCCESS] Took 6.57 seconds ================================================================
--- Terminal on /dev/cu.usbmodem142401 | 9600 8-N-1
--- Available filters and text transformations: colorize, debug, default, direct, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at https://bit.ly/pio-monitor-filters
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H
q4꥗a3����B"�rRH
               (�Hp P+� ���0���$�j�<�

[Jason2866]

@nicklasb Have you tried to add the monitor speed to the port?

[env:uno]
; Arduino Uno Rev3
platform = atmelavr
board = uno
framework = arduino
lib_extra_dirs = components

monitor_port = /dev/cu.usbmodem142401
upload_port = /dev/cu.usbmodem142401

Or maybe i do not understand your issue at all?

[nicklasb]

@Jason2866
I am afraid that is not the problem, it should not connect to the port at all.
I am running the native platform, it should not randomly connect to (and lock) recently used USB ports.

[Jason2866]

Imho it is a platformio user config error to use upload and monitor for native.

[ivankravets]

Sorry, I don't understand the problem at all :( See this commit what we did da93a68

It means, when there is an open serial port monitor and you try to use a "native" dev-platform, we will not close the active monitors.

It seems we should remove that part of the code because the use case is very rare.

[nicklasb]

@ivankravets @Jason2866
I would not disagree with either of you; I wrote this above:

In its current state,perhaps the Native platforms' "Upload and monitor" should have a "Run"-task instead to separate it properly? Unless there is some upload support for Native to for example some RaspberryPy or something?

But as you rather asked about "Upload and monitor", I answered and tested. And here we are now. :-)

[nicklasb]

@ivankravets @Jason2866
Actually, I just saw that "Build" runs the code now (it wasn't my impression it did earlier?).
So "Upload and monitor" and "upload" could be removed without any consequence, at least for me.

[birnam]

This fix appears to have created undesireable behavior. As of 3.1 (I've rolled back to 3.0 and verified this is new to this version), clicking on "Upload and Monitor" subsequent times causes a "The task ... is already active. [Terminate Task] [Restart Task]" dialog.

This appears to be a regression of this bug: #2266

[nicklasb]

@birnam

This fix appears to have created undesireable behavior.

Same here, just noticed that I am getting this (had so many that correctly was like this that I didn't see those who shouldn't be...)

[ivankravets]

@birnam , could I ask you to open a new issue at https://github.com/platformio/platformio-vscode-ide/issues?

[birnam]

@ivankravets @nicklasb New issue created here: #3656