Multiple issues with 'extends' option in platformio.ini

[lnlp]

PlatformIO Core
Bug in / multiple issues with extends option in platformio.ini.

Description of problem

1. The extends option does not work as expected and behaves inconsistently:

   • Values of multi-value parameters like lib_deps and build_flags are either replaced or not used at all instead of being extended ('merged').
     This is not as expected and renders use of the extends option useless for many applications.

   • Which section has priority if a parameter is defined in multiple sections is unclear and handled inconsistently:

     • If a parameter is defined in extending and in extended sections, the value in the extending section is used (overwrites already defined value).
     • If a parameter is defined in multiple extended sections (but not in the extending section): The value in the first section in the extends= parameter values list is used and all other extended sections are ignored. Once a value is defined it is not overwritten by any other extended sections in the list.
       In one situation an already defined parameter is overwritten, while in the other situation it is not overwritten, which is illogical and inconsistent.

2. Documentation is poor. It lacks essential information about how the extends option works.

   • What happens with parameters ('keys') that are defined in multiple sections (multiple extended sections and/or extending section(s))?
   • Are values overwritten or not? When and when not?
   • What is the priority order in determining the final value(s) of a parameter?
   • Is there any difference in how single value and multi value options are handled by using the extends option?
     (There should be.)
   • What about lib_deps and build_flags?

This issue is related to #3788 but this issue is wider in scope.
For me the 'overwriting' of lib_deps and build_flags ('not merging the values') is a bug and not a feature request.
(The word 'overwriting' is only partially correct here because behavior of extends is inconsistent.)

Steps to Reproduce

Based on the extends documentation I performed two tests. For each test platformio.ino and main.cpp are listed below in the source code section. Their serial output is listed in Actual Results.

To reproduce:

1. Connect some PlatformIO (Arduino) supported board to the serial port.
2. In platformio.ini update platform and board parameters for this board.
3. Compile, upload and run the included platformio.ini and main.cpp for Test 1 and check the output on the serial monitor.
4. Compile, upload and run the included platformio.ini and main.cpp for Test 2 and check the output on the serial monitor.

Actual Results

Test 1

serial output

platformio.ini extends test

FIVE is defined
PlatformIO 'extends' in platformio.ini is bugged

Test 2

serial output

platformio.ini extends test

ONE is defined
PlatformIO 'extends' in platformio.ini is bugged

It looks like:

• a) If defined (build_flags) in the extending section (env:main) the value in the extending section is used
• b) If not defined (build_flags) in the extending section, but defined in multiple extended sections then the value in the first extended session in the list of extends = values will be used.

Unexpected:

• a) already defined build_flags values are overwritten in the extending section.
• b) the value defined in the first section in the list of extends = is used. This value is NOT overwritten/replaced by the next section in the list and neither is it replaced in any extended section that is both extended and extending section.
  Behavior of extends = is inconsistent and different when used nested.

Expected Results

Test 1

serial output

platformio.ini extends test

ONE is defined
TWO is defined
THREE is defined
FOUR is defined
FIVE is defined

Test 2

serial output

platformio.ini extends test

ONE is defined
TWO is defined
THREE is defined
FOUR is defined

Source files to reproduce issue

Test 1

platformio.ini

[platformio]
default_envs = main

[one]
build_flags = 
    -D ONE=1

[two]
build_flags = 
    -D TWO=2

[three]
build_flags = 
    -D THREE=3

[four]
extends = three
build_flags = 
    -D FOUR=4

[env:main]
extends = one, two, four
platform = espressif32
board = nodemcu-32s
framework = arduino
monitor_speed = 115200
build_flags = 
    -D FIVE=5

main.cpp

#include <Arduino.h>

void setup() 
{
    Serial.begin(115200);

    Serial.println("\n\nplatformio.ini extends test\n");

    #ifdef ONE
        Serial.println("ONE is defined");
    #endif

    #ifdef TWO
        Serial.println("TWO is defined");
    #endif

    #ifdef THREE
        Serial.println("THREE is defined");
    #endif

    #ifdef FOUR
        Serial.println("FOUR is defined");
    #endif

    #ifdef FIVE
        Serial.println("FIVE is defined");
    #endif    

    #if !defined(ONE) || !defined(TWO) || !defined(THREE) || !defined(FOUR) || !defined(FIVE)
        Serial.println("PlatformIO 'extends' in platformio.ini is bugged");
    #endif
}

void loop() 
{
}

---

Test 2

platformio.ini

[platformio]
default_envs = main

[one]
build_flags = 
    -D ONE=1

[two]
build_flags = 
    -D TWO=2

[three]
build_flags = 
    -D THREE=3

[four]
extends = three
build_flags = 
    -D FOUR=4

[env:main]
extends = one, two, four
platform = espressif32
board = nodemcu-32s
framework = arduino
monitor_speed = 115200

main.cpp

#include <Arduino.h>

void setup() 
{
    Serial.begin(115200);

    Serial.println("\n\nplatformio.ini extends test\n");

    #ifdef ONE
        Serial.println("ONE is defined");
    #endif

    #ifdef TWO
        Serial.println("TWO is defined");
    #endif

    #ifdef THREE
        Serial.println("THREE is defined");
    #endif

    #ifdef FOUR
        Serial.println("FOUR is defined");
    #endif

    #if !defined(ONE) || !defined(TWO) || !defined(THREE) || !defined(FOUR)
        Serial.println("PlatformIO 'extends' in platformio.ini is bugged");
    #endif
}

void loop() 
{
}

Additional info

At least values of lib_deps and build_flags in extended and extending sections should be merged. The same may be valid for other multi-value parameters, but I have not had a look at these myself to see if merging makes sense or not.

[ivankravets]

extends NEVER merges "values" of "options". Just merges OPTIONS if they are missed on the upper layer.

[lnlp]

You closed this issue but the info in your reply is lacking from the documentation which was one of the issues I mentioned.
I therefore don't understand why you close this issue without any further actions.

If extends would have been properly documented that would have saved me many hours of figuring things out and would have saved me the efforts of creating an issue.

[ivankravets]

Sorry, reopened! We will improve our docs.

Happy coding with PlatformIO! 😊

[ceewanna]

[featureA]
build_flags = ...
lib_deps = ...

[featureB]
build_flags = ...
lib_deps = ...

[env:release]
extends = featureA, featureB

The way extends work is just knock out featureA and take featureB. There should be way to combine/merge (reporting any conflicts) featureA and featureB, without going to repeating like this.

[env:release]
build_flags = ${featureA.build_flags}, ${featureB.build_flags}
lib_deps = ${featureA.lib_deps}, ${featureB.lib_deps}

Also in this case, the final release just want whether it's asyncweb or syncweb. With extends it is not possible and have to go over the details again.

[env:release]
extends = env:cpu1
???

[env:cpu1]
extends = env:board1
build_flags = ...

[asyncweb]
build_flags =
...

[syncweb]
build_flags =
...

if we can merge, then it's easy, clean and short and simple. If not, it's messy.