Prototype a PlatformIO/SCons state-dump bridge for scripted-project compatibility

[zackees]

Parent: #34

Problem

There may be a practical middle path between a small declarative shim and full SCons hosting: execute PlatformIO/SCons just far enough to resolve extra_scripts, framework/platform scripts, and build graph decisions, dump the resulting state, then let fbuild perform the actual native compile/link/deploy.

Why this is worth testing

• PlatformIO already executes PRE scripts, the main platform build script, and POST scripts before the integration dump stage: .cache/platformio-core/platformio/builder/main.py:167-180, .cache/platformio-core/platformio/builder/main.py:214-227
• PlatformIO exposes construction environments (env, projenv) and post-stage scripts run after the build environment is fully constructed: https://docs.platformio.org/en/latest/scripting/launch_types.html, https://docs.platformio.org/en/latest/scripting/construction_environments.html
• Existing metadata export already proves some state can be serialized (build_type, defines, includes, compile flags, tool paths, targets, flash images): .cache/platformio-core/platformio/builder/tools/piointegration.py:141-173

Core hypothesis

Inject a final POST script (or equivalent harness) that runs after project scripts, serializes the resolved build state to JSON, and exits before actual compilation. fbuild then translates that JSON into a native build spec.

Important caveat

The existing __idedata / IsIntegrationDump() path is probably not sufficient by itself because PlatformIO documents that extra scripts are re-executed during the integration stage and scripts may intentionally return early when env.IsIntegrationDump() is true.

Prototype tasks

1. Create a minimal injected POST script that runs after project scripts
2. Dump at least:
   • BUILD_TYPE
   • tool paths (CC, CXX, AR, linker, objcopy when present)
   • env / projenv scopes: CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS
   • resolved source/object nodes (PIOBUILDFILES if available)
   • program path / linker script / extra flash image info
3. Verify whether this dump reflects common extra_scripts mutations on:
   • env.Append(...)
   • projenv.Append(...)
   • build_unflags
   • generated external sources
4. Assess whether exact compile and link commands can be dumped in addition to flag scopes

Success criteria

• Prototype works on at least one real scripted project and one minimal synthetic fixture
• Dumped state is rich enough for fbuild to reproduce compile and link steps without running pio run for the actual build
• The result clearly states whether this bridge can cover ?simple SCons flag files? in practice

Failure criteria

• If too much behavior is target-sensitive, non-serializable, or only known at actual action execution time, document the blockers and feed that back into platformio.ini advanced build configurations are not yet compatible with PlatformIO/SCons flag workflows #34 as evidence against this path

[zackees]

Expanded implementation notes

I think this feature is feasible as a runtime bridge, not as a parser.

Concrete proposal:

• invoke PlatformIO/SCons normally enough that PRE scripts, framework/platform scripts, library logic, build_unflags, and POST scripts all run
• inject a final dump hook after user scripts
• serialize the resolved state into a BuildSpec
• replay that BuildSpec in fbuild?s native compiler/linker pipeline

Important nuance:

• existing __idedata / project metadata proves that some state is exportable, but it is not automatically sufficient because scripts may branch on integration/service mode
• the bridge should therefore use a dedicated dump mode or injected final script, not rely blindly on IDE metadata

The dump should include, at minimum:

• env/projenv scopes: CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS
• tool paths: CC, CXX, AR, linker, objcopy
• build type, linker script resolution, flash images
• source/object graph if available
• exact compile/link argv if available

If exact argv can be dumped, this becomes much stronger than a flags-only shim and should cover a large class of ?simple SCons flag files?.

The practical target is not arbitrary SCons completeness; it is deterministic extraction of enough resolved state that fbuild can do the real build itself.

[zackees]

Prototype implemented in PR #42. The branch executes supported extra_scripts mutations inside a constrained Python runtime, converts the resulting state into a typed native overlay, and replays that overlay into compile database generation, compilation, and linking.

[zackees]

Implemented and merged in PR #42. The prototype direction proved viable as a constrained runtime bridge: fbuild now runs a narrow script harness for common extra_scripts flag/path mutations, converts the resulting state into native compile/link overlays, and rejects unsupported scripting patterns with a clear fallback recommendation to use --platformio.