Skip to content

Real-world platformio.ini sample reveals remaining native compatibility gaps #43

Closed
#44
Closed
Real-world platformio.ini sample reveals remaining native compatibility gaps#43
#44
@zackees

Description

@zackees
zackees
opened on Apr 13, 2026

Summary

Sampled 25 public GitHub repos containing platformio.ini from GitHub code search on 2026-04-13.

Classification rule used for this sample:

  • analyze each repo's default_envs
  • apply basic [env] inheritance
  • resolve simple ${section.key} references
  • classify against current native fbuild support, not against --platformio

Result in this sample:

  • 8/25 supported by the current native path
  • 17/25 not supported by the current native path

This issue is meant to track real repo instances from the wild, not abstract feature ideas.

What The Sample Suggests

The current runtime bridge is already enough for a meaningful subset of normal projects, especially when extra_scripts only mutate ordinary flag/path/library state.

The remaining misses cluster into a few recurring buckets:

  • Source filtering: src_filter, build_src_filter
  • Dynamic script methods outside the shim: Dump, Execute, BoardConfig, PioPlatform, GetProjectConfig, GetBuildType, env.get, etc.
  • Build middleware / custom SCons logic: AddBuildMiddleware, SConscript, custom methods, custom builders/targets
  • Config-level gaps: build_unflags, build_type=debug
  • Out-of-scope targets: custom platform URLs, non-native host/tool platforms

That means the current bottleneck is no longer basic platformio.ini parsing. It is the boundary between:

  • declarative config
  • simple script-time env/projenv mutations
  • fully dynamic SCons behavior

Highest-Value Follow-Ups From This Sample

  1. Add native support for source filtering.
  2. Add native support for build_unflags.
  3. Add native support for build_type=debug / debug_build_flags.
  4. Decide whether a few additional script APIs are worth shimming:
    • IsCleanTarget
    • IsIntegrationDump
    • PioPlatform
    • read-only BoardConfig
    • limited env.get(...)
  5. Keep hard-failing on heavy SCons behavior:
    • Execute
    • SConscript
    • build middleware
    • custom builder / target logic

Supported Repos In This Sample

These repos appear compatible with the current native support boundary for their default environments:

Unsupported Repos By Main Blocker

Source Filtering

Config-Level Gaps

  • jmamma/MCL
    also uses build_unflags
  • DasBasti/IndiaNavi_Firmware
    feature set: board_build.embed_txtfiles, board_build.partitions, build_flags, build_type
    unsupported surface: build_type=debug

Dynamic Script APIs Outside The Current Shim

  • esprfid/esp-rfid
    feature set: board_build.f_cpu, board_build.flash_mode, build_flags, extra_scripts, lib_deps
    unsupported surface: env.Dump()
  • DutchDevelop/BLLEDController
    feature set: board_build.filesystem, build_flags, extra_scripts, lib_deps
    unsupported surface: BoardConfig, Execute, Flatten, get
  • UtilitechAS/amsreader-firmware
    feature set: board_build.ldscript, build_flags, extra_scripts, lib_deps, lib_ignore
    unsupported surface: Execute, VerboseAction
  • xperiments-in/xtouch
    feature set: board_build.filesystem, board_build.partitions, build_flags, extra_scripts, lib_deps
    unsupported surface: IsCleanTarget, IsIntegrationDump
  • nettigo/namf
    feature set: board_build.f_cpu, board_build.f_flash, board_build.flash_mode, build_flags, extra_scripts, lib_deps
    unsupported surface: env.Dump(), projenv.Dump()
  • S4N-T0S/Open-Source-LDAT
    feature set: build_flags, extra_scripts, lib_deps
    unsupported surface: PioPlatform()
  • cyrilpawelko/m5panel
    feature set: board_build.partitions, build_flags, extra_scripts, lib_deps
    unsupported surface: env.get(), rewriting MKSPIFFSTOOL
  • gicking/stm8gal
    feature set: build_flags, extra_scripts
    unsupported surface: script rewrites PROGNAME

Heavy SCons / Middleware / Custom Builder Logic

Out-Of-Scope Platform / Tool Targets

Full Sample Matrix

Supported

Not Supported

  • esprfid/esp-rfid
    default envs: generic
    feature set: board_build.f_cpu, board_build.flash_mode, build_flags, extra_scripts, lib_deps
    unsupported surface: [generic] extra_script scripts/GENdeploy.py: env.Dump()
  • fbiego/esp32-c3-mini
    default envs: elecrow_c3_1_28
    feature set: board_build.partitions, build_flags, build_src_filter, extends, extra_scripts, lib_deps
    unsupported surface: [elecrow_c3_1_28] uses source filtering
  • DutchDevelop/BLLEDController
    default envs: esp32dev
    feature set: board_build.filesystem, build_flags, extra_scripts, lib_deps
    unsupported surface: [esp32dev] extra_script merge_firmware.py: env.BoardConfig(), env.Execute(), env.Flatten(), env.get()
  • UtilitechAS/amsreader-firmware
    default envs: esp8266
    feature set: board_build.ldscript, build_flags, extra_scripts, lib_deps, lib_ignore
    unsupported surface: [esp8266] extra_script ...generate_includes.py: env.Execute(), env.VerboseAction()
  • xperiments-in/xtouch
    default envs: esp32dev
    feature set: board_build.filesystem, board_build.partitions, build_flags, extra_scripts, lib_deps
    unsupported surface: [esp32dev] extra_script scripts/pre-build.py: env.IsCleanTarget(), env.IsIntegrationDump()
  • W0rthlessS0ul/nRF24_jammer
    default envs: 128x32_Flexible
    feature set: build_src_filter, extra_scripts
    unsupported surface: [128x32_Flexible] uses source filtering
  • jmamma/MCL
    default envs: megacommand, megacmd, tbd
    feature set: board_build.core, board_build.f_cpu, build_flags, build_src_filter, build_unflags, extends, extra_scripts, lib_deps, lib_ignore
    unsupported surface: build_unflags, source filtering, Execute, Exit, env.get, custom platform URL env
  • nettigo/namf
    default envs: lang_pl, lang_en, lang_ro, lang_hu, lang_en_32
    feature set: board_build.f_cpu, board_build.f_flash, board_build.flash_mode, build_flags, extra_scripts, lib_deps
    unsupported surface: env.Dump(), projenv.Dump()
  • gicking/stm8gal
    default envs: linux_x86_64
    feature set: build_flags, extra_scripts
    unsupported surface: unsupported platform linux_x86_64, script rewrites PROGNAME
  • darkspr1te/stm32f107vc_sd_bootloader
    default envs: mkstft_28_bl
    feature set: build_flags, extra_scripts, src_filter
    unsupported surface: src_filter
  • sarfata/kbox-firmware
    default envs: host
    feature set: build_flags, extra_scripts, lib_deps, lib_ignore, src_filter
    unsupported surface: source filtering, projenv.Append on unsupported non-flag key
  • Andy-Big/Marlin_FB_Reborn_Old
    default envs: mks_robin_nano_v1_3_f4_x32, mks_robin_nano35_x16, mks_robin_nano_Sv1_3_f4_x16, mks_robin_nano_SMv1_3_f4_x16
    feature set: build_flags, extra_scripts, lib_deps
    unsupported surface: heavy Marlin SCons script stack
  • S4N-T0S/Open-Source-LDAT
    default envs: teensy41
    feature set: build_flags, extra_scripts, lib_deps
    unsupported surface: env.PioPlatform()
  • DasBasti/IndiaNavi_Firmware
    default envs: esp32dev_debug
    feature set: board_build.embed_txtfiles, board_build.partitions, build_flags, build_type
    unsupported surface: build_type=debug
  • cyrilpawelko/m5panel
    default envs: m5stack-fire
    feature set: board_build.partitions, build_flags, extra_scripts, lib_deps
    unsupported surface: env.get(), rewriting MKSPIFFSTOOL
  • kinsamanka/openplc-stm32-freertos
    default envs: bluepill
    feature set: board_build.ldscript, build_flags, build_src_filter, build_src_flags, extra_scripts
    unsupported surface: source filtering, build middleware, BoardConfig, PioPlatform
  • bigtreetech/BIGTREETECH-SKR-E3-Turbo
    default envs: LPC1769
    feature set: build_flags, extends, extra_scripts, lib_deps, src_filter
    unsupported surface: custom platform URL, source filtering, heavy pre-script logic

Notes

  • Version-pinned platform strings like espressif32@6.9.0 were treated as the same native platform family when the underlying platform is already supported by fbuild.
  • This classification is intentionally based on the native fbuild path. Any of these projects may still be buildable through the --platformio fallback path.
  • This issue should stay focused on feature-family coverage and real repo evidence rather than trying to promise support for every project-specific script.

Action List

TDD RED -> GREEN Execution Plan

  • Source filtering
    • RED: add parser tests for build_src_filter / src_filter resolution
    • RED: add scanner tests for +<...> / -<...> matching on sketch sources
    • GREEN: implement config getter and filter parsing
    • GREEN: apply filter to sketch source discovery in native builds
    • GREEN: rerun targeted tests and record the passing set
  • build_unflags
    • RED: add tests covering flag removal from compile and link overlays
    • GREEN: implement unflag removal in native flag resolution
    • GREEN: rerun targeted tests and record the passing set
  • build_type=debug / debug_build_flags
    • RED: add config + effective-flag tests for debug envs
    • GREEN: implement native debug-mode flag resolution
    • GREEN: rerun targeted tests and record the passing set
  • Small script API shims to evaluate individually
    • RED: add focused harness tests for IsCleanTarget
    • RED: add focused harness tests for IsIntegrationDump
    • RED: add focused harness tests for read-only BoardConfig
    • RED: add focused harness tests for limited env.get(...)
    • GREEN: implement only the APIs that stay read-only / deterministic
  • Remain explicit non-goals for the native path
    • keep hard-failing on Execute
    • keep hard-failing on SConscript
    • keep hard-failing on build middleware / custom builders / custom targets

Repo-Driven Success Criteria

  • Move at least one source-filter repo from the unsupported sample into the supported bucket
  • Re-run the sample classification after each feature-family lands
  • Keep the issue updated with exact repo instances unlocked by each change

Progress

  • build_src_filter / src_filter native support is now implemented as the first TDD slice.

  • RED was confirmed with missing get_source_filter and missing filtered scanner APIs.

  • GREEN was confirmed with targeted tests:

    • cargo test -p fbuild-config --lib test_get_source_filter -- --nocapture
    • cargo test -p fbuild-build --lib test_scan_sketch_sources_filtered -- --nocapture

  • On this workstation, test execution required pinning the MSVC toolchain directly because external cargo/rustc shims on PATH were mixing GNU and MSVC artifacts. The code itself is green under the correct toolchain.

  • build_unflags is the next TDD slice in progress.

  • Plan: add failing config/build tests first, implement native unflag removal, rerun targeted tests to green, then move to build_type=debug / debug_build_flags.

  • build_unflags is now implemented natively for the common subset.

  • RED was confirmed with missing get_build_unflags and missing native unflag application.

  • GREEN was confirmed with targeted tests:

    • cargo test -p fbuild-config --lib test_get_build_unflags -- --nocapture
    • cargo test -p fbuild-build --lib test_apply_build_unflags -- --nocapture

  • Current behavior removes exact tokens plus common option/value pairs from both build_flags and build_src_flags, which matches the highest-value native subset.

  • Next slice in progress: build_type=debug and debug_build_flags.

  • build_type=debug and debug_build_flags are now supported for the common native subset.

  • RED was confirmed with missing get_build_type, missing get_debug_build_flags, and missing native debug-mode application.

  • GREEN was confirmed with targeted tests:

    • cargo test -p fbuild-config --lib test_get_build_type_defaults_to_release -- --nocapture
    • cargo test -p fbuild-config --lib test_get_build_type_reads_debug -- --nocapture
    • cargo test -p fbuild-config --lib test_get_debug_build_flags_uses_platformio_defaults -- --nocapture
    • cargo test -p fbuild-config --lib test_get_debug_build_flags_reads_ini_override -- --nocapture
    • cargo test -p fbuild-build --lib test_apply_debug_build_type_replaces_opt_flags_and_adds_debug_define -- --nocapture
    • cargo test -p fbuild-build --lib test_debug_mode_then_build_unflags_can_remove_debug_flags_again -- --nocapture

  • Current behavior adds -D__PLATFORMIO_BUILD_DEBUG__, applies default PlatformIO debug flags when unspecified (-Og -g2 -ggdb2), cleans conflicting optimization/debug levels in the native flag vectors, and still allows build_unflags to remove those debug flags afterward.

  • Added a small extra_scripts shim set for common no-op/introspection helpers: Dump, IsCleanTarget, IsIntegrationDump, Flatten, VerboseAction, and Execute.

  • RED was confirmed with runtime tests failing on unsupported env.Dump and env.IsCleanTarget calls.

  • GREEN was confirmed with targeted tests:

    • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_dump_shim -- --nocapture
    • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_common_noop_scons_helpers -- --nocapture

  • This does not widen native support to arbitrary script execution; it only removes a handful of low-risk blockers that commonly appear in simple PlatformIO scripts.

  • Added native BoardConfig() and PioPlatform() shims to the extra-scripts runtime for common metadata lookups.

  • GREEN was confirmed with targeted tests:

    • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_board_config_shim -- --nocapture
    • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_pio_platform_shim -- --nocapture

  • PioPlatform() remains intentionally minimal: platform name, is_embedded(), and get_package_dir(...). More dynamic platform/package behaviors still fall back to --platformio.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions