Skip to content

Remove environment name from build_cache_dir cache keys #4574

Open
@LordMike

Description

@LordMike

What kind of issue is this?

  • Feature Request.
    Start by telling us what problem you’re trying to solve. Often a solution
    already exists! Don’t send pull requests to implement new features without first getting our
    support. Sometimes we leave features out on purpose to keep the project small.

Configuration

Operating system: docker/linux, host is Debian 10, image is ghcr.io/esphome/esphome-hassio:2023.3.0
PlatformIO Version (platformio --version): PlatformIO Core, version 6.1.6

Description of problem

I'm using esphome to manage tens of mostly identical devices with identical sets of source code sans a few items such as their individual name and some encryption keys. When I use esphome to compile the firmwares for these devices, they create a separate environment in platformio for each device, which is good for isolation. It does take a long time to compile the mostly identical source code though, for all these devices, when updates are available. I'd like to alleviate that.

In another issue, I've been digging in to what can be done, and I've experimented with the build_cache_dir setting. It's not currently used by esphome. This setting works wonders for a single device: if I recompile after cleaning the output, the build is instantaneous (as can be), as it uses the cache.

The cached output from one device is not used in another, and I've found this topic (https://community.platformio.org/t/build-cache-dir-will-not-share-object-files-between-envs/10011/8) on your community forums that provides a crucial hint. It seems the env:name is part of the cache key, and as each device in esphome is a separate environment, I have my issue.

I hope this issue can lead to the env:name being removed from the cache key, to allow the build cache to be reused across environments. It's my impression that the rest of the build commandline includes all the details that are in environments, such as boards, platforms and libraries.

Steps to Reproduce

  1. Create two environments of identical board, platform, dependencies, code
  2. Set build_cache_dir to some directory
  3. Build the two environments
  4. Observe that the identical code is compiled twice

Actual Results

The cache is not reused between identical builds, if they have two environments.

Expected Results

Environment name should not affect the cache key. It's my impression that the rest of the build commandline (which is part of the key) is enough to distinguish one output from another.

If problems with PlatformIO Build System:

The content of platformio.ini:
This file is modified by me, locally, to include the cache_build_dir. It is identical to other devices I have, except for the environment name.

; Auto generated code by esphome

[common]
lib_deps =
build_flags =
upload_flags =

; ========== AUTO GENERATED CODE BEGIN ===========
[platformio]
description = ESPHome 2023.3.0
build_cache_dir = /esphome_cache/
[env:light-extra01]
board = esp8285
board_build.flash_mode = dout
board_build.ldscript = eagle.flash.1m.ld
build_flags =
    -DESPHOME_LOG_LEVEL=ESPHOME_LOG_LEVEL_INFO
    -DNEW_OOM_ABORT
    -DPIO_FRAMEWORK_ARDUINO_LWIP2_HIGHER_BANDWIDTH_LOW_FLASH
    -DUSE_ARDUINO
    -DUSE_ESP8266
    -DUSE_ESP8266_FRAMEWORK_ARDUINO
    -DUSE_STORE_LOG_STR_IN_FLASH
    -Wno-nonnull-compare
    -Wno-sign-compare
    -Wno-unused-but-set-variable
    -Wno-unused-variable
    -fno-exceptions
extra_scripts =
    post:post_build.py
framework = arduino
lib_deps =
    ottowinter/ESPAsyncTCP-esphome@1.2.3
    esphome/ESPAsyncWebServer-esphome@2.1.0
    DNSServer
    ESP8266WiFi
    ESP8266mDNS
    esphome/noise-c@0.1.4
    bblanchon/ArduinoJson@6.18.5
    ${common.lib_deps}
lib_ldf_mode = off
platform = platformio/espressif8266 @ 3.2.0
platform_packages =
    platformio/framework-arduinoespressif8266 @ ~3.30002.0
; =========== AUTO GENERATED CODE END ============

Additional info

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions