Skip to content

Commit

Permalink
ESP-IDF v5.1 (#7733)
Browse files Browse the repository at this point in the history
* Initial changes to compile under ESP-IDF v5.1

* Initial import for ESP-IDF v5.1 libs

* Update toolchain

* Update esp32-hal-psram.c

* Add missing LDs

* Update platform.txt

* Stop some CI jobs, because they will always fail

* Fix examples

* Update app_httpd.cpp

* Update ResetReason.ino

* Warnings fixes

* Added the example guideline and template (#7665)

* Added the example guideline and template

* PR review changes with some typos and grammar fixes

* Changes according to the PR review

* Added ESP32-S3 link to the datasheet (#7738)

* Update HiFreq_ADC.ino

* Replace periph_ctrl.h use because of deprecation

* Replace esp_spi_flash.h use because of deprecation

* Add includes to male mDNS::enableWorkstation compile

* Fix ssl_client mbedtls_pk_parse_key callback

* Update temperature sensor driver

* Allow sketch_utils to compile with arduino-cli

* Run CI with arduino-cli

* Fix arduino-cli CI build on Windows

* Refactor platform.txt to not use components installed through the board manager when running from git

* Initial Peripheral Manager Implementation

* Update SigmaDelta driver to use the new ESP-IDF driver API

* Small improvements to peripheral manager and SigmaDelta

* Remove deleted function from SigmaDelta header

* Update DAC driver to use the new ESP-IDF driver API

* Adds softAp(String) to make it compatible with ESP8266 (#7801)

* Fix commentary (#7800)

Minor fix based on observation done in #7795 (comment)

* add adafruit new board feather esp32s2 reserve tft (#7794)

* bugfix: add <stdint.h> for uint8_t to avoid compilation failure (GCC 11.2.0) (#7744)

* Adding 3rd party boards for VALTRACK-V4-VTS-ESP32-C3 & VALTRACK-V4-MFW-ESP32-C3 (#7735)

* Added VALTRACK-V4-VTS-ESP32-C3 board definition

Created pins_arduino.h & made changes to boards.txt with necessary changes

* Modified the URL

* Renamed json

* renamed all auRL

* Adding VALTRACK-V4 series board definitions

Added VALTRACK-V4-VTS-ESP32C3 & VALTRACK-V4-MFW-ESP32-C3 board variants

* Adding VALTRACK-V4 series board definitions

Added VALTRACK-V4-VTS-ESP32C3 & VALTRACK-V4-MFW-ESP32-C3 board variants

* Reverted package_esp32_index.template.json

restored package_esp32_index.template.json from edits

* Reverted package_esp32_index.template.json

Added new line to package_esp32_index.template.json

* Update Platformio CI (#7725)

* WiFiClient example fix (#7711)

* Modified WiFiClient example to use thingspeak instead of non-functionig sparkfun

* Moved instructions to README

* Fixed spelling

* Added link to S3 datasheet

---------

Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>

* Mirror update from Heltec repository (#7709)

Heltec updated the I2C pins in Heltec-Aaron-Lee/WiFi_Kit_series@b10f4bf

* Fixes BLE data printing (#7699)

* Fixes BLE data printing

BLE data has no '\0' terminator, therefore it can't be printed as a regular C string.
This fix just prints the BLE data based on its length.

* Simplify printing to a single call

* split menu options + lora_32_V3 fix (#7697)

* Change header gaurd name (#7696)

* Fix Name (#7691)

Wrong name in definitions.

* Fix error in WiFiClient.cpp where the connect function fails for timeouts below 1 second (#7686)

* Update WiFiClient.cpp

This change will allow specifying connect timeouts below 1 second. Without this change, if connect timeouts under 1 second are given, the connect defaults to 0ms and fails. 
This will also allow timeouts in fractions of seconds, e.g. 1500ms. Without this change, connect timeouts are truncated to full second increments.

* Make parameter timeout_ms clear

* Change connection timeout_ms name for clarity

---------

Co-authored-by: Rodrigo Garcia <rodrigo.garcia@espressif.com>

* fixed the function header (#7674)

* fixed the function header

* fixed function name and paramaters

---------

Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>

* Ticker fix solving #6155 (#7664)

* Wrapped Ticker functions with #pragma disabling -Wcast-function-type

* Revert "Wrapped Ticker functions with #pragma disabling -Wcast-function-type"

This reverts commit 160be7e.

* Fixed Ticker example

* Modified Ticker example

* Fixed LED_BUILTIN err for ESP32

---------

Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>

* setPins fix ESP32 "specified pins are not supported by this chip." (#7646)

[ESP32: SDMMCFS::begin hardcodes the usage of slot 1, only check if the pins match slot 1 pins.]

setPins() was testing pins D1, D2 and D3 all against D1 ... fine in 1 pin mode when all are -1 not so much if you're trying to get 4 pin mode working.
I now see this function doesn't really do anything on the ESP32...accept now correctly checks that you are trying to use the slot 1 pins.

Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>

* Allow passing IP as connect method parameter in WiFiClientSecure and skip unnecessary host-ip conversions (#7643)

* Add LED_BUILTIN* definitions and initialization for LEDs to stop them floating. (#7636)

Co-authored-by: Rodrigo Garcia <rodrigo.garcia@espressif.com>

* Expand path to tinuf2 image when checking existence in platformio-build.py (#7631)

* Expand path to tinuf2 image when checking existence

* More isFiles fixed

* Remove (useless) trailing semicolon from Print.cpp (#7622)

* ADD: New variant Edgebox-ESP-100 (#7771)

* ADD: New variant Edgebox-ESP-100

* FIX: Edgebox-ESP-100 Board.txt usb mode option change back to default value as ESP32S3

* Add Crabik Slot ESP32-S3 board (#7790)

* Added Crabik Slot ESP32-S3

* Adding CPU frequency settings and removing excess from partition scheme settings

* new variant LilyGO T-Display-S3 (#7763)

* new variant LilyGO T-Display-S3

https://github.com/Xinyuan-LilyGO/T-Display-S3

* Add boards.txt definition

---------

Co-authored-by: Me No Dev <me-no-dev@users.noreply.github.com>

* Update get.py to support Apple ARM64

* Update package_esp32_index.template.json

* WString Return bool (#7774)

* Add Roboheart Hercules development board to the esp32-core (#7672)

* added Roboheart Hercules pin definitions and board.txt entries

* added package_roboheat.json for prototyping

* Roboheart Hercules pins

* Updated the pins

* Delete package_roboheart.json

* Requested changes

---------

Co-authored-by: renebohne <rene.bohne@gmail.com>

* Reword "ESP-IDF as Component" (#7812)

I think "Arduino as an ESP-IDF component" or just "As ESP-IDF component" instead of  "ESP-IDF as Component" is more correct way to name the link.

1. "ESP-IDF as Component" would imply that ESP-IDF is some sort of library for Arduino, which is (IMO) misleading, because it's true the other way around.
2. It's written as "Arduino as an ESP-IDF component" on the webpage it points to as well.

- Also I removed the capitalization from "Component" as I have not found a reason why is it capitalized.

* add new board Adafruit Feather ESP32-S3 Reverse TFT (#7811)

* Multi threading examples (tasks, queues, semaphores, mutexes) (#7660)

* Moved and renamed example ESP32/FreeRTOS to MultiThreading/BasicMultiThreading

* Added dummy files

* Modified original example

* Fixed BasicMultiThreading.ino

* Added Example demonstrating use of queues

* Extended info in BasicMultiThreading

* Renamed Queues to singular Queue

* Added Mutex example

* Added Semaphore example

* Moved info from example to README

* Moved doc from Mutex to README

* Added Queue README

* Removed unecesary text

* Fixed grammar

* Increased stack size for Sempahore example

* Added headers into .ino files

* Added word Example at the end of title in README

* removed unused line

* Added forgotten README

* Modified BasicMultiThreading example

* Added missing S3 entry in README

* moved location

* Update ESP-IDF libs

* Update CMakeLists.txt

* Update esptool to v4.4

* Add function timerAttachInterruptFlag (#7809)

* Update esptool to v4.5

* ADC refactoring (#7827)

* Adc refactored + periman implementation

Peripheral manager still needs to be checked if the implementation is right.

* switched to working solution for milivolts read

* Periman detachbus fix

* coding style

* fix CI warnings

* fix FreeRTOS example

* Update ETH.cpp

* Update FunctionalInterruptStruct.ino

* Update package_esp32_index.template.json

* Update package_esp32_index.template.json

* Fixes for the latest IDF v5.1

* update esp-idf libs and toolchain

* Turn OFF auto crystal frequency for ESP32 (needed by TWAI)

* Update examples

* Switch build to mostly use flags from files

Includes can not be done this way

* Reorganize flag files

* Optimize chip build flags further

* Revert defines from file. MBEDTLS_CONFIG_FILE does not properly expand

* Add support for includes and defines from file

* Replace old sdk path references in platform.txt

* use gcc-ar (#8013)

* Makes F_CPU generic for all SoC (#8007)

Based on CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ that is correctly defined in the sdkconfig file for each SoC.

* TIMER refactoring (#7904)

* refactor using GPtimer

* Updated timer HW test

* fix examples

* Add v2.0.7 in issue template (#7871)

* refactor using GPtimer

* Updated timer HW test

* fix examples

* Revert "Add v2.0.7 in issue template (#7871)"

This reverts commit fcc3b17.

* Update upload-artifact to v3 in HW CI

* Revert "Update upload-artifact to v3 in HW CI"

This reverts commit 1ba2280.

* replace resolution with frequency

* remove count_down option

* countup removed from examples + header

* Refactored timer object

* code cleanup + examples and tests fixes

* TimerAlarm fix

---------

Co-authored-by: Vojtěch Bartoška <76958047+VojtechBartoska@users.noreply.github.com>

* [Docs] ADC and Timer API Update (+some docs fixes) (#7906)

* updated docs

* remove hall sensor docs

Removed Hall sensor documentation as its no longer supported in IDF-5

* Fixed ESPNow examples location in docs

* Last timer refactored API + gpio small fix

* AlarmWrite fix

* Fixes APLL/PLL with RTC Frequency (#8025)

log_d() was displaying APLL for any SoC, but S3 and C3 has not such option, causing compilation errors.

* Update IDF libs and fix OPI PSRAM on S3

* Add setMode function HardwareSerial.c to set the esp32 uart mode for use with RS485 auto RTS (#7935)

* Added setMode function to set the esp32 uart mode

Used to set the esp32 uart mode for use with RS485 Half Duplex and the auto RTS pin mode. This will set/clear the RTS pin output to control the RE/DE pin on most RS485 chips.

* Add Success (bool) return in some functions

* Add Success (bool) return code to some functions

* Add Success (bool) return to some functions

* Add Success (bool) return to some functions

* Fix uartSetRxTimeout return type

---------

Co-authored-by: Rodrigo Garcia <rodrigo.garcia@espressif.com>

* Add support for esp-elf-gdb

* WFG Crashfix (#8044)

* Update component libs

* IDF release/v5.1 (#8061)

* IDF release/v5.1 bb9200acec

* Update Esp.cpp

* IDF release/v5.1 420ebd208a

* Update esp32-hal-psram.c

* Switch SDK to be an external package

* fix path (#8096)

* Makes UART work at any APB Frequency (#8097)

Fixes HardwareSerial to work with IDF 5.1 on any CPU/APB Frequency (240MHz to 10MHZ), including user created low power modes.

* Add required callbacks for TinyUSB DFU

* Update version to 3.0.0

* Add ESP.getCoreVersion() and update ESP.getChipModel()

* Update timer hal for the latest 5.1

* Use separate RX and TX buffer sizes in HTTP client

optimizes download by allowing up to 4K packets to be received

* Rename clock tree enum name in latest 5.1

* ESP32-H4 support was removed in ESP-IDF v5.1

* IDF release/v5.1 2004bf4e11 (#8165)

* Deinit previous bus first (#8180)

* TIMER - add timer_started flag, fix timerEnd() + timer HW test (#8135)

* Add timer_started flag and stop before disable

* Fix timer HW test

* TOUCH - Peripheral manager implementation (#8129)

* Touch periman implemented

* Deinit previous bus first

* LEDC Refactoring - Peripheral manager implemented (#8126)

* LEDC periman implementation

* Fix examples

* Rework tone

* Update ledc docs

* fix missing bracket

* Update analog funtions esp32-hal.h

* Update CameraWebServer example

* Fix HiFreq_ADC example

* minor fixes - typos

* Avoid calling tone/notone when tone already runs on dif. pin

* Remove unused channels_resolution

* GPIO - Peripheral manager implementation (#8179)

* periman-implementation

* fix RGB_BUILTIN and remove space

* Enforces more consistency into Peripheral Manager (#8188)

* Avoid log_i() message the first time a bus is assigned

* Prevent operation with ESP32_BUS_TYPE_INIT

* keeps coding style

* do not print messages on INIT bus type

* [Arduino Core 3.0.0] RMT IDF5.1 Refactoring (#7994)

* RMT IDF5.1 refactoring

* Fixes initial value setting

* removed rmtRead() with user callback

* simplify/remove Read data structure

* Deep API simplification

* fixes the examples

* fix rmt.h

* adds support to APB different frequencies

* fixes CI and not defined RGB_BUILTIN

* new RMT API and examples

* fixing commentaties

* Update esp32-hal-rgb-led.c

* changes Filter API

* Fixes example with Filter API

* Update PlatformIO scripts for the upcoming 3.0 core (#8183)

* Update PlatformIO scripts for the upcoming 3.0 core

* Dynamically select proper framework-arduinoespressif32-libs package

With this change the dev-platform will be dynamically configured to
pull the latest .zip package with precompiled libraries from extracted from
package_esp32_index.template.json

* free memory on detach (#8264)

* SPI - Peripheral manager implementation  (#8255)

* spi periman implementation

* fix header file

* remove unused struct

* fix missing braces

* Update esp32-hal-rmt.c (#8216)

Optimizing Peripheral Manager Test

* I2C - Peripheral manager implementation (#8220)

* i2c-master periman initial commit

* i2c-master make detachbus static + comment remove

* i2c-slave periman implementation

* SetPinBus to INIT on i2cDeinits

* Fix slave pins deinit

* remove dbg logs

* set ret to ESP_FAIL instead of returning

* Fix warnings in hal-spi caused by pariman transition

* Update esptool.py to version 4.6

* Add platform support for ESP_SR

* Add USB Type and valid pin check to periman

* replace bus with spi->num+1 (#8279)

* Remove default pins from SPI HAL

* Add commented out handlers for esptool.js in TinyUSB CDC

For future use

* Add build defines for host os and fqbn (for debug purposes)

* Provide proper memory caps total size

* Update Esp.cpp

* SDMMC - Peripheral manager implementation (#8289)

* sdmmc periman implemented

* save pins when SOC_SDMMC_USE_IOMUX

* IDF release/v5.1 4bc762621d (#8292)

* Adds missing pinMode (#8312)

* Adds missing pinMode

The example code lacks a pinMode() to initialize the GPIO 0 (button). In Arduino Core 3.0.0, it prints an error message when trying to read a not initialized GPIO.

* Update KeyboardLogout.ino

Adds <buttonPin> to keep code standard

* Update KeyboardReprogram.ino

Adds <buttonPin> to keep code standard

* LEDC Fade implementation (#8338)

* fade API + pointer fixes

* Add fade api

* Add fade example

* update ledc docs

* remove unused variables

* fix path to example

* Adds USB to Peripheral Manager - Arduino Core 3.0.0 (#8335)

* ETHERNET - Peripheral manager implementation (#8297)

* Peripheral manager implemented

* remove unused variable

* Add all RMII pins

* fix typo

* Adds HardwareSerial to Peripheral Manager Arduino 3.0.0 (#8328)

* Do not limit ETHERNET in periman to only ESP32. SPI is also an option

* Initial support for ESP32-C6 (#8337)

* Add checks for SOC defines (#8351)

* Add checks for SOC defines

* Add SoC checks to BLE library

* fix i2c compilation error

* fix wrong placement of include

* add check to SPI library

* add check to USB library

* add checks to Wire library

* Feature/esp32h2 support (#8373)

* Initial support for ESP32H2

* Additional changes for ESP32H2

* Update libs for ESP32H2

* Fix flashing on ESP32-H2

* Fix GPIO Configs for ESP32-C6 and ESP32-H2

* Update Timer test sketch

* Fix upload flash parameters

* Use ets_write_char_uart instead of ets_printf in log_printfv

* Print full chip report when log level is sufficient (#8282)

* ESP32-C3 does not have ets_write_char_uart

* Fix BLE gap event name

* HW Testing - Pytest update (#8389)

* update tests requirements

* remove already handled components

* Update version of pytest

* Add missing ESP32-H2 to hil.yml

* Updated FreeRTOS names (#8418)

* HW Testing -  ESP32-C6 + ESP32-H2 fixes (#8404)

* add C6/H2 to tests cfg.json

* remove ,

* workflow runs-on runner by matrix

* Add need for arduino tag to select runner

* Add cryptography to requirements.txt

* Removed duplicate TX1 define for H2 (#8402)

* Fix broken examples

* Fixes RMT filter & idle timing and setup (#8359)

* Fixes Filter and Idle parameter to uint32

* Fixes Filter and Idle setup

* Fixes it to 5.1Libs branch

* fix RMT CLK source and Filter API

* fixes missing ;

* fixes missing ;

* fixes RMT example

* IDF release/v5.1 a7b62bbcaf (#8438)

* Add workflow to build executables from python scripts (#8290)

* Add workflow to build executables from python scripts

* Push binary to tools

* Enable executable signing on Windows

* Update get.py

* Push binary to tools

* Try with multiple files

* Try more actions

* Try powershell

* Restore tools so they do not get rebuilt

* Finalize scripts

* Push binary to tools

* App rollback should be after PSRAM is initialized

* Correcting RX1 to GPIO4 and TX1 to GPIO5 to be consistent with documentation. Previous pin use works but is inconsistent with C6 docs.

* Fixes Memory Leak (#8486)

* fixes preprocessor test (#8485)

* fixes preprocessor test

When using `#define USE_SOFT_AP` 
Change
`&& not USE_SOFT_AP` ==> `&& !defined(USE_SOFT_AP)`

* Adds any BLE capable device in WiFiProv.ino

Removing ESP32 restriction for BLE Provisioning.

* fix flash mode read out for C6

* Add option for custom partitions without restrictions

* SD_MMC update (#8298)

* Updated SD_MMC lib and examples

* Removed getter implementation and commented usage in examples

* squashed updates

* IDF release/v5.1 f0437b945f (#8599)

* Update package_esp32_index.template.json

* Fix printf format build error in BTAdvertisedDeviceSet.cpp

---------

Co-authored-by: Pedro Minatel <pedro.minatel@espressif.com>
Co-authored-by: Rodrigo Garcia <rodrigo.garcia@espressif.com>
Co-authored-by: Ha Thach <thach@tinyusb.org>
Co-authored-by: Martin Turski <quiret@vfemail.net>
Co-authored-by: raviypujar <raviypujar@gmail.com>
Co-authored-by: Jason2866 <24528715+Jason2866@users.noreply.github.com>
Co-authored-by: Tomáš Pilný <34927466+PilnyTomas@users.noreply.github.com>
Co-authored-by: Jan Procházka <90197375+P-R-O-C-H-Y@users.noreply.github.com>
Co-authored-by: Daniel Berlin <dberlin@dberlin.org>
Co-authored-by: Nima Askari (نیما عسکری) <nimaltd@yahoo.com>
Co-authored-by: rtpmsys <106180646+rtpmsys@users.noreply.github.com>
Co-authored-by: bytiful <55647551+bytiful@users.noreply.github.com>
Co-authored-by: tmfarrington <tmfarrington@users.noreply.github.com>
Co-authored-by: Krzysiek S <chris.streh@gmail.com>
Co-authored-by: surt <carl.olsson@gmail.com>
Co-authored-by: Max Scheffler <max.scheffler@pm.me>
Co-authored-by: Clemens Kirchgatterer <clemens@1541.org>
Co-authored-by: Peter Pan's Techland <twinkle-pirate@hotmail.com>
Co-authored-by: Roman <programmeofficemilkyway@gmail.com>
Co-authored-by: Eistee <Eistee82@users.noreply.github.com>
Co-authored-by: David McCurley <44048235+mrengineer7777@users.noreply.github.com>
Co-authored-by: Gaya3N25 <30388176+Gaya3N25@users.noreply.github.com>
Co-authored-by: renebohne <rene.bohne@gmail.com>
Co-authored-by: Olivér Remény <25034625+remenyo@users.noreply.github.com>
Co-authored-by: davidk88 <david.kotar@gmail.com>
Co-authored-by: Vojtěch Bartoška <76958047+VojtechBartoska@users.noreply.github.com>
Co-authored-by: James Armstrong <jamesarmstrong3@me.com>
Co-authored-by: Valerii Koval <valeros@users.noreply.github.com>
Co-authored-by: Peter G. Jensen <root@petergjoel.dk>
  • Loading branch information
1 parent df287cc commit 6f7a1ca
Show file tree
Hide file tree
Showing 9,393 changed files with 6,859 additions and 2,097,902 deletions.
The diff you're trying to view is too large. We only load the first 3000 changed files.
78 changes: 78 additions & 0 deletions .github/pytools/Sign-File.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
[CmdletBinding()]
param (
[Parameter()]
[String]
$Path
)


function FindSignTool {
$SignTool = "signtool.exe"
if (Get-Command $SignTool -ErrorAction SilentlyContinue) {
return $SignTool
}
$SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\x64\signtool.exe"
if (Test-Path -Path $SignTool -PathType Leaf) {
return $SignTool
}
$SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\x86\signtool.exe"
if (Test-Path -Path $SignTool -PathType Leaf) {
return $SignTool
}
$sdkVers = "10.0.22000.0", "10.0.20348.0", "10.0.19041.0", "10.0.17763.0"
Foreach ($ver in $sdkVers)
{
$SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\${ver}\x64\signtool.exe"
if (Test-Path -Path $SignTool -PathType Leaf) {
return $SignTool
}
}
"signtool.exe not found"
Exit 1
}

function SignEsptool {
param(
[Parameter()]
[String]
$Path
)

$SignTool = FindSignTool
"Using: $SignTool"
$CertificateFile = [system.io.path]::GetTempPath() + "certificate.pfx"

if ($null -eq $env:CERTIFICATE) {
"CERTIFICATE variable not set, unable to sign the file"
Exit 1
}

if ("" -eq $env:CERTIFICATE) {
"CERTIFICATE variable is empty, unable to sign the file"
Exit 1
}

$SignParameters = @("sign", "/tr", 'http://timestamp.digicert.com', "/td", "SHA256", "/f", $CertificateFile, "/fd", "SHA256")
if ($env:CERTIFICATE_PASSWORD) {
"CERTIFICATE_PASSWORD detected, using the password"
$SignParameters += "/p"
$SignParameters += $env:CERTIFICATE_PASSWORD
}
$SignParameters += $Path

[byte[]]$CertificateBytes = [convert]::FromBase64String($env:CERTIFICATE)
[IO.File]::WriteAllBytes($CertificateFile, $CertificateBytes)

&$SignTool $SignParameters

if (0 -eq $LASTEXITCODE) {
Remove-Item $CertificateFile
} else {
Remove-Item $CertificateFile
"Signing failed"
Exit 1
}

}

SignEsptool ${Path}
Binary file added .github/pytools/espressif.ico
Binary file not shown.
14 changes: 10 additions & 4 deletions .github/scripts/install-platformio-esp32.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
export PLATFORMIO_ESP32_PATH="$HOME/.platformio/packages/framework-arduinoespressif32"
PLATFORMIO_ESP32_URL="https://github.com/platformio/platform-espressif32.git"

TOOLCHAIN_VERSION="8.4.0+2021r2-patch5"
ESPTOOLPY_VERSION="~1.40400.0"
TOOLCHAIN_VERSION="12.2.0+20230208"
ESPTOOLPY_VERSION="~1.40501.0"
ESPRESSIF_ORGANIZATION_NAME="espressif"

echo "Installing Python Wheel ..."
Expand All @@ -30,9 +30,15 @@ replace_script+="data['packages']['toolchain-riscv32-esp']['owner']='$ESPRESSIF_
# Update versions to use the upstream
replace_script+="data['packages']['toolchain-xtensa-esp32']['version']='$TOOLCHAIN_VERSION';"
replace_script+="data['packages']['toolchain-xtensa-esp32s2']['version']='$TOOLCHAIN_VERSION';"
replace_script+="data['packages']['toolchain-xtensa-esp32s3']['version']='$TOOLCHAIN_VERSION';"
replace_script+="data['packages']['toolchain-riscv32-esp']['version']='$TOOLCHAIN_VERSION';"
# Add ESP32-S3 Toolchain
replace_script+="data['packages'].update({'toolchain-xtensa-esp32s3':{'type':'toolchain','optional':True,'owner':'$ESPRESSIF_ORGANIZATION_NAME','version':'$TOOLCHAIN_VERSION'}});"
# Add new "framework-arduinoespressif32-libs" package
# Read "package_esp32_index.template.json" to extract a url to a zip package for "esp32-arduino-libs"
replace_script+="fpackage=open(os.path.join('package', 'package_esp32_index.template.json'), 'r+');"
replace_script+="package_data=json.load(fpackage);"
replace_script+="fpackage.close();"
replace_script+="libs_package_archive_url=next(next(system['url'] for system in tool['systems'] if system['host'] == 'x86_64-pc-linux-gnu') for tool in package_data['packages'][0]['tools'] if tool['name'] == 'esp32-arduino-libs');"
replace_script+="data['packages'].update({'framework-arduinoespressif32-libs':{'type':'framework','optional':False,'version':libs_package_archive_url}});"
replace_script+="data['packages']['toolchain-xtensa-esp32'].update({'optional':False});"
# esptool.py may require an upstream version (for now platformio is the owner)
replace_script+="data['packages']['tool-esptoolpy']['version']='$ESPTOOLPY_VERSION';"
Expand Down
4 changes: 4 additions & 0 deletions .github/scripts/on-push.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ if [ "$BUILD_PIO" -eq 0 ]; then
FQBN_ESP32S2="espressif:esp32:esp32s2:PSRAM=enabled,PartitionScheme=huge_app"
FQBN_ESP32S3="espressif:esp32:esp32s3:PSRAM=opi,USBMode=default,PartitionScheme=huge_app"
FQBN_ESP32C3="espressif:esp32:esp32c3:PartitionScheme=huge_app"
FQBN_ESP32C6="espressif:esp32:esp32c6:PartitionScheme=huge_app"
FQBN_ESP32H2="espressif:esp32:esp32h2:PartitionScheme=huge_app"

SKETCHES_ESP32="\
$ARDUINO_ESP32_PATH/libraries/WiFiClientSecure/examples/WiFiClientSecure/WiFiClientSecure.ino\
Expand All @@ -85,6 +87,8 @@ if [ "$BUILD_PIO" -eq 0 ]; then
build "esp32s3" $FQBN_ESP32S3 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32
build "esp32s2" $FQBN_ESP32S2 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32XX
build "esp32c3" $FQBN_ESP32C3 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32XX
build "esp32c6" $FQBN_ESP32C6 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32XX
build "esp32h2" $FQBN_ESP32H2 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32XX
build "esp32" $FQBN_ESP32 $CHUNK_INDEX $CHUNKS_CNT $SKETCHES_ESP32
else
source ${SCRIPTS_DIR}/install-platformio-esp32.sh
Expand Down
4 changes: 3 additions & 1 deletion .github/scripts/on-release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,6 @@ cp -f "$GITHUB_WORKSPACE/tools/gen_insights_package.py" "$PKG_DIR/tools/"
cp -f "$GITHUB_WORKSPACE/tools/gen_insights_package.exe" "$PKG_DIR/tools/"
cp -Rf "$GITHUB_WORKSPACE/tools/partitions" "$PKG_DIR/tools/"
cp -Rf "$GITHUB_WORKSPACE/tools/ide-debug" "$PKG_DIR/tools/"
cp -Rf "$GITHUB_WORKSPACE/tools/sdk" "$PKG_DIR/tools/"
cp -f $GITHUB_WORKSPACE/tools/platformio-build*.py "$PKG_DIR/tools/"

# Remove unnecessary files in the package folder
Expand All @@ -197,10 +196,13 @@ find "$PKG_DIR" -name '*.git*' -type f -delete
echo "Generating platform.txt..."
cat "$GITHUB_WORKSPACE/platform.txt" | \
sed "s/version=.*/version=$ver$extent/g" | \
sed 's/tools.esp32-arduino-libs.path={runtime.platform.path}\/tools\/esp32-arduino-libs/tools.esp32-arduino-libs.path=\{runtime.tools.esp32-arduino-libs.path\}/g' | \
sed 's/tools.xtensa-esp32-elf-gcc.path={runtime.platform.path}\/tools\/xtensa-esp32-elf/tools.xtensa-esp32-elf-gcc.path=\{runtime.tools.xtensa-esp32-elf-gcc.path\}/g' | \
sed 's/tools.xtensa-esp32s2-elf-gcc.path={runtime.platform.path}\/tools\/xtensa-esp32s2-elf/tools.xtensa-esp32s2-elf-gcc.path=\{runtime.tools.xtensa-esp32s2-elf-gcc.path\}/g' | \
sed 's/tools.xtensa-esp32s3-elf-gcc.path={runtime.platform.path}\/tools\/xtensa-esp32s3-elf/tools.xtensa-esp32s3-elf-gcc.path=\{runtime.tools.xtensa-esp32s3-elf-gcc.path\}/g' | \
sed 's/tools.xtensa-esp-elf-gdb.path={runtime.platform.path}\/tools\/xtensa-esp-elf-gdb/tools.xtensa-esp-elf-gdb.path=\{runtime.tools.xtensa-esp-elf-gdb.path\}/g' | \
sed 's/tools.riscv32-esp-elf-gcc.path={runtime.platform.path}\/tools\/riscv32-esp-elf/tools.riscv32-esp-elf-gcc.path=\{runtime.tools.riscv32-esp-elf-gcc.path\}/g' | \
sed 's/tools.riscv32-esp-elf-gdb.path={runtime.platform.path}\/tools\/riscv32-esp-elf-gdb/tools.riscv32-esp-elf-gdb.path=\{runtime.tools.riscv32-esp-elf-gdb.path\}/g' | \
sed 's/tools.esptool_py.path={runtime.platform.path}\/tools\/esptool/tools.esptool_py.path=\{runtime.tools.esptool_py.path\}/g' | \
sed 's/debug.server.openocd.path={runtime.platform.path}\/tools\/openocd-esp32\/bin\/openocd/debug.server.openocd.path=\{runtime.tools.openocd-esp32.path\}\/bin\/openocd/g' | \
sed 's/debug.server.openocd.scripts_dir={runtime.platform.path}\/tools\/openocd-esp32\/share\/openocd\/scripts\//debug.server.openocd.scripts_dir=\{runtime.tools.openocd-esp32.path\}\/share\/openocd\/scripts\//g' | \
Expand Down
8 changes: 8 additions & 0 deletions .github/scripts/sketch_utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex
esp32s2_opts="PSRAM=enabled,PartitionScheme=huge_app"
esp32s3_opts="PSRAM=opi,USBMode=default,PartitionScheme=huge_app"
esp32c3_opts="PartitionScheme=huge_app"
esp32c6_opts="PartitionScheme=huge_app"
esp32h2_opts="PartitionScheme=huge_app"

# Select the common part of the FQBN based on the target. The rest will be
# appended depending on the passed options.
Expand All @@ -91,6 +93,12 @@ function build_sketch(){ # build_sketch <ide_path> <user_path> <path-to-ino> [ex
"esp32s3")
fqbn="espressif:esp32:esp32s3:${options:-$esp32s3_opts}"
;;
"esp32c6")
fqbn="espressif:esp32:esp32c6:${options:-$esp32c6_opts}"
;;
"esp32h2")
fqbn="espressif:esp32:esp32h2:${options:-$esp32h2_opts}"
;;
esac

# Make it look like a JSON array.
Expand Down
11 changes: 11 additions & 0 deletions .github/scripts/upload_py_tools.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash
CHANGED_FILES=$1
echo "Pushing '$CHANGED_FILES' as $GITHUB_ACTOR"
git config --global github.user "$GITHUB_ACTOR"
git config --global user.name "$GITHUB_ACTOR"
git config --global user.email "$GITHUB_ACTOR@users.noreply.github.com"
for tool in $CHANGED_FILES; do
git add tools/$tool.exe
done
git commit -m "Push binary to tools"
git push
141 changes: 141 additions & 0 deletions .github/workflows/build_py_tools.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
name: Build Python Tools

on:
pull_request:
paths:
- 'tools/get.py'
- 'tools/espota.py'
- 'tools/gen_esp32part.py'
- 'tools/gen_insights_package.py'

jobs:
find-changed-tools:
name: Check if tools have been changed
runs-on: ubuntu-20.04
outputs:
any_changed: ${{ steps.verify-changed-files.outputs.any_changed }}
all_changed_files: ${{ steps.verify-changed-files.outputs.all_changed_files }}
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.ref }}
- name: Verify Python Tools Changed
uses: tj-actions/changed-files@v36
id: verify-changed-files
with:
fetch_depth: '2'
since_last_remote_commit: 'true'
files: |
tools/get.py
tools/espota.py
tools/gen_esp32part.py
tools/gen_insights_package.py
- name: List all changed files
shell: bash
run: |
for file in ${{ steps.verify-changed-files.outputs.all_changed_files }}; do
echo "$file was changed"
done
build-pytools-binaries:
name: Build python tools binaries for ${{ matrix.os }}
runs-on: ${{ matrix.os }}
needs: find-changed-tools
if: needs.find-changed-tools.outputs.any_changed == 'true'
strategy:
fail-fast: false
matrix:
os: [windows-latest, macos-latest, ubuntu-20.04, ARM, ARM64]
include:
- os: windows-latest
TARGET: win64
EXTEN: .exe
SEPARATOR: ';'
- os: macos-latest
TARGET: macos
SEPARATOR: ':'
- os: ubuntu-20.04
TARGET: linux-amd64
SEPARATOR: ':'
- os: ARM
CONTAINER: python:3.8-bullseye
TARGET: arm
SEPARATOR: ':'
- os: ARM64
CONTAINER: python:3.8-bullseye
TARGET: arm64
SEPARATOR: ':'
container: ${{ matrix.CONTAINER }} # use python container on ARM
env:
DISTPATH: pytools-${{ matrix.TARGET }}
PIP_EXTRA_INDEX_URL: "https://dl.espressif.com/pypi"
steps:
- name: List changed tools
shell: bash
run: |
CHANGED_FILES=()
for file in ${{ needs.find-changed-tools.outputs.all_changed_files }}; do
file="${file#*\/}"
file="${file%\.*}"
CHANGED_FILES+=("$file")
done
CHANGED_FILES="${CHANGED_FILES[@]}"
echo "CHANGED_TOOLS=$CHANGED_FILES" >> "$GITHUB_ENV"
for tool in ${{ env.CHANGED_TOOLS }}; do
echo "tool $tool was changed"
done
- name: Checkout repository
uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.ref }}
- name: Set up Python 3.8
# Skip setting python on ARM because of missing compatibility: https://github.com/actions/setup-python/issues/108
if: matrix.os != 'ARM' && matrix.os != 'ARM64'
uses: actions/setup-python@master
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pyinstaller requests
- name: Build with PyInstaller
shell: bash
run: |
for tool in ${{ env.CHANGED_TOOLS }}; do
pyinstaller --distpath ./${{ env.DISTPATH }} -F --icon=.github/pytools/espressif.ico tools/$tool.py
done
- name: Sign binaries
if: matrix.os == 'windows-latest'
env:
CERTIFICATE: ${{ secrets.CERTIFICATE }}
CERTIFICATE_PASSWORD: ${{ secrets.CERTIFICATE_PASSWORD }}
shell: pwsh
run: |
$data = Write-Output ${{ env.CHANGED_TOOLS }}
foreach ( $node in $data )
{
./.github/pytools/Sign-File.ps1 -Path ./${{ env.DISTPATH }}/$node.exe
}
- name: Test binaries
shell: bash
run: |
for tool in ${{ env.CHANGED_TOOLS }}; do
./${{ env.DISTPATH }}/$tool${{ matrix.EXTEN }} -h
done
- name: Push binary to tools
if: matrix.os == 'windows-latest'
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
shell: bash
run: |
for tool in ${{ env.CHANGED_TOOLS }}; do
cp -f ./${{ env.DISTPATH }}/$tool.exe tools/$tool.exe
done
bash .github/scripts/upload_py_tools.sh "${{ env.CHANGED_TOOLS }}"
- name: Archive artifact
uses: actions/upload-artifact@master
with:
name: ${{ env.DISTPATH }}
path: ${{ env.DISTPATH }}
13 changes: 4 additions & 9 deletions .github/workflows/hil.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
chip: ['esp32', 'esp32s2', 'esp32s3', 'esp32c3']
chip: ['esp32', 'esp32s2', 'esp32s3', 'esp32c3', 'esp32c6', 'esp32h2']
chunks: ${{fromJson(needs.gen_chunks.outputs.chunks)}}
steps:
- name: Checkout Repository
Expand All @@ -66,17 +66,12 @@ jobs:
Test:
needs: [gen_chunks, Build]
name: ${{matrix.chip}}-Test#${{matrix.chunks}}
runs-on:
- ESP32
- ESP32-S2
- ESP32-S3
- ESP32-C3

strategy:
fail-fast: false
matrix:
chip: ['esp32', 'esp32s2', 'esp32s3', 'esp32c3']
chip: ['esp32', 'esp32s2', 'esp32s3', 'esp32c3', 'esp32c6', 'esp32h2']
chunks: ${{fromJson(needs.gen_chunks.outputs.chunks)}}
runs-on: [arduino, "${{matrix.chip}}"]
container:
image: python:3.10.1-bullseye
options: --privileged
Expand All @@ -94,7 +89,7 @@ jobs:
- name: Install dependencies
run: |
pip install -U pip
pip install -r tests/requirements.txt
pip install -r tests/requirements.txt --extra-index-url https://dl.espressif.com/pypi
apt update && apt install -y -qq jq
- name: Run Tests
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/lib.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ jobs:
- esp32s2
- esp32c3
- esp32s3
- esp32c6
- esp32h2

include:
- target: esp32
Expand All @@ -46,6 +48,10 @@ jobs:
fqbn: espressif:esp32:esp32c3
- target: esp32s3
fqbn: espressif:esp32:esp32s3
- target: esp32c6
fqbn: espressif:esp32:esp32c6
- target: esp32h2
fqbn: espressif:esp32:esp32h2


steps:
Expand Down
Loading

0 comments on commit 6f7a1ca

Please sign in to comment.