diff --git a/assets/img/authors/marek-fiala.webp b/assets/img/authors/marek-fiala.webp new file mode 100644 index 000000000..21e710e74 Binary files /dev/null and b/assets/img/authors/marek-fiala.webp differ diff --git a/content/blog/2025/10/idf_py_extension/featured.webp b/content/blog/2025/10/idf_py_extension/featured.webp new file mode 100644 index 000000000..5c8d77117 Binary files /dev/null and b/content/blog/2025/10/idf_py_extension/featured.webp differ diff --git a/content/blog/2025/10/idf_py_extension/index.md b/content/blog/2025/10/idf_py_extension/index.md new file mode 100644 index 000000000..7f23b8f57 --- /dev/null +++ b/content/blog/2025/10/idf_py_extension/index.md @@ -0,0 +1,293 @@ +--- +title: "Extending idf.py: Create custom commands for your ESP-IDF workflow" +date: "2025-10-10" +showAuthor: false +authors: + - "marek-fiala" +tags: ["ESP-IDF", "idf.py", "CLI", "Extensions", "Development Tools"] +summary: "Learn how to extend idf.py with custom commands for your development workflow. This guide covers both component-based extensions for project-specific tools and Python package extensions for reusable commands, with practical examples and best practices for seamless integration." +--- + + + +What if you could extend `idf.py` with your own custom commands tailored to your specific workflow? With the ESP-IDF v6.0 and newer (to be released soon), you can do exactly that through a powerful extension system that lets you add project-specific tools or distribute reusable commands across your projects. + +Before we dive into extensions, let’s recall what `idf.py` gives you out of the box. It’s the central command-line tool for ESP-IDF that allows you to: + +- Set the target chip using `idf.py set-target` (like esp32) +- Tweak your project settings using `idf.py menuconfig` +- Build your application using `idf.py build` +- Flash it using `idf.py -p PORT flash` +- Watch the logs in real time using `idf.py monitor` + +For most developers, the daily cycle is simply **build → flash → monitor** — all streamlined under one command. + +## Why extend idf.py? + +Sometimes, though, the built-in commands aren’t enough. Maybe you need a custom deployment command that packages your firmware with metadata, or perhaps you want to integrate with your CI/CD pipeline through specialized build targets. Rather than maintaining separate scripts, you can now integrate these directly into `idf.py`, giving you: + +- **Unified interface**: All your tools accessible through the familiar `idf.py` command +- **Consistent help system**: Your commands appear in `idf.py --help` with proper documentation +- **Shared options**: Leverage existing global options like `--port` and `--build-dir` +- **Dependency management**: Ensure commands run in the right order automatically + +## Two ways to extend idf.py + +ESP-IDF supports two extension mechanisms, each suited for different use cases: + +- Component-based extensions +- Python package extensions + +### Component-based extensions + +This is the case for project-specific commands that should only be available when working with a particular project or component. + +**How it works**: Place a file named `idf_ext.py` in your component directory. ESP-IDF automatically discovers and loads extensions from this file **after the project is configured** with `idf.py reconfigure` or `idf.py build`. Within the component-based extension, the name of the file is important, as `idf.py` searches exactly for `idf_ext.py`. + +**Note**: You may also place `idf_ext.py` in the project root instead of a component. This option has existed in earlier ESP-IDF versions and works the same way, but using a dedicated component is recommended for clarity and reusability. + +#### Step 1: Create the extension file + +- Create a new component (or use an existing one). +- Inside the component, add a file named `idf_ext.py`. +- This file must implement an `action_extensions` function returning a dictionary that describes your new commands, options, and callbacks. + +**Example (sensor manager)**: +In this example, we’ll add a new command sensor-info that prints configuration details about sensors in your project. Start by creating a component called `sensor_manager`: + +```bash +# Create the component using idf.py +idf.py create-component -C components sensor_manager +``` + +Then, inside your component directory `components/sensor_manager/`, create `idf_ext.py` Python file and place the following code: + +```python +from typing import Any +import click + +def action_extensions(base_actions: dict, project_path: str) -> dict: + def sensor_info(subcommand_name: str, ctx: click.Context, global_args: dict, **action_args: Any) -> None: + sensor_type = action_args.get('type', 'all') + verbose = getattr(global_args, 'detail', False) + + print(f"Running {subcommand_name} for sensor type: {sensor_type}") + if verbose: + print(f"Project path: {project_path}") + print("Detailed sensor configuration would be displayed here...") + + def global_callback_detail(ctx: click.Context, global_args: dict, tasks: list) -> None: + if getattr(global_args, 'detail', False): + print(f"About to execute {len(tasks)} task(s): {[t.name for t in tasks]}") + + return { + "version": "1", + "global_options": [ + { + "names": ["--detail", "-d"], + "is_flag": True, + "help": "Enable detailed output for all commands", + } + ], + "global_action_callbacks": [global_callback_detail], + "actions": { + "sensor-info": { + "callback": sensor_info, + "short_help": "Display sensor configuration", + "help": "Show detailed information about sensor configuration and status", + "options": [ + { + "names": ["--type", "-t"], + "help": "Sensor type to query (temperature, humidity, pressure, or all)", + "default": "all", + "type": click.Choice(['temperature', 'humidity', 'pressure', 'all']), + } + ] + }, + }, + } +``` + +#### Step 2: Register the component + +- Ensure the new component is registered in your project’s CMakeLists.txt. +- Further information on how to register commponents can be found in [Espressif documentation](https://docs.espressif.com/projects/esp-idf/en/stable/api-guides/build-system.html#component-requirements). + + + +**Example (sensor manager)**: +Update your project's main `CMakeLists.txt` +```cmake +idf_component_register( + SRCS "main.c" + INCLUDE_DIRS "." + REQUIRES "sensor_manager" # This makes the extension available +) +``` + +#### Step 3: Load and test +- Reconfigure or build the project to let ESP-IDF discover the extension. +- Run idf.py help to check that your new command appears. +- Test the new command with its options. + +**Example (sensor manager)**: In our case, the extension adds the `sensor-info` command: + +```bash +# Configure the project to discover the extension +idf.py reconfigure + +# Check that your command appears in help +idf.py --help + +# Try your new command +idf.py sensor-info --type temperature +idf.py --detail sensor-info --type all +``` + +### Python package extensions + +This is ideal for reusable tools that you want to share across multiple projects or distribute to your team. + +**How it works**: Create a Python package with an entry point in the `idf_extension` group. Once installed, the extension is available globally for all projects. + +#### Step 1: Create the package structure + +- Create a new folder for your tool. +- Add a `pyproject.toml` file to describe the package. +- Inside the folder, create a subfolder with the same name, which will contain your Python code. +- Inside that subfolder, add `__init__.py` and a Python file for the extension (e.g., `esp_ext.py`). + - Unlike the fixed `idf_ext.py` in component-based extensions, the filename here is flexible because it is explicitly referenced in `pyproject.toml`. + - For clarity and consistency, it’s recommended to prefix it with your tool name and suffix it with `_ext.py`. + +The resulting structure should look like this: + +```bash +my_sensor_tools/ +├── pyproject.toml # describe the package here +├── my_sensor_tools/ # place your Python code here +│ ├── __init__.py +│ └── esp_ext.py +``` + +#### Step 2: Fill the extension file + +- Implement the `action_extensions` function inside your package’s Python file. + +**Example (sensor manager)**: + +Here we simply copy the `action_extensions` function from the component example into `my_sensor_tools/esp_ext.py`. + +#### Step 3: Configure and install + +- Define the Python entry-point in your `pyproject.toml` under `[project.entry-points.idf_extension]`. + - Use the format name: `package.module:function`. +- Install the package (for development, use `pip install -e .`). +- The new command will now be globally available in any ESP-IDF project. + + +**Example (sensor manager)**: + +The `pyproject.toml` file for our example could look like this: + +```toml +[project] +name = "my-sensor-tools" +version = "1.0.0" + +# Register the extension under the `idf_extension` group, +# so ESP-IDF can automatically discover it +[project.entry-points.idf_extension] +my_sensor_tools = "my_sensor_tools.esp_ext:action_extensions" +``` + +Install and use: + +```bash +# Install in development mode +cd my_sensor_tools +pip install -e . + +# Your command is now available in any ESP-IDF project +cd my_sensor_tools/ +idf.py sensor-info --type temperature +idf.py --detail sensor-info --type all +``` + +## Naming conventions + +- **Avoid conflicts**: Your commands cannot override built-in `idf.py` commands like `build`, `flash`, or `monitor` +- **Use descriptive names**: Prefer `sensor-info` over `info` to avoid ambiguity +- **Package prefixes**: For Python package extensions, consider prefixing commands with your tool name + +## Advanced features + +Do you need something extra? Beyond simple commands, the extension system also gives you ways to define global options, control execution order, and build richer command-line interfaces. These features let you create tools that feel fully integrated with the rest of `idf.py`. + +### Global options and callbacks + +The extension system supports sophisticated features for power users: + +**Global options**: Define options that work across all commands. Can be exposed under `global_args` parameter. + +**Global callbacks**: Functions that run before any tasks execute, perfect for validation, logging, or injecting additional tasks based on global options. + +### Dependencies and order management + +Ensure your commands run in the correct sequence: + +```python +"actions": { + "deploy": { + "callback": deploy_firmware, + "dependencies": ["all"], # Always build before deploying + "order_dependencies": ["flash"], # If flash is requested, run it before deploy + "help": "Deploy firmware to production servers" + } +} +``` + +### Rich argument support + +Support complex command-line interfaces: + +```python +"options": [ + { + "names": ["--config-file", "-c"], + "type": click.Path(exists=True), + "help": "Configuration file path" + }, + { + "names": ["--verbose", "-v"], + "count": True, # -v, -vv, -vvv for different verbosity levels + "help": "Increase verbosity (use multiple times)" + } +], +"arguments": [ + { + "names": ["targets"], + "nargs": -1, # Accept multiple targets + "required": True + } +] +``` + +For more details on the extension API and additional features, see the [Click documentation](https://click.palletsprojects.com/) for argument types and the [ESP-IDF documentation](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/tools/idf-py.html#extending-idf-py) for the complete extension reference. + +## Conclusion + +The `idf.py` extension system opens up powerful possibilities for customizing your ESP-IDF development workflow. Whether you're adding simple project-specific helpers or building sophisticated development tools, extensions let you integrate seamlessly with the existing ESP-IDF ecosystem. + +Start small with a component-based extension for your current project, then graduate to distributable packages as your tools mature. + +## What's next? + +- Explore the [full extension API documentation](https://docs.espressif.com/projects/esp-idf/en/latest/api-guides/tools/idf-py.html#extending-idf-py) for advanced features +- Check out existing extensions in the ESP-IDF codebase for inspiration + +Happy extending! diff --git a/data/authors/marek-fiala.json b/data/authors/marek-fiala.json index 8caadff59..a88558e72 100644 --- a/data/authors/marek-fiala.json +++ b/data/authors/marek-fiala.json @@ -1,5 +1,5 @@ { "name": "Marek Fiala", - "bio": "", - "image": "" + "bio": "SW Developer at Espressif", + "image": "img/authors/marek-fiala.webp" } \ No newline at end of file