From 35ebe65696eda93666fa07089b609aa5958278ba Mon Sep 17 00:00:00 2001 From: dervoeti Date: Wed, 19 Nov 2025 11:35:06 +0100 Subject: [PATCH 1/6] docs: extended patchable readme --- rust/patchable/README.md | 109 +++++++++++++++++++++++++++++++++++---- 1 file changed, 99 insertions(+), 10 deletions(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index 9b1334d1e..10079a4b1 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -59,29 +59,46 @@ cargo patchable export druid 26.0.0 git status ``` -> ![CAUTION] +> [!CAUTION] > `cargo patchable export` exports whatever is currently checked out (`HEAD`) in the worktree. > If you use `jj edit` (or `git switch`) then you _must_ go back to the tip before exporting, or > any patches after that point will be omitted from the export. -### Initialize a new product +### Adding a new product -If you're adding a completely new product, you need to initialize the product-level config once -using patchable: +If you're adding a completely new product, start by creating the product directory with `mkdir product-name`. +Note that you can also create dependencies in subdirectories, like `product/my-dependency` or `shared/my-dependency`. +Next, create `Dockerfile` and `boil-config.toml` in that directory. +Then create a fork on GitHub (main branch is fine, patchable ensures tags for configured versions are mirrored). + +Initialize the product-level config by running: ```sh -cargo patchable init product druid \ - --upstream https://github.com/apache/druid.git \ - --default-mirror https://github.com/stackabletech/druid.git +cargo patchable init product product-name \ + --upstream https://github.com/upstream-org/product-name.git \ + --default-mirror https://github.com/stackabletech/product-name.git ``` -This will create the product-level configuration in `docker-images/druid/stackable/patches/patchable.toml` -containing the following fields: +This will create the product-level configuration in `docker-images/product-name/stackable/patches/patchable.toml` containing the following fields: - `upstream` - the URL of the upstream repository (such as `https://github.com/apache/druid.git`) - `default_mirror` - optional: default URL of a mirror repository (such as `https://github.com/stackabletech/druid.git`) -### Initialize a new patch series +Initialize the product version (`base` can be a branch, tag, or commit): + +```sh +cargo patchable init version product-name 1.2.3 --base=product-name-1.2.3 --mirror +``` + +Checkout the source code and switch into the directory with `pushd $(cargo patchable checkout product-name 1.2.3)`. +Create your patches and commit them, then export the patches as .patch files by running: + +```sh +popd +cargo patchable export product-name 1.2.3 +``` + +### Adding a new version to an existing product Patchable stores metadata about each patch series in its version-level config, and will not be able to check out a patch series that lacks one. It can be generated using the following command: @@ -99,6 +116,51 @@ with the base commit hash and the default mirror URL from the product-level conf You can optionally provide the `--ssh` flag to use SSH instead of HTTPS for Git operations. +### Creating a forked dependency + +You can use the same patchable workflow for custom dependencies (e.g., Java libraries) as for regular products. + +For example, to create a patched version of a Java dependency, start by creating a directory for the dependency with `mkdir -p shared/jackson-dataformat-xml`. +Then create a `Dockerfile` with build instructions and `boil-config.toml` in that directory. Create a fork of the dependency on GitHub for mirroring the source code. + +Initialize patchable product configuration: + +```sh +cargo patchable init product shared/jackson-dataformat-xml \ + --upstream https://github.com/FasterXML/jackson-dataformat-xml.git \ + --default-mirror https://github.com/stackabletech/jackson-dataformat-xml.git +``` + +Initialize a specific version: + +```sh +cargo patchable init version shared/jackson-dataformat-xml 2.12.7 \ + --base=jackson-dataformat-xml-2.12.7 --mirror +``` + +Create and export patches: + +```sh +pushd $(cargo patchable checkout shared/jackson-dataformat-xml 2.12.7) +# Make your changes +git commit -m "Apply security patch" +popd +cargo patchable export shared/jackson-dataformat-xml 2.12.7 +``` + +Use the dependency in other products. For a Java dependency for example it would look like this in the product's `Dockerfile`: + +```dockerfile +FROM local-image/shared/jackson-dataformat-xml AS patched-jackson-dataformat-xml +COPY --chown=${STACKABLE_USER_UID}:0 --from=patched-jackson-dataformat-xml /stackable/.m2/repository /stackable/patched-jackson-dataformat-xml-libs + +# Copy patched libs into local Maven repository +mkdir -p /stackable/.m2/repository +cp -r /stackable/patched-jackson-dataformat-xml-libs/* /stackable/.m2/repository +``` + +Now the patched dependency will be used instead of pulling the original one. + ### Importing patch series into Patchable Patchable is stricter about applying invalid patches (both metadata and patches themselves) than Git is. @@ -169,6 +231,33 @@ Druid 26.0.0 series: git cherry-pick patchable/26.0.0 ``` +### Backporting a patch + +To backport a specific upstream commit to a product version, first checkout the source code with: + +```sh +pushd $(cargo patchable checkout zookeeper 3.9.3) +``` + +Fetch the upstream commit using: + +```sh +git fetch https://github.com/apache/zookeeper 8532163 +``` + +Then cherry-pick it with: + +```sh +git cherry-pick 8532163 +``` + +Finally, export the patches as .patch files: + +```sh +popd +cargo patchable export zookeeper 3.9.3 +``` + ## Notes - patchable only supports linear patch series (no merges beyond the base commit) From c4d220709d9178b5291e01e8bfd1cea154bd918c Mon Sep 17 00:00:00 2001 From: Lukas Krug Date: Wed, 19 Nov 2025 15:21:39 +0100 Subject: [PATCH 2/6] Update rust/patchable/README.md Co-authored-by: Techassi --- rust/patchable/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index 10079a4b1..ac434d0e9 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -67,7 +67,7 @@ git status ### Adding a new product If you're adding a completely new product, start by creating the product directory with `mkdir product-name`. -Note that you can also create dependencies in subdirectories, like `product/my-dependency` or `shared/my-dependency`. +Note that you can also create dependencies in subdirectories, like `product-name/my-dependency` or `shared/my-dependency`. Next, create `Dockerfile` and `boil-config.toml` in that directory. Then create a fork on GitHub (main branch is fine, patchable ensures tags for configured versions are mirrored). From 32f5a1dbe95b10dc443580512228ad175154ad96 Mon Sep 17 00:00:00 2001 From: Lukas Krug Date: Wed, 19 Nov 2025 15:21:49 +0100 Subject: [PATCH 3/6] Update rust/patchable/README.md Co-authored-by: Techassi --- rust/patchable/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index ac434d0e9..5bb566d33 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -68,7 +68,7 @@ git status If you're adding a completely new product, start by creating the product directory with `mkdir product-name`. Note that you can also create dependencies in subdirectories, like `product-name/my-dependency` or `shared/my-dependency`. -Next, create `Dockerfile` and `boil-config.toml` in that directory. +Next, create a `Dockerfile` and a `boil-config.toml` file in that directory. Then create a fork on GitHub (main branch is fine, patchable ensures tags for configured versions are mirrored). Initialize the product-level config by running: From 5e8d1c30b6363a4cd34f40eb2ef6eef33099bcb4 Mon Sep 17 00:00:00 2001 From: Lukas Krug Date: Wed, 19 Nov 2025 15:21:57 +0100 Subject: [PATCH 4/6] Update rust/patchable/README.md Co-authored-by: Techassi --- rust/patchable/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index 5bb566d33..6258e6baa 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -69,7 +69,7 @@ git status If you're adding a completely new product, start by creating the product directory with `mkdir product-name`. Note that you can also create dependencies in subdirectories, like `product-name/my-dependency` or `shared/my-dependency`. Next, create a `Dockerfile` and a `boil-config.toml` file in that directory. -Then create a fork on GitHub (main branch is fine, patchable ensures tags for configured versions are mirrored). +Then create a fork on GitHub (the default branch, like `main`, is fine - patchable ensures tags for configured versions are mirrored). Initialize the product-level config by running: From 680fe1a477807f7ad882dadd46fcc07a4ffa9b57 Mon Sep 17 00:00:00 2001 From: Lukas Krug Date: Wed, 19 Nov 2025 17:25:10 +0100 Subject: [PATCH 5/6] Update rust/patchable/README.md Co-authored-by: Techassi --- rust/patchable/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index 6258e6baa..059877953 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -121,7 +121,7 @@ You can optionally provide the `--ssh` flag to use SSH instead of HTTPS for Git You can use the same patchable workflow for custom dependencies (e.g., Java libraries) as for regular products. For example, to create a patched version of a Java dependency, start by creating a directory for the dependency with `mkdir -p shared/jackson-dataformat-xml`. -Then create a `Dockerfile` with build instructions and `boil-config.toml` in that directory. Create a fork of the dependency on GitHub for mirroring the source code. +Then create a `Dockerfile` with build instructions and a `boil-config.toml` file in that directory. Create a fork of the dependency on GitHub for mirroring the source code. Initialize patchable product configuration: From 09b7d6f70157e29e81d220eca4534479e55c6122 Mon Sep 17 00:00:00 2001 From: dervoeti Date: Wed, 19 Nov 2025 17:31:13 +0100 Subject: [PATCH 6/6] chore: use commit hash with letters --- rust/patchable/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rust/patchable/README.md b/rust/patchable/README.md index 059877953..83ebc8064 100644 --- a/rust/patchable/README.md +++ b/rust/patchable/README.md @@ -242,13 +242,13 @@ pushd $(cargo patchable checkout zookeeper 3.9.3) Fetch the upstream commit using: ```sh -git fetch https://github.com/apache/zookeeper 8532163 +git fetch https://github.com/apache/zookeeper 3d6c0d1164dc9ec96a02de383e410b1b0ef64565 ``` Then cherry-pick it with: ```sh -git cherry-pick 8532163 +git cherry-pick 3d6c0d1 ``` Finally, export the patches as .patch files: