From 4c7fd880a90d08448db366bde40d0d385b45f75e Mon Sep 17 00:00:00 2001 From: Diego Marquez Date: Wed, 20 Mar 2024 11:42:36 -0400 Subject: [PATCH] fix: correct deep-remove and deep-preserve regexes (#2572) In this PR we fix the `deep-remove` and `deep-preserve` regexes in `.OwlBot.yaml` files in the monorepo by creating a temporary `.OwlBot.hermetic.yaml` with the corrected paths. ### Why? Because the paths specified in `deep-remove-regex` and `deep-preserve-regex` [start from the library name](https://github.com/googleapis/google-cloud-java/blob/14f7146c98fdf03de2f113215e03347a21e83f9a/java-alloydb/.OwlBot.yaml#L17), so if we run `copy-code` from inside a library, these regexes won't match any files. ### What about deep-copy? We don't need to correct the library path because `copy-code` sends the files from `googleapis-gen` (or our built temp folder in our case) to the `owl-bot-staging` folder. These regexes don't deal with the monorepo, they deal with the source repo, which for our case is the temp folder we build after calling generate_library. This is why `deep-copy` was always being "respected". ### Proof After running `python generate_repo.py generate --generation-config-yaml google-cloud-java/generation_config.yaml --repository-path google-cloud-java --target-library-api-shortname merchantapi &> out`, we get ![image](https://github.com/googleapis/sdk-platform-java/assets/22083784/087e0df1-baf3-4939-94e0-61027797d19e) --------- Co-authored-by: Joe Wang <106995533+JoeWang1127@users.noreply.github.com> --- library_generation/postprocess_library.sh | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/library_generation/postprocess_library.sh b/library_generation/postprocess_library.sh index 66c0c1a020..c6a5b4020d 100755 --- a/library_generation/postprocess_library.sh +++ b/library_generation/postprocess_library.sh @@ -62,7 +62,22 @@ fi # we determine the location of the .OwlBot.yaml file by checking if the target # folder is a monorepo folder or not if [[ "${is_monorepo}" == "true" ]]; then - owlbot_yaml_relative_path=".OwlBot.yaml" + # the deep-remove-regex and deep-preserve-regex of the .OwlBot.yaml + # files in the monorepo libraries assume that `copy-code` is run + # from the root of the monorepo. However, we call `copy-code` from inside each + # library, so a path like `/java-asset/google-.*/src` will not have + # any effect. We solve this by creating a temporary owlbot yaml with + # the patched paths. + # For example, we convert + # - "/java-asset/google-.*/src" + # to + # - "/google-.*/src" + + library_name=$(basename "${postprocessing_target}") + cat "${postprocessing_target}/.OwlBot.yaml" \ + | sed "s/- \"\/${library_name}/ - \"/" \ + > "${postprocessing_target}/.OwlBot.hermetic.yaml" + owlbot_yaml_relative_path=".OwlBot.hermetic.yaml" else owlbot_yaml_relative_path=".github/.OwlBot.yaml" fi @@ -109,6 +124,11 @@ docker run --rm \ --source-repo="${preprocessed_libraries_binding}" \ --config-file="${owlbot_yaml_relative_path}" +# clean the custom owlbot yaml +if [[ "${is_monorepo}" == "true" ]]; then + rm "${postprocessing_target}/.OwlBot.hermetic.yaml" +fi + # we clone the synthtool library and manually build it mkdir -p /tmp/synthtool pushd /tmp/synthtool