Skip to content

[MINOR][BUILD] Mark spark-deps manifest as auto-generated - #57851

Open
pan3793 wants to merge 1 commit into
apache:masterfrom
pan3793:mark-spark-deps-as-generated
Open

[MINOR][BUILD] Mark spark-deps manifest as auto-generated#57851
pan3793 wants to merge 1 commit into
apache:masterfrom
pan3793:mark-spark-deps-as-generated

Conversation

@pan3793

@pan3793 pan3793 commented Aug 7, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

dev/test-dependencies.sh now writes a header at the top of each manifest it generates (dev/deps/spark-deps-*):

#
# Generated by `dev/test-dependencies.sh --replace-manifest`.
# Do not edit manually.
#

The same header is prepended to the committed dev/deps/spark-deps-hadoop-3-hive-2.3 so the manifest matches generator output and the self-check (dev/test-dependencies.sh without --replace-manifest) stays green.

Only POSIX-portable utilities are used in the new code path (printf '%s\n', cat, rm); the existing awk already uses only POSIX-standard functions, so the script remains compatible across macOS (BSD awk) and Linux (gawk).

Why are the changes needed?

The manifest is machine-generated and regenerated by dev/test-dependencies.sh --replace-manifest; hand edits get overwritten on the next regeneration. A header stating this prevents accidental manual edits to a checked-in generated file.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

  • bash -n dev/test-dependencies.sh
  • Verified byte-for-byte that the header the script emits matches the header prepended to the committed manifest, so the manifest self-check still passes.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: GLM 5.2

@pan3793

pan3793 commented Aug 7, 2026

Copy link
Copy Markdown
Member Author

I find some agents tend to modify it manually without this guidance, for example, #57779

@uros-b

uros-b commented Aug 7, 2026

Copy link
Copy Markdown
Member

LGTM, thank you @pan3793

The manifest under dev/deps/spark-deps-* is produced by dev/test-dependencies.sh --replace-manifest and should not be edited by hand. Emit a header from the generator and prepend the same header to the committed manifest so the file carries the information at its head.

Assisted-by: GLM 5.2
@pan3793
pan3793 force-pushed the mark-spark-deps-as-generated branch from 418be33 to f82cc0a Compare August 9, 2026 02:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants