Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Nov 7, 2025

What changes were proposed in this pull request?

This PR aims to use an absolute output path ("$SELF/_site/llms.txt") instead of a relative path (_site/llms.txt) when dev/create-release/release-build.sh uses generate-llms-txt.py.

python "$SELF/generate-llms-txt.py" \
--docs-path . \
--output _site/llms.txt \

Why are the changes needed?

Currently, Apache Spark release CI has been broken. (https://github.com/apache/spark/actions/workflows/release.yml)
This is a release blocker for Apache Spark 4.1.0.

+ echo 'Generating llms.txt...'
Generating llms.txt...
+ python /opt/spark-rm/generate-llms-txt.py --docs-path . --output _site/llms.txt --version 4.2.0-SNAPSHOT
Traceback (most recent call last):
  File "/opt/spark-rm/generate-llms-txt.py", line 206, in <module>
    main()
  File "/opt/spark-rm/generate-llms-txt.py", line 202, in main
    generate_llms_txt(docs_path, output_path, args.version)
  File "/opt/spark-rm/generate-llms-txt.py", line 159, in generate_llms_txt
    with open(output_path, "w", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/_site/llms.txt'
+ echo 'Generating llms.txt...'
Generating llms.txt...
+ python /opt/spark-rm/generate-llms-txt.py --docs-path . --output _site/llms.txt --version 4.2.0-SNAPSHOT
Traceback (most recent call last):
  File "/opt/spark-rm/generate-llms-txt.py", line 206, in <module>
    main()
  File "/opt/spark-rm/generate-llms-txt.py", line 202, in main
    generate_llms_txt(docs_path, output_path, args.version)
  File "/opt/spark-rm/generate-llms-txt.py", line 159, in generate_llms_txt
    with open(output_path, "w", encoding="utf-8") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/_site/llms.txt'

Does this PR introduce any user-facing change?

No behavior change because this is a release script.

How was this patch tested?

Manual test.

$ mkdir /tmp/SPARK-53666
$ dev/create-release/do-release-docker.sh -t SPARK-53666 -d /tmp/SPARK-53666 -n -s docs
...
= Building spark-rm image with tag SPARK-53666...
...

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun dongjoon-hyun marked this pull request as draft November 7, 2025 06:09
@dongjoon-hyun dongjoon-hyun changed the title [SPARK-53666][INFRA] Use absolute output path for generate-llms-txt.py [SPARK-53666][INFRA][FOLLOWUP] Use absolute output path for generate-llms-txt.py Nov 7, 2025
@dongjoon-hyun dongjoon-hyun marked this pull request as ready for review November 7, 2025 06:41
@dongjoon-hyun
Copy link
Member Author

Thank you, @zhengruifeng .

@dongjoon-hyun
Copy link
Member Author

Let me merge this and run the CI.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-53666 branch November 7, 2025 06:48
Copy link
Contributor

@allisonwang-db allisonwang-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants