Skip to content

[SPARK-56807][INFRA] Downgrade black to 23.12.1 for older branches in Python CodeGen check#55784

Closed
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:SPARK-56807
Closed

[SPARK-56807][INFRA] Downgrade black to 23.12.1 for older branches in Python CodeGen check#55784
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:SPARK-56807

Conversation

@LuciferYang
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This pr aim to downgrade black version from 26.3.1 to 23.12.1 for older branches in Python CodeGen check

Why are the changes needed?

To restore Protobuf breaking change detection and Python CodeGen check task in branch-4.1/branch-4.0 daily tests:

Oh no! 💥 💔 💥 The required version `23.12.1` does not match the running version `26.3.1`!
Traceback (most recent call last):
Start checking the generated codes in pyspark-connect.
  File "/home/runner/work/spark/spark/./dev/check-protos.py", line 84, in <module>
    check_protos(
  File "/home/runner/work/spark/spark/./dev/check-protos.py", line 49, in check_protos
RUN: /home/runner/work/spark/spark/dev/gen-protos.sh connect /tmp/check_connect__protosjcoo70pe
    run_cmd(f"{SPARK_HOME}/dev/gen-protos.sh {module_name} {tmp}")
  File "/home/runner/work/spark/spark/./dev/check-protos.py", line 43, in run_cmd
    return subprocess.check_output(cmd.split(" ")).decode("utf-8")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/runner/work/spark/spark/dev/gen-protos.sh', 'connect', '/tmp/check_connect__protosjcoo70pe']' returned non-zero exit status 1.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Monitor the daily test results for branch-4.1/branch-4.0 after merging.

Was this patch authored or co-authored using generative AI tooling?

No

@LuciferYang
Copy link
Copy Markdown
Contributor Author

Merged into master. Thanks @HyukjinKwon

@HyukjinKwon
Copy link
Copy Markdown
Member

should this be merged to branch-4.x and branch-4.2 too?

@LuciferYang
Copy link
Copy Markdown
Contributor Author

Is this just for code alignment? branch-4.x / branch-4.2 won’t trigger daily tests for other branches, so functionally there’s no need to do it.

@HyukjinKwon
Copy link
Copy Markdown
Member

ah okok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants