Skip to content

Conversation

@zero323
Copy link
Member

@zero323 zero323 commented Oct 7, 2020

What changes were proposed in this pull request?

This PR replaces dynamically generated annotations for following modules:

  • pyspark.resource.information
  • pyspark.resource.profile
  • pyspark.resource.requests

Why are the changes needed?

These modules where not manually annotated in pyspark-stubs, but are part of the public API and we should provide more precise annotations.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

MyPy tests:

mypy --no-incremental --config python/mypy.ini python/pyspark

@HyukjinKwon
Copy link
Member

HyukjinKwon commented Oct 7, 2020

@zero323, let's file a separate JIRA instead of making followups at SPARK-32714 since that ticket basically targets to port as is. It's easier to revert, edit the fixed version, etc. when there's a separate JIRA so it's usually more preferred to create a separate JIRA when possible (at least in this project as far as I can tell).

@zero323
Copy link
Member Author

zero323 commented Oct 7, 2020

@zero323, let's file a separate JIRA instead of making followups at SPARK-32714 since that ticket basically targets to port as is. It's easier to revert, etc when there's a separate JIRA so it's usually more preferred to create a separate JIRA when possible (at least in this project as far as I can tell).

Sure thing. Please give me a second.

@zero323 zero323 changed the title [SPARK-32714][FOLLOW-UP] Add static annotations for pyspark.resource [SPARK-33086][PYTHON] Add static annotations for pyspark.resource Oct 7, 2020
Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HyukjinKwon
Copy link
Member

Yeah, this is fine too. LGTM

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Test build #129517 has finished for PR 29969 at commit 98dcb7a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Test build #129518 has finished for PR 29969 at commit 27752ee.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@zero323
Copy link
Member Author

zero323 commented Oct 7, 2020

Sorry, @HyukjinKwon one more minor style change.

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34122/

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Test build #129519 has finished for PR 29969 at commit 90d89a6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34123/

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34122/

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34125/

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34123/

@SparkQA
Copy link

SparkQA commented Oct 7, 2020

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34125/

@HyukjinKwon
Copy link
Member

Merged to master.

@zero323 zero323 deleted the SPARK-32714-FOLLOW-UP-RESOURCE branch October 8, 2020 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants