Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(glue): adds timeout and workerType validation for Ray jobs #29613

Closed
wants to merge 2 commits into from

Conversation

haljarrett
Copy link

Issue

Closes #29612

Reason for this change

In the course of building some Ray on Glue jobs, I ran into deployment failures both from specifying a timeout and from not specifying a worker type. In both cases, the construct synthesizes, but fails to deploy - see the linked issue for a small reproduction.

I opened the above linked issue, which this PR addresses.

Description of changes

This change introduces two new validations to the constructor of Job from @aws-cdk/aws-glue-alpha:

  • validation that Ray-type jobs don't specify timeout, which is not a supported job property for Ray jobs
  • validation that Ray-type jobs do specify the workerType as Z_2X - to deploy, this may not be null and must be Z_2X

Description of how you validated changes

I added three new unit tests to the other constructor validation tests in job.test.ts, under Job -> new -> ray job:

  • testing that specifying timeout will throw an error
  • testing that a null workerType
  • testing that a workerType other that Z_2X will throw an error

From my local machine, I validated these changes with yarn build and yarn test, as well as linking the updated package into my repro sample package and manually testing. Additionally, I ran yarn lint and confirmed there were no outstanding format issues.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. p2 labels Mar 26, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team March 26, 2024 03:37
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@haljarrett haljarrett changed the title fix(aws-glue-alpha): adds timeout and workerType validation for Ray jobs fix(glue): adds timeout and workerType validation for Ray jobs Mar 26, 2024
@haljarrett
Copy link
Author

Exemption Request: this change only affects construct validation, so I don't think there is anything testable with integration / snapshot tests - please advise if I am misunderstanding something here though, this is my first contribution.

@aws-cdk-automation aws-cdk-automation added pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Mar 26, 2024
@TheRealAmazonKendra
Copy link
Contributor

There is actually a group working on redesigning some of the rough edges in this module. I'd like to make sure that any changes we take doesn't cause code conflicts with their work so I'm going to add the do-not-merge label to this while I discuss with them.

@TheRealAmazonKendra TheRealAmazonKendra added the pr/do-not-merge This PR should not be merged at this time. label Mar 28, 2024
This was referenced Apr 1, 2024
@aws-cdk-automation
Copy link
Collaborator

This PR has been in the CHANGES REQUESTED state for 3 weeks, and looks abandoned. To keep this PR from being closed, please continue work on it. If not, it will automatically be closed in a week.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 3c6059b
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error.

@aws-cdk-automation aws-cdk-automation added the closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. label Apr 25, 2024
@aws-cdk-automation
Copy link
Collaborator

The pull request linter fails with the following errors:

❌ Fixes must contain a change to an integration test file and the resulting snapshot.

PRs must pass status checks before we can provide a meaningful review.

If you would like to request an exemption from the status checks or clarification on feedback, please leave a comment on this PR containing Exemption Request and/or Clarification Request.

✅ A exemption request has been requested. Please wait for a maintainer's review.

@haljarrett
Copy link
Author

@TheRealAmazonKendra Any updates from the Glue team on if this is a PR they are interested in taking? Looks like this was closed for staleness today, but I can reopen (resubmit?) if this is something that has a chance of landing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. p2 pr/do-not-merge This PR should not be merged at this time. pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(glue): Timeout and workerType missing validation for Ray job types
3 participants