Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark Operator example update with best practices #853

Merged
merged 10 commits into from
Aug 23, 2022

Conversation

vara-bonthu
Copy link
Contributor

@vara-bonthu vara-bonthu commented Aug 14, 2022

What does this PR do?

Spark Operator example updated with best practices

  • Multiple node groups (Core and Spark)
  • VPC endpoints for worker node private access
  • Addons config updated to support large clusters e.g., coredns, fluentbit etc.
  • Spark team created to support the Sample Spark jobs
  • Spark Operator manifest files updated

Closes Issue: #850

🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.

Motivation

More

  • Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
  • Yes, I have added a new example under examples to support my PR
  • Yes, I have created another PR for add-ons under add-ons repo (if applicable)
  • Yes, I have updated the docs for this feature
  • Yes, I ran pre-commit run -a with this PR

Note: Not all the PRs required examples and docs except a new pattern or add-on added.

For Moderators

  • E2E Test successfully complete before merge?

Additional Notes

@vara-bonthu vara-bonthu temporarily deployed to EKS Blueprints Test August 14, 2022 21:34 Inactive
@vara-bonthu vara-bonthu temporarily deployed to EKS Blueprints Test August 15, 2022 08:02 Inactive
Copy link
Contributor

@askulkarni2 askulkarni2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vara-bonthu looking good, some comments.

@vara-bonthu vara-bonthu temporarily deployed to EKS Blueprints Test August 16, 2022 07:25 Inactive
Copy link
Contributor

@askulkarni2 askulkarni2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor unresolved comment, otherwise LGTM!

examples/analytics/spark-k8s-operator/main.tf Show resolved Hide resolved
@vara-bonthu vara-bonthu temporarily deployed to EKS Blueprints Test August 23, 2022 08:49 Inactive
@vara-bonthu vara-bonthu merged commit fafa793 into main Aug 23, 2022
@vara-bonthu vara-bonthu deleted the spark-operator-update branch August 23, 2022 12:50
allamand pushed a commit to allamand/terraform-aws-eks-blueprints that referenced this pull request Dec 15, 2022
* Spark Operator example update with best practices

* pre-commit hooks updates

* tflint fix added

* Removed default values from values.yaml

* added comment for docker image

* comments with - updated with same length

* removed vpc cidr from locals

* upated locals with vpc endpoints

* Updated the link for asg tags
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants