Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EmrEtlRunner: move max attempts configuration to EMR cluster configuration #3246

Closed
BenFradet opened this issue Jun 13, 2017 · 0 comments
Closed
Assignees
Milestone

Comments

@BenFradet
Copy link
Contributor

BenFradet commented Jun 13, 2017

need to be removed in favor of:

https://github.com/snowplow/snowplow/blob/master/3-enrich/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb#L32

as

{
  "Classification" => "yarn-site",
  "Properties" => {
    "yarn.resourcemanager.am.max-attempts" => "1"
  }
}

unless rslifka/elasticity#138 is merged, in this case just use the elasticity api.

@BenFradet BenFradet added this to the R9x [HAD] StorageLoader reboot milestone Jun 13, 2017
@BenFradet BenFradet self-assigned this Jun 13, 2017
@chuwy chuwy closed this as completed in 07d3fce Jul 26, 2017
chuwy pushed a commit to snowplow/snowplow-rdb-loader that referenced this issue Sep 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant