Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EmrEtlRunner: recover from request timeout #3943

Closed
BenFradet opened this issue Dec 25, 2018 · 0 comments
Closed

EmrEtlRunner: recover from request timeout #3943

BenFradet opened this issue Dec 25, 2018 · 0 comments
Assignees

Comments

@BenFradet
Copy link
Contributor

BenFradet commented Dec 25, 2018

We've been getting more and more request timeout from emr lately, so we need to recover from them.

RestClient::RequestTimeout: Request Timeout
                transmit at uri:classloader:/gems/rest-client-1.8.0/lib/restclient/request.rb:427
                 execute at uri:classloader:/gems/rest-client-1.8.0/lib/restclient/request.rb:176
                 execute at uri:classloader:/gems/rest-client-1.8.0/lib/restclient/request.rb:41
                    post at uri:classloader:/gems/rest-client-1.8.0/lib/restclient.rb:69
                  submit at uri:classloader:/gems/elasticity-6.0.12/lib/elasticity/aws_session.rb:29
       add_jobflow_steps at uri:classloader:/gems/elasticity-6.0.12/lib/elasticity/emr.rb:63
                add_step at uri:classloader:/gems/elasticity-6.0.12/lib/elasticity/job_flow.rb:163
            block in run at uri:classloader:/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb:684
                    each at org/jruby/RubyArray.java:1801
                     run at uri:classloader:/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb:683
                 send_to at uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_reference.rb:43
               call_with at uri:classloader:/gems/contracts-0.11.0/lib/contracts/call_with.rb:76
block in redefine_method at uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_handler.rb:138
                     run at uri:classloader:/emr-etl-runner/lib/snowplow-emr-etl-runner/runner.rb:138
                 send_to at uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_reference.rb:43
               call_with at uri:classloader:/gems/contracts-0.11.0/lib/contracts/call_with.rb:76
block in redefine_method at uri:classloader:/gems/contracts-0.11.0/lib/contracts/method_handler.rb:138
                  <main> at uri:classloader:/emr-etl-runner/bin/snowplow-emr-etl-runner:41
                    load at org/jruby/RubyKernel.java:994
                  <main> at uri:classloader:/META-INF/main.rb:1
                 require at org/jruby/RubyKernel.java:970
                  (root) at uri:classloader:/META-INF/main.rb:1
                  <main> at uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
ERROR: org.jruby.embed.EvalFailedException: (RequestTimeout) Request Timeout
@BenFradet BenFradet self-assigned this Dec 25, 2018
peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 26, 2020
peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 28, 2020
peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 28, 2020
peel pushed a commit to snowplow/emr-etl-runner that referenced this issue May 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant