Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

2.13.0 regression: "cannot send data if channel has declared eof" #253

Closed
xaviershay opened this Issue · 8 comments

5 participants

Xavier Shay Mario Visic Roy Liu Dylan Thacker-Smith cheister
Xavier Shay

Running deploy:setup:

Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:250:in `send_data': cannot send data if channel has declared eof (EOFError)
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:245:in `block in sudo_behavior_callback'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:235:in `[]'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:235:in `block (4 levels) in open_channels'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `call'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `do_data'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:556:in `channel_data'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:456:in `dispatch_incoming_packets'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:213:in `preprocess'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:17:in `block in process_iteration'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:43:in `block in ensure_each_session'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:41:in `each'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:41:in `ensure_each_session'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:17:in `process_iteration'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:165:in `block (2 levels) in process!'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:164:in `loop'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:164:in `block in process!'
    from /Users/xavier/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:163:in `process!'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:134:in `process'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:183:in `block in run_tree'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:198:in `block in execute_on_servers'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `each'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `each_slice'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `execute_on_servers'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:181:in `run_tree'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:153:in `run'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:89:in `invoke_command'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/inspect.rb:34:in `capture'

(Sorry I don't have a minimal reproducible test case yet.)

Have confirmed that this works on 2.12.0 but is broken on 2.13.0.

Mario Visic

Same issue here, upgrading from 2.12.0 => 2.13.0

Roy Liu

Shoot, it looks like this is pretty serious. Maybe you guys could help me diagnose the problem. Here's what I know/speculate:

  1. I run on AWS Ubuntu, which doesn't require a sudo password, and hence I did not try the interactive sudo functionality. What is your setup? I reckon that somewhere down the line there's a passworded sudo invocation. Am I correct?
  2. Does any of the functionality you use ever touch this commit? I know it's the culprit. The pull request creator and I discussed it pretty extensively, so I am going to CC @dylanahsmith.
  3. My best hypothesis is that the some commands that we thought were non-interactive are actually interactive because of passworded sudo behavior. An easy way to confirm this is to strip your Capfile down to a single command running under sudo.

Let me know if the above diagnosis makes sense.

Dylan Thacker-Smith

@carsomyr you are spot on in your diagnosis. I am very sorry to have caused this regression, but have created a pull request that fixes the issue.

Xavier Shay

Yep, we're using interactive sudo. Thanks for the prompt responses!

Roy Liu carsomyr was assigned
Roy Liu carsomyr closed this in 51296cb
cheister

This problem is still happening for Capistrano 2.13.x versions. Here is the minimal Capfile I came up with. I substituted in testhost.example.com for the real server name.

role(:app) { "testhost.example.com" }

task :sudo_capture do
  default_run_options[:pty] = true
  run("#{sudo :as => 'test-user'} -H -E sh -c 'echo run works'")
  capture("#{sudo :as => 'test-user'} -H -E sh -c 'echo capture works'")
end

resulting in the following output

~ cap sudo_capture
  * executing `sudo_capture'
  * executing "sudo -p 'sudo password: ' -u test-user -H -E sh -c 'echo run works'"
    servers: ["testhost.example.com"]
    [testhost.example.com] executing command
Password: 
 ** [out :: testhost.example.com] 
 ** [out :: testhost.example.com] run works
    command finished in 3271ms
  * executing "sudo -p 'sudo password: ' -u test-user -H -E sh -c 'echo capture works'"
    servers: ["testhost.example.com"]
    [testhost.example.com] executing command
/Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:250:in `send_data': cannot send data if channel has declared eof (EOFError)
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:247:in `block in sudo_behavior_callback'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:235:in `[]'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:235:in `block (4 levels) in open_channels'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `call'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `do_data'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:556:in `channel_data'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:456:in `dispatch_incoming_packets'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:213:in `preprocess'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:17:in `block in process_iteration'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:43:in `block in ensure_each_session'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:41:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:41:in `ensure_each_session'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:17:in `process_iteration'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:165:in `block (2 levels) in process!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:164:in `loop'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:164:in `block in process!'
    from /Users/cheister/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:163:in `process!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:134:in `process'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:185:in `block in run_tree'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:198:in `block in execute_on_servers'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `each_slice'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `execute_on_servers'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:183:in `run_tree'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:155:in `run'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:89:in `invoke_command'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/inspect.rb:34:in `capture'
    from Capfile:7:in `block in load'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:46:in `block in execute_requested_actions'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:45:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:45:in `execute_requested_actions'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/help.rb:19:in `execute_requested_actions_with_help'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:34:in `execute!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:14:in `execute'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/bin/cap:4:in `<top (required)>'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/bin/cap:23:in `load'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/bin/cap:23:in `<main>'
Killed by signal 1.

Currently I've rolled back to 2.12 to fix the issue

Dylan Thacker-Smith

@cheister Thanks for bringing up the capture with sudo issue, I think I have addressed it in pull request #272.

Roy Liu

Aha, another sudo invocation! This will be easy to review...

Roy Liu

Thanks again, @dylanahsmith.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.