2.13.0 regression: "cannot send data if channel has declared eof" #253

Closed
xaviershay opened this Issue Aug 18, 2012 · 8 comments

Projects

None yet

5 participants

@xaviershay

Running deploy:setup:

Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:250:in `send_data': cannot send data if channel has declared eof (EOFError)
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:245:in `block in sudo_behavior_callback'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:235:in `[]'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:235:in `block (4 levels) in open_channels'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `call'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `do_data'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:556:in `channel_data'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:456:in `dispatch_incoming_packets'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:213:in `preprocess'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:17:in `block in process_iteration'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:43:in `block in ensure_each_session'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:41:in `each'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:41:in `ensure_each_session'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/processable.rb:17:in `process_iteration'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:165:in `block (2 levels) in process!'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:164:in `loop'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:164:in `block in process!'
    from /Users/xavier/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:163:in `process!'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/command.rb:134:in `process'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:183:in `block in run_tree'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:198:in `block in execute_on_servers'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `each'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `each_slice'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/connections.rb:186:in `execute_on_servers'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:181:in `run_tree'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:153:in `run'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/invocation.rb:89:in `invoke_command'
    from /Users/xavier/.rvm/gems/ruby-1.9.3-p194/gems/capistrano-2.13.0/lib/capistrano/configuration/actions/inspect.rb:34:in `capture'

(Sorry I don't have a minimal reproducible test case yet.)

Have confirmed that this works on 2.12.0 but is broken on 2.13.0.

@mariovisic

Same issue here, upgrading from 2.12.0 => 2.13.0

@carsomyr
Contributor

Shoot, it looks like this is pretty serious. Maybe you guys could help me diagnose the problem. Here's what I know/speculate:

  1. I run on AWS Ubuntu, which doesn't require a sudo password, and hence I did not try the interactive sudo functionality. What is your setup? I reckon that somewhere down the line there's a passworded sudo invocation. Am I correct?
  2. Does any of the functionality you use ever touch this commit? I know it's the culprit. The pull request creator and I discussed it pretty extensively, so I am going to CC @dylanahsmith.
  3. My best hypothesis is that the some commands that we thought were non-interactive are actually interactive because of passworded sudo behavior. An easy way to confirm this is to strip your Capfile down to a single command running under sudo.

Let me know if the above diagnosis makes sense.

@dylanahsmith
Contributor

@carsomyr you are spot on in your diagnosis. I am very sorry to have caused this regression, but have created a pull request that fixes the issue.

@xaviershay

Yep, we're using interactive sudo. Thanks for the prompt responses!

@carsomyr carsomyr was assigned Aug 19, 2012
@carsomyr carsomyr closed this in 51296cb Aug 20, 2012
@cheister

This problem is still happening for Capistrano 2.13.x versions. Here is the minimal Capfile I came up with. I substituted in testhost.example.com for the real server name.

role(:app) { "testhost.example.com" }

task :sudo_capture do
  default_run_options[:pty] = true
  run("#{sudo :as => 'test-user'} -H -E sh -c 'echo run works'")
  capture("#{sudo :as => 'test-user'} -H -E sh -c 'echo capture works'")
end

resulting in the following output

~ cap sudo_capture
  * executing `sudo_capture'
  * executing "sudo -p 'sudo password: ' -u test-user -H -E sh -c 'echo run works'"
    servers: ["testhost.example.com"]
    [testhost.example.com] executing command
Password: 
 ** [out :: testhost.example.com] 
 ** [out :: testhost.example.com] run works
    command finished in 3271ms
  * executing "sudo -p 'sudo password: ' -u test-user -H -E sh -c 'echo capture works'"
    servers: ["testhost.example.com"]
    [testhost.example.com] executing command
/Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:250:in `send_data': cannot send data if channel has declared eof (EOFError)
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:247:in `block in sudo_behavior_callback'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:235:in `[]'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:235:in `block (4 levels) in open_channels'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `call'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/channel.rb:568:in `do_data'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:556:in `channel_data'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:456:in `dispatch_incoming_packets'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/net-ssh-2.5.2/lib/net/ssh/connection/session.rb:213:in `preprocess'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:17:in `block in process_iteration'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:43:in `block in ensure_each_session'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:41:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:41:in `ensure_each_session'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/processable.rb:17:in `process_iteration'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:165:in `block (2 levels) in process!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:164:in `loop'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:164:in `block in process!'
    from /Users/cheister/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:163:in `process!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/command.rb:134:in `process'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:185:in `block in run_tree'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:198:in `block in execute_on_servers'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `each_slice'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/connections.rb:186:in `execute_on_servers'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:183:in `run_tree'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:155:in `run'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/invocation.rb:89:in `invoke_command'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/actions/inspect.rb:34:in `capture'
    from Capfile:7:in `block in load'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:138:in `instance_eval'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:138:in `invoke_task_directly'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/callbacks.rb:25:in `invoke_task_directly_with_callbacks'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:89:in `execute_task'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/configuration/execution.rb:101:in `find_and_execute_task'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:46:in `block in execute_requested_actions'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:45:in `each'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:45:in `execute_requested_actions'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/help.rb:19:in `execute_requested_actions_with_help'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:34:in `execute!'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/lib/capistrano/cli/execute.rb:14:in `execute'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/gems/capistrano-2.13.3/bin/cap:4:in `<top (required)>'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/bin/cap:23:in `load'
    from /Users/cheister/.rvm/gems/ruby-1.9.3-p194@global/bin/cap:23:in `<main>'
Killed by signal 1.

Currently I've rolled back to 2.12 to fix the issue

@dylanahsmith
Contributor

@cheister Thanks for bringing up the capture with sudo issue, I think I have addressed it in pull request #272.

@carsomyr
Contributor

Aha, another sudo invocation! This will be easy to review...

@carsomyr
Contributor

Thanks again, @dylanahsmith.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment