Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline shutdown can be blocked by stdin input #1769

Closed
jordansissel opened this issue Sep 23, 2014 · 16 comments

Comments

Projects
None yet
7 participants
@jordansissel
Copy link
Contributor

commented Sep 23, 2014

For whatever reason, I can't figure out how to close stdin in JRuby and interrupt any currently-blocked reads on stdin.

This confuses users and is generally annoying.

The symptom is that ^C should terminate logstash cleanly, but stdin never really stops so the pipeline stays up until you close stdin as a user (via ^D)

(related: #1767)

@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented Sep 23, 2014

From another thread, I have attempted:

  • java.lang.System.in.close()
  • $stdin.close
  • Thread#raise
  • IO::select([$stdin], ...)

None are effective. Closing doesn't interrupt any active reads. IO::select always says $stdin is ready to read even when there is no data.

@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented Sep 23, 2014

Contrast to MRI 2.1.2:

  • $stdin.sysread(1) - gets a RuntimeError: "`sysread': unhandled exception"
  • IO.select([$stdin], nil, nil, nil) - gets same error as above, except it says `select'.

@colinsurprenant colinsurprenant added this to the v1.5.0 milestone Sep 23, 2014

@colinsurprenant colinsurprenant self-assigned this Sep 23, 2014

@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented Oct 2, 2014

rvm default do ruby -e 't = Thread.new { begin ; p :sysread => $stdin.sysread(1); puts "Done"; rescue => e; p :e => e; raise; end }; sleep 3; puts "Closing $stdin"; $stdin.close; puts "Interrupting "; t.raise; t.join'

@talevy

This comment has been minimized.

Copy link
Contributor

commented Nov 11, 2014

@suyograo suyograo added the v2.0.0 label Nov 11, 2014

@suyograo suyograo modified the milestones: 2.0, v1.5.0 Nov 11, 2014

@colinsurprenant

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2014

Just a few notes, as I looked at this a bit last evening.

  • stdin but stdio in general is not selectable in Java NIO. (Not sure if it is in NIO2). Doing IO.select([$stdin], nil, nil, 0) in JRuby will not work as intended.
  • blocking read on stdin in a thread in Java NIO is non-interruptible, period. ❗️

The only options I see for this are

  • if NIO2 support selectable stdio we could implement this using NIO2, but only for Java 7+
  • implement our own stdin reader either by just wrapping needed POSIX system calls with FFI and maybe injecting a little C for better performance.
@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented Nov 26, 2014

(reposting from hipchat)
+1 on FFI because it will work on both MRI and JRuby.

@seanfahey

This comment has been minimized.

Copy link

commented Mar 10, 2015

Is it possible to have the Getting Started documentation updated, until the issue is fixed? http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash in the "Logstash in two commands" section.

@josegonzalez

This comment has been minimized.

Copy link

commented Mar 11, 2015

EDIT: Seems my issue is actually #1779. Sorry!

Note: I'm unable to get it to close, even with CTRL+D. I have to manually kill the agent using kill -term or kill -9. I'm running under the openjre in case that helps.

Also, here is the config i'm running with, which doesn't use stdio as far as I can tell:

/opt/logstash/bin/logstash -e 'input { redis { key => "metricsd" data_type => "list" } } output { elasticsearch { host => localhost } stdout { codec => "json" } }'

When I had it running under stdin instead of redis, I was able to kill the agent, though it did take a while for it to die.

I can also provide a Vagrantfile where this issue is reproducible (at least for me) if that helps.

@colinsurprenant

This comment has been minimized.

Copy link
Contributor

commented Apr 1, 2015

@josegonzalez for the shutdown + redis issue see #2871

@colinsurprenant

This comment has been minimized.

Copy link
Contributor

commented May 13, 2015

just a quick update to say I found a way to extract an InterruptibleChannel from System.in - I'll be making a test integration in the stdin input plugin which should allow a simple ^C to correctly shutdown logstash when using stdin input.

I looks like this solution only works on Java 8 though.

@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2015

@colinsurprenant I'm OK with this - Java 7 is EOL anyway, so I think it's reasonable to expect that Java 8 should become more common very soon.

@jordansissel

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2015

<3 you so much for finding this solution

@josegonzalez

This comment has been minimized.

Copy link

commented May 14, 2015

Yay java 8! Good work @colinsurprenant :)

@colinsurprenant

This comment has been minimized.

Copy link
Contributor

commented May 15, 2015

landed the jruby-stdin-channel gem for it. just need to insert that into the stdin input and see how that behave :P

@colinsurprenant

This comment has been minimized.

Copy link
Contributor

commented May 17, 2015

inserted in the stdin input in logstash-plugins/logstash-input-stdin#3

@suyograo suyograo removed this from the v2.0.0 milestone Jun 18, 2015

@suyograo suyograo added v2.1.0 v5.0.0 and removed v2.0.0 v2.1.0 labels Oct 15, 2015

@jsvd

This comment has been minimized.

Copy link
Member

commented Mar 11, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.