Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Warning: conflicting chdir during another chdir block #5649

Open
Adithya-copart opened this Issue Mar 12, 2019 · 5 comments

Comments

Projects
None yet
2 participants
@Adithya-copart
Copy link

Adithya-copart commented Mar 12, 2019

Expected Behavior

  • If the following script returns a warning like MRI, It will be helpful.
    MRI is warning the user with warning: conflicting chdir during another chdir block
require 'tempfile'

thr = []
100.times do
  thr << Thread.new do
    Dir.mktmpdir do |dir|
      Dir.chdir(dir) { p `ls -a`}
    end
  end
end

thr.map(&:join)

Actual Behavior

JRuby:

root@28a7559be929:/opt/copart# JRUBY_OPTS=-d jruby create_tmpdir.rb 
".\n..\n"
".\n..\n"
".\n..\n"
".\n..\n"
".\n..\n"
".\n..\n"
".\n..\n"
".\n..\n"
warning: thread "Ruby-0-Thread-3: create_tmpdir.rb:1" terminated with exception (report_on_exception is true):warning: thread "Ruby-0-Thread-5: create_tmpdir.rb:1" terminated with exception (report_on_exception is true):

Errno::ENOENT: No such file or directory - /tmp/d20190312-11478-xl6rt8
             chdir at org/jruby/RubyDir.java:401
  create_tmpdir.rb at create_tmpdir.rb:7
          mktmpdir at /opt/jruby/lib/ruby/stdlib/tmpdir.rb:89
  create_tmpdir.rb at create_tmpdir.rb:6
Errno::ENOENT: No such file or directory - /tmp/d20190312-11478-39p9vj
             chdir at org/jruby/RubyDir.java:401
  create_tmpdir.rb at create_tmpdir.rb:7
          mktmpdir at /opt/jruby/lib/ruby/stdlib/tmpdir.rb:89
  create_tmpdir.rb at create_tmpdir.rb:6
org.jruby.exceptions.SystemCallError: (ENOENT) No such file or directory - /tmp/d20190312-11478-xl6rt8org.jruby.exceptions.SystemCallError: (ENOENT) No such file or directory - /tmp/d20190312-11478-39p9vj

	at org.jruby.RubyDir.chdir(org/jruby/RubyDir.java:401)
	at create_tmpdir.create_tmpdir.rb(create_tmpdir.rb:7)
	at RUBY.mktmpdir(/opt/jruby/lib/ruby/stdlib/tmpdir.rb:89)
	at org.jruby.RubyDir.chdir(org/jruby/RubyDir.java:401)	at create_tmpdir.create_tmpdir.rb(create_tmpdir.rb:6)

	at create_tmpdir.create_tmpdir.rb(create_tmpdir.rb:7)
	at RUBY.mktmpdir(/opt/jruby/lib/ruby/stdlib/tmpdir.rb:89)
	at create_tmpdir.create_tmpdir.rb(create_tmpdir.rb:6)
Errno::ENOENT: No such file or directory - /tmp/d20190312-11478-39p9vj
             chdir at org/jruby/RubyDir.java:401
  create_tmpdir.rb at create_tmpdir.rb:7
          mktmpdir at /opt/jruby/lib/ruby/stdlib/tmpdir.rb:89
  create_tmpdir.rb at create_tmpdir.rb:6

MRI:

adpentela@MB-ADPENTELA ~/Desktop/work/ycs-api (document_management) $ ruby create_tmpdir.rb 
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
create_tmpdir.rb:7: warning: conflicting chdir during another chdir block
".\n..\n"
create_tmpdir

Thanks!

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 12, 2019

Ok I think I know what's happening here. chdir is not thread-safe in any implementation, including MRI, so this code is always going to run into problems. At the very least in MRI you may end up running code against the wrong dir, since some other thread may chdir right after you do. That's why they warn...this code is not expected to work.

In JRuby we do localize the current dir to a given JRuby instance, so that we can support more than one current dir per JVM process, but we do not localize it on a per-thread basis.

The error you're getting here is likely because there are two threads attempting to create, chdir, and remove a temporary dir, and there's a number of ways they could be stepping on each other. It manifests as an ENOENT probably because one of the tmp dirs is getting wiped out while in use. MRI does not manifest the same error because it's at least only running one thread at a time; they still could conflict (and potentially ENOENT) if there's a context switch from one thread to another, but it would be more rare.

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 13, 2019

I have two patches for this. The first just adds the warning, using a ReentrantLock to detect whether some other thread is doing the chdir, but otherwise proceeding without locking. The second version actually does attempt to wait for the lock, making chdir thread-safe.

With the first patch, I do see the warnings you expect, but also the errors you got. This may be the best we can do for this code, since it's rather problematic.

The second patch, as you'd expect, does not present the errors. There's a string of warnings as the threads get going, but then they execute the chdir blocks in serial.

I'm not sure which way to go with this for now. If we want to actually make stacked chdirs block, we probably should raise this as a feature change in MRI as well. If we don't, this code will warn but otherwise still usually error out.

@Adithya-copart

This comment has been minimized.

Copy link
Author

Adithya-copart commented Mar 13, 2019

@headius Thanks for the response and the detailed explanation. The above code-snippet results in ENOENT errors with MRI as well.

I started seeing intermittent ENOENT errors and started with debugging the block passed to chdir, isolated the code and noticed the behavior of chdir.

As part of the JRuby issue template, I tried running the code in MRI and noticed the warning conflicting chdir during another chdir block which gave the impression that it is not thread-safe and a quick google search led me to a five-year old closed issue in MRI about this.

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 13, 2019

@Adithya-copart Thanks for the background research! I'll look into filing an issue with ruby-lang about this, and perhaps we can come to some agreement.

@headius

This comment has been minimized.

Copy link
Member

headius commented Mar 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.