Skip to content
This repository

Backport of bug fixes from master to jruby-1_6 #244

Merged
merged 40 commits into from over 1 year ago

11 participants

David Kellum Charles Oliver Nutter jrubyci Hiro Asari Uwe Kubosch Hiroshi Nakamura James Abley Ben Browning Thomas E Enebo Andy Lindeman Yoko Harada
David Kellum

I was able to cherry-pick the following 35 fixes from master on to jruby-1_6 with no conflicts and with all tests passing for each addition. Proposal for a 1.6.8 release forthcoming. In any case are these all well accepted, conservative fixes?

Note that I simply rejected anything that wasn't labelled a fix or that didn't apply cleanly. There might be some additional fix opportunities that would just require straight forward conflict resolution. Please let me know if you know of any you would like me to try that on and I'll add them here.

and others added some commits September 18, 2011
Charles Oliver Nutter Add rdoc to gems we install for dist, since hoe appears to depend on …
…it now.
40e87d7
Charles Oliver Nutter Fix JRUBY-6489
FileLock.release() throwing Java::JavaLang::NullPointerException

Strategy works like this...we may want a more conservative change
for jruby-1_6 branch:

* Check if lock state would change as a result. If not, return success
* Synchronize on FileChannel
* Check again
* Perform operation

This should reduce the cost of repeated locks and unlocks to
nearly zero, make concurrent locking/unlocking safe, and localize
synchronization to only the code that needs it.
202a45c
Hiro Asari Fix JRUBY-6538: 'obj !~ thing' is not the same as obj.send(:!~, thing)
Credit goes to headius.
1ef14f1
Uwe Kubosch * JRUBY-6560 Fix File.expand_path for jar:file:/ paths 358883f
Hiro Asari Implement Kernel#Hash. Fixes JRUBY-6496. 61f2cf0
Hiroshi Nakamura JRUBY-6559: GzipReader#rewind crashes when the input stream is serial…
…ly-multiplexed like RubyGems

This commit adds regression spec.  The actual fix is at
539f45c.
7b370c2
Hiro Asari Fix JRUBY-6571: StackOverflow when calling java.util.regex.Matcher#end fd9ca2e
Charles Oliver Nutter Fix JRUBY-5863: Named captures cause crash when there is no match
Compiler calls match2AndUpdateScope[19], which did not pass through
the match result and instead fell back to logic that used the
scope's match and assumed it was non-nill.
c591134
Charles Oliver Nutter Fix JRUBY-6570: autoload :Time, 'time' doesn't work
Instead of always using Object when calling autoload from an
instance method, we use self's real class (skipping singletons)
as the autoload location. This isn't exactly how MRI does it
(they use cbase, which I think is euivalent to 1.8's ruby_class),
but it should be correct for the majority of cases.
fc17ab4
Charles Oliver Nutter Regression test for JRUBY-5863. 22a52eb
Charles Oliver Nutter Mitigation for JRUBY-6576
Because in JRuby, concurrent selects can trigger blocking mode
exceptions, I've modified IO.select logic to raise a concurrency
error when multiple threads try to select on the same sockets at
the same time (and fail to do so cleanly). At present, there's
no way to do this since we use these channels in both blocking and
nonblocking mode; we can't synchronize the blocking mode changes
with multiple threads all trying to do select operations.
cbbbf38
Hiro Asari Add spec for JRUBY-6571 4b462fd
Hiro Asari Add copyright notice for Yecht. This resolves JRUBY-4843. 2e6d4c8
Uwe Kubosch * JRUBY-6624 Strip jar: prefix when searching for files in a JAR
Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
defb411
Uwe Kubosch * JRUBY-6626 Ensure thread is set on finalize b8e182d
Charles Oliver Nutter Regression spec for JRUBY-6634. 2ce65e1
James Abley JRUBY-6611: Fix resource leak in LoadServiceResource 0fe664a
Ben Browning Check if all throwables are Java exceptions in interpreted mode (JRUB…
…Y-6641)

This fixes a simple typo in the RuntimeHelpers.isJavaExceptionHandled logic.
8623090
Remove -mimpure-text from CFLAGS. Fixes JRUBY-6633. 7153b89
Thomas E Enebo JRUBY-6464: Use "mvn versions:set -DnewVersion=<whatever>" instead of…
… our Ruby rake code for updating poms
82a2f2d
Charles Oliver Nutter Fix JRUBY-6642.
RubyException.printBacktrace will now print all lines by default,
and has a separate form for skipping some number of lines. All
internal uses that expected the first line to be skipped use the
second form.
9a122ed
Charles Oliver Nutter Fix JRUBY-6517
This seems to be a flaw in the patch I provided for
http://bugs.ruby-lang.org/issues/5698. Since that patch was only
to improve perf of REXML in Ruby 1.8 mode, I think it's better to
have slower and working than faster and broken. Reverting.
280e435
Charles Oliver Nutter Fix JRUBY-6248
I'm committing a cardinal sin here by using Thread#stop, but I
think in this case it's warranted (and probably safe enough). The
threads in question are doing nothing but spinning on two IO
streams, and even if they leave those streams unusable we're
walking away from them anyway.

This fix makes thread count stay stable for a tight backquote loop
as described in the bug.

I did try using our Executor, but the problem isn't reusing
threads, it's terminating them completely.
5f65879
Thomas E Enebo JRUBY-6708: NullPointerException instead of proper exceptions output c82a360
Hiro Asari Remove Enumerable#join. This resolves JRUBY-6703. 566ae5d
Andy Lindeman Fixes JRUBY-6677
* Passes the `mode` argument correctly from Kernel#open to IO#popen, so
  if a non-default mode is specified (e.g., "w"), it is honored

Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
df5b336
Charles Oliver Nutter Fix JRUBY-6706
Collections.shuffle works incorreclty on Ruby arrays

We were not implementing List.set properly; it should return the
previous value at that index, and we were just returning the
given value.
ddc48ae
Thomas E Enebo Errant printf removed a62f4af
Yoko Harada Fix for JRUBY-6728. This fixes Nokogiri's 1.9 test error of PrettyPri…
…nt as well as sparklemotion/nokogiri#657.

The change didn't break test:mri19. However, oddly, test_condvar_wait_exception_handling of test/externals/ruby1.9/ruby/test_thread.rb has become failing often  on my macbook. But, not always. This change might have revealed threadding issue?
0324c3d
Charles Oliver Nutter Fix JRUBY-6745
ClassCastException calling Fixnum#== with a Java Integer

The problem here was that when given multiple overloads that are
all dispatchable with a given set of arguments, we just picked the
first one. This specific case had both compareTo(Object) and
compareTo(Integer), and we picked the Object version. Because the
coercion from Fixnum to Object produces a Long, we eventually hit
a ClassCastException when the Object version tried to call the
Integer version. By fixing Java method dispatch to narrow
overloads to the most specific version, we now will call the
Integer version and coerce properly.

I believe this was working in 1.6.7 because we had other bugs
relating to how we coerce versus how we dispatch that caused us
to coerce to Integer but still call the Object version.
44b8f31
Charles Oliver Nutter Fix for JRUBY-6725. See jruby/ruby repo for description. f9b745f
Charles Oliver Nutter Fix JRUBY-6724
NO_FILE_CLASS Profile does disables File but not FileStats that depends on File

Fix NO_FILE_CLASS profile to also disable FileStat.
400c73a
Charles Oliver Nutter Fix JRUBY-6722
Can't instantiate ThreadGroup subclass with arguments

We had defined 'new' on ThreadGroup, when what we needed was an
ObjectAllocator for the normal 'new' logic to work. With that fix,
subclass initialize methods work fine.
98724c3
Charles Oliver Nutter Fix JRUBY-6753
class_eval should inherit __name__ from the caller

class_eval/module_eval were still being framed, which caused them
to interfere with getting the actual caller name. Removed the
framing and the bug is fixed.
413edcb
Charles Oliver Nutter Fix JRUBY-6775: RubyRandom is not thread-safe
I opted to simply synchronize around the logic in question. The
primary issue is the potential for "left" to get decremented below
zero under concurrency while it is in use. I tried to figure out
an AtomicInteger-based algorithm for guaranteeing both that left
would never cause a <0 array dereference *and* would not produce
the same random number twice, but was unable to find a way due
to the multiple state changes that happen.

Synchronization does slow things down a bit, but perhaps not
enough to worry about:

Before (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  3.670000   0.130000   3.800000 (  3.317000)
  3.880000   0.150000   4.030000 (  3.761000)
  4.550000   0.150000   4.700000 (  4.495000)
  4.500000   0.160000   4.660000 (  4.460000)
  4.500000   0.150000   4.650000 (  4.452000)

After (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  4.420000   0.130000   4.550000 (  4.031000)
  4.460000   0.170000   4.630000 (  4.353000)
  5.000000   0.160000   5.160000 (  4.954000)
  4.980000   0.160000   5.140000 (  4.924000)
  5.070000   0.160000   5.230000 (  5.020000)

After (100 threads, same workload):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(100).map {Thread.new{500000.times{rand}}}.map(&:join) } }'
 13.900000   5.400000  19.300000 (  8.227000)
 12.900000   5.250000  18.150000 (  7.938000)
 13.800000   5.680000  19.480000 (  8.903000)
 13.540000   5.750000  19.290000 (  8.627000)
 13.390000   5.560000  18.950000 (  8.331000)
684ce37
Uwe Kubosch * JRUBY-6785 Allow ScriptingContainer#callMethod to be called on a Ja…
…va object

* Allow ScriptingContainer#callMethod to be called using nil as a receiver
a90bb11
Uwe Kubosch * JRUBY-6785 Make the receiver for callMethod more consistent
* Allow nil as a receiver
* Allow Java objects as receivers.  These will be wrapped before use.
7c8dba1
Charles Oliver Nutter Fix JRUBY-6731
Can't load class files in WARs compiled by warbler using jruby 1.7.0.preview1

When requiring a file with an explicit .class extension, assume
the .class filename is the only filename to be searched.

Because a require of .class almost always will indicate that the
user wants to load a specific precompiled Ruby file, there's no
good reason (I know of) to search for anything else.

The original logic did not work at all; require 'foo.class' would
search for 'foo.class.rb' and 'foo.class.class' (in that order)
but never 'foo.class'.

The claim is that this worked in 1.6.7.2, but I could not
reproduce that with a simple case. The logic involved does not
appear to have changed in a substantial way since well before
1.6.7.2, and in any case I believe the fix here is the right way
for it to work.
86ef07e
Charles Oliver Nutter Fix JRUBY-6395: Coerce Fixnum to Number
Simple enough fix: added Number (and Serializable) to the map of
types to converters in JavaUtil. Because our Numeric types were
already reporting that their "natural" classes were Number
subclasses, they were being selected for Number-receiving methods.
But the coercion logic did not know about Number, so it failed.
b8d0ca6
Charles Oliver Nutter Fix JRUBY-6514
java.lang.NullPointerException when using ScriptingCotainer from an Applet

Properties retrieved in a restricted environment may be null. We
must handle that case and not blindly try to getBytes().
8115b8c
Charles Oliver Nutter
Owner

Ok, time to broach the issue of the long delay in releases since 1.6.7, I guess.

We are not opposed to merging fixes into the 1.6 branch, but because we're busy with 1.7 release cycle now it's unlikely that we will have the cycles necessary to do a 1.6.8 release.

However...a few folks have proposed that we allow community-managed releases of earlier releases. We would not be responsible for them and would not make any guarantees about them, but if someone else is willing to go through the process of validating and building new releases, maybe it's worth exploring.

I'll talk to @enebo about it today.

Charles Oliver Nutter
Owner

Ok...so after talking with @enebo, we both agree it's worth doing a 1.6.8 release if we can get community help merging fixes over. This means you and others.

I'm going to merge this request in, and we'll happily accept more for as many fixes as re easily portable to 1.6.8. I would probably recommend avoiding fixes for 1.9 issues unless they're small and easy to port over, since master has changed substantially since 1.6.7 and many fixes will require much more complicated and error-prone merging. But fixes that are small or for 1.8.7 logic are open game, as are fixes to subsystems that have not changed significantly like Java integration.

jrubyci jrubyci merged commit 8115b8c into from August 06, 2012
jrubyci jrubyci closed this August 06, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 40 unique commits by 11 authors.

Aug 02, 2012
Charles Oliver Nutter Add rdoc to gems we install for dist, since hoe appears to depend on …
…it now.
40e87d7
Charles Oliver Nutter Fix JRUBY-6489
FileLock.release() throwing Java::JavaLang::NullPointerException

Strategy works like this...we may want a more conservative change
for jruby-1_6 branch:

* Check if lock state would change as a result. If not, return success
* Synchronize on FileChannel
* Check again
* Perform operation

This should reduce the cost of repeated locks and unlocks to
nearly zero, make concurrent locking/unlocking safe, and localize
synchronization to only the code that needs it.
202a45c
Hiro Asari Fix JRUBY-6538: 'obj !~ thing' is not the same as obj.send(:!~, thing)
Credit goes to headius.
1ef14f1
Uwe Kubosch * JRUBY-6560 Fix File.expand_path for jar:file:/ paths 358883f
Hiro Asari Implement Kernel#Hash. Fixes JRUBY-6496. 61f2cf0
Hiroshi Nakamura JRUBY-6559: GzipReader#rewind crashes when the input stream is serial…
…ly-multiplexed like RubyGems

This commit adds regression spec.  The actual fix is at
539f45c.
7b370c2
Hiro Asari Fix JRUBY-6571: StackOverflow when calling java.util.regex.Matcher#end fd9ca2e
Charles Oliver Nutter Fix JRUBY-5863: Named captures cause crash when there is no match
Compiler calls match2AndUpdateScope[19], which did not pass through
the match result and instead fell back to logic that used the
scope's match and assumed it was non-nill.
c591134
Charles Oliver Nutter Fix JRUBY-6570: autoload :Time, 'time' doesn't work
Instead of always using Object when calling autoload from an
instance method, we use self's real class (skipping singletons)
as the autoload location. This isn't exactly how MRI does it
(they use cbase, which I think is euivalent to 1.8's ruby_class),
but it should be correct for the majority of cases.
fc17ab4
Charles Oliver Nutter Regression test for JRUBY-5863. 22a52eb
Charles Oliver Nutter Mitigation for JRUBY-6576
Because in JRuby, concurrent selects can trigger blocking mode
exceptions, I've modified IO.select logic to raise a concurrency
error when multiple threads try to select on the same sockets at
the same time (and fail to do so cleanly). At present, there's
no way to do this since we use these channels in both blocking and
nonblocking mode; we can't synchronize the blocking mode changes
with multiple threads all trying to do select operations.
cbbbf38
Hiro Asari Add spec for JRUBY-6571 4b462fd
Hiro Asari Add copyright notice for Yecht. This resolves JRUBY-4843. 2e6d4c8
Uwe Kubosch * JRUBY-6624 Strip jar: prefix when searching for files in a JAR
Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
defb411
Uwe Kubosch * JRUBY-6626 Ensure thread is set on finalize b8e182d
Charles Oliver Nutter Regression spec for JRUBY-6634. 2ce65e1
James Abley JRUBY-6611: Fix resource leak in LoadServiceResource 0fe664a
Ben Browning Check if all throwables are Java exceptions in interpreted mode (JRUB…
…Y-6641)

This fixes a simple typo in the RuntimeHelpers.isJavaExceptionHandled logic.
8623090
Remove -mimpure-text from CFLAGS. Fixes JRUBY-6633. 7153b89
Thomas E Enebo JRUBY-6464: Use "mvn versions:set -DnewVersion=<whatever>" instead of…
… our Ruby rake code for updating poms
82a2f2d
Charles Oliver Nutter Fix JRUBY-6642.
RubyException.printBacktrace will now print all lines by default,
and has a separate form for skipping some number of lines. All
internal uses that expected the first line to be skipped use the
second form.
9a122ed
Charles Oliver Nutter Fix JRUBY-6517
This seems to be a flaw in the patch I provided for
http://bugs.ruby-lang.org/issues/5698. Since that patch was only
to improve perf of REXML in Ruby 1.8 mode, I think it's better to
have slower and working than faster and broken. Reverting.
280e435
Charles Oliver Nutter Fix JRUBY-6248
I'm committing a cardinal sin here by using Thread#stop, but I
think in this case it's warranted (and probably safe enough). The
threads in question are doing nothing but spinning on two IO
streams, and even if they leave those streams unusable we're
walking away from them anyway.

This fix makes thread count stay stable for a tight backquote loop
as described in the bug.

I did try using our Executor, but the problem isn't reusing
threads, it's terminating them completely.
5f65879
Thomas E Enebo JRUBY-6708: NullPointerException instead of proper exceptions output c82a360
Hiro Asari Remove Enumerable#join. This resolves JRUBY-6703. 566ae5d
Andy Lindeman Fixes JRUBY-6677
* Passes the `mode` argument correctly from Kernel#open to IO#popen, so
  if a non-default mode is specified (e.g., "w"), it is honored

Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
df5b336
Charles Oliver Nutter Fix JRUBY-6706
Collections.shuffle works incorreclty on Ruby arrays

We were not implementing List.set properly; it should return the
previous value at that index, and we were just returning the
given value.
ddc48ae
Thomas E Enebo Errant printf removed a62f4af
Yoko Harada Fix for JRUBY-6728. This fixes Nokogiri's 1.9 test error of PrettyPri…
…nt as well as sparklemotion/nokogiri#657.

The change didn't break test:mri19. However, oddly, test_condvar_wait_exception_handling of test/externals/ruby1.9/ruby/test_thread.rb has become failing often  on my macbook. But, not always. This change might have revealed threadding issue?
0324c3d
Charles Oliver Nutter Fix JRUBY-6745
ClassCastException calling Fixnum#== with a Java Integer

The problem here was that when given multiple overloads that are
all dispatchable with a given set of arguments, we just picked the
first one. This specific case had both compareTo(Object) and
compareTo(Integer), and we picked the Object version. Because the
coercion from Fixnum to Object produces a Long, we eventually hit
a ClassCastException when the Object version tried to call the
Integer version. By fixing Java method dispatch to narrow
overloads to the most specific version, we now will call the
Integer version and coerce properly.

I believe this was working in 1.6.7 because we had other bugs
relating to how we coerce versus how we dispatch that caused us
to coerce to Integer but still call the Object version.
44b8f31
Charles Oliver Nutter Fix for JRUBY-6725. See jruby/ruby repo for description. f9b745f
Charles Oliver Nutter Fix JRUBY-6724
NO_FILE_CLASS Profile does disables File but not FileStats that depends on File

Fix NO_FILE_CLASS profile to also disable FileStat.
400c73a
Charles Oliver Nutter Fix JRUBY-6722
Can't instantiate ThreadGroup subclass with arguments

We had defined 'new' on ThreadGroup, when what we needed was an
ObjectAllocator for the normal 'new' logic to work. With that fix,
subclass initialize methods work fine.
98724c3
Charles Oliver Nutter Fix JRUBY-6753
class_eval should inherit __name__ from the caller

class_eval/module_eval were still being framed, which caused them
to interfere with getting the actual caller name. Removed the
framing and the bug is fixed.
413edcb
Charles Oliver Nutter Fix JRUBY-6775: RubyRandom is not thread-safe
I opted to simply synchronize around the logic in question. The
primary issue is the potential for "left" to get decremented below
zero under concurrency while it is in use. I tried to figure out
an AtomicInteger-based algorithm for guaranteeing both that left
would never cause a <0 array dereference *and* would not produce
the same random number twice, but was unable to find a way due
to the multiple state changes that happen.

Synchronization does slow things down a bit, but perhaps not
enough to worry about:

Before (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  3.670000   0.130000   3.800000 (  3.317000)
  3.880000   0.150000   4.030000 (  3.761000)
  4.550000   0.150000   4.700000 (  4.495000)
  4.500000   0.160000   4.660000 (  4.460000)
  4.500000   0.150000   4.650000 (  4.452000)

After (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  4.420000   0.130000   4.550000 (  4.031000)
  4.460000   0.170000   4.630000 (  4.353000)
  5.000000   0.160000   5.160000 (  4.954000)
  4.980000   0.160000   5.140000 (  4.924000)
  5.070000   0.160000   5.230000 (  5.020000)

After (100 threads, same workload):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(100).map {Thread.new{500000.times{rand}}}.map(&:join) } }'
 13.900000   5.400000  19.300000 (  8.227000)
 12.900000   5.250000  18.150000 (  7.938000)
 13.800000   5.680000  19.480000 (  8.903000)
 13.540000   5.750000  19.290000 (  8.627000)
 13.390000   5.560000  18.950000 (  8.331000)
684ce37
Uwe Kubosch * JRUBY-6785 Allow ScriptingContainer#callMethod to be called on a Ja…
…va object

* Allow ScriptingContainer#callMethod to be called using nil as a receiver
a90bb11
Uwe Kubosch * JRUBY-6785 Make the receiver for callMethod more consistent
* Allow nil as a receiver
* Allow Java objects as receivers.  These will be wrapped before use.
7c8dba1
Charles Oliver Nutter Fix JRUBY-6731
Can't load class files in WARs compiled by warbler using jruby 1.7.0.preview1

When requiring a file with an explicit .class extension, assume
the .class filename is the only filename to be searched.

Because a require of .class almost always will indicate that the
user wants to load a specific precompiled Ruby file, there's no
good reason (I know of) to search for anything else.

The original logic did not work at all; require 'foo.class' would
search for 'foo.class.rb' and 'foo.class.class' (in that order)
but never 'foo.class'.

The claim is that this worked in 1.6.7.2, but I could not
reproduce that with a simple case. The logic involved does not
appear to have changed in a substantial way since well before
1.6.7.2, and in any case I believe the fix here is the right way
for it to work.
86ef07e
Charles Oliver Nutter Fix JRUBY-6395: Coerce Fixnum to Number
Simple enough fix: added Number (and Serializable) to the map of
types to converters in JavaUtil. Because our Numeric types were
already reporting that their "natural" classes were Number
subclasses, they were being selected for Number-receiving methods.
But the coercion logic did not know about Number, so it failed.
b8d0ca6
Charles Oliver Nutter Fix JRUBY-6514
java.lang.NullPointerException when using ScriptingCotainer from an Applet

Properties retrieved in a restricted environment may be null. We
must handle that case and not blindly try to getBytes().
8115b8c
Something went wrong with that request. Please try again.