Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Backport of bug fixes from master to jruby-1_6 #244

Merged
merged 40 commits into from
@dekellum

I was able to cherry-pick the following 35 fixes from master on to jruby-1_6 with no conflicts and with all tests passing for each addition. Proposal for a 1.6.8 release forthcoming. In any case are these all well accepted, conservative fixes?

Note that I simply rejected anything that wasn't labelled a fix or that didn't apply cleanly. There might be some additional fix opportunities that would just require straight forward conflict resolution. Please let me know if you know of any you would like me to try that on and I'll add them here.

headius and others added some commits
@headius headius Add rdoc to gems we install for dist, since hoe appears to depend on …
…it now.
40e87d7
@headius headius Fix JRUBY-6489
FileLock.release() throwing Java::JavaLang::NullPointerException

Strategy works like this...we may want a more conservative change
for jruby-1_6 branch:

* Check if lock state would change as a result. If not, return success
* Synchronize on FileChannel
* Check again
* Perform operation

This should reduce the cost of repeated locks and unlocks to
nearly zero, make concurrent locking/unlocking safe, and localize
synchronization to only the code that needs it.
202a45c
@BanzaiMan BanzaiMan Fix JRUBY-6538: 'obj !~ thing' is not the same as obj.send(:!~, thing)
Credit goes to headius.
1ef14f1
@donv donv * JRUBY-6560 Fix File.expand_path for jar:file:/ paths 358883f
@BanzaiMan BanzaiMan Implement Kernel#Hash. Fixes JRUBY-6496. 61f2cf0
@nahi nahi JRUBY-6559: GzipReader#rewind crashes when the input stream is serial…
…ly-multiplexed like RubyGems

This commit adds regression spec.  The actual fix is at
539f45c.
7b370c2
@BanzaiMan BanzaiMan Fix JRUBY-6571: StackOverflow when calling java.util.regex.Matcher#end fd9ca2e
@headius headius Fix JRUBY-5863: Named captures cause crash when there is no match
Compiler calls match2AndUpdateScope[19], which did not pass through
the match result and instead fell back to logic that used the
scope's match and assumed it was non-nill.
c591134
@headius headius Fix JRUBY-6570: autoload :Time, 'time' doesn't work
Instead of always using Object when calling autoload from an
instance method, we use self's real class (skipping singletons)
as the autoload location. This isn't exactly how MRI does it
(they use cbase, which I think is euivalent to 1.8's ruby_class),
but it should be correct for the majority of cases.
fc17ab4
@headius headius Regression test for JRUBY-5863. 22a52eb
@headius headius Mitigation for JRUBY-6576
Because in JRuby, concurrent selects can trigger blocking mode
exceptions, I've modified IO.select logic to raise a concurrency
error when multiple threads try to select on the same sockets at
the same time (and fail to do so cleanly). At present, there's
no way to do this since we use these channels in both blocking and
nonblocking mode; we can't synchronize the blocking mode changes
with multiple threads all trying to do select operations.
cbbbf38
@BanzaiMan BanzaiMan Add spec for JRUBY-6571 4b462fd
@BanzaiMan BanzaiMan Add copyright notice for Yecht. This resolves JRUBY-4843. 2e6d4c8
@donv donv * JRUBY-6624 Strip jar: prefix when searching for files in a JAR
Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
defb411
@donv donv * JRUBY-6626 Ensure thread is set on finalize b8e182d
@headius headius Regression spec for JRUBY-6634. 2ce65e1
@jabley jabley JRUBY-6611: Fix resource leak in LoadServiceResource 0fe664a
@bbrowning bbrowning Check if all throwables are Java exceptions in interpreted mode (JRUB…
…Y-6641)

This fixes a simple typo in the RuntimeHelpers.isJavaExceptionHandled logic.
8623090
Wayne Meissner Remove -mimpure-text from CFLAGS. Fixes JRUBY-6633. 7153b89
@enebo enebo JRUBY-6464: Use "mvn versions:set -DnewVersion=<whatever>" instead of…
… our Ruby rake code for updating poms
82a2f2d
@headius headius Fix JRUBY-6642.
RubyException.printBacktrace will now print all lines by default,
and has a separate form for skipping some number of lines. All
internal uses that expected the first line to be skipped use the
second form.
9a122ed
@headius headius Fix JRUBY-6517
This seems to be a flaw in the patch I provided for
http://bugs.ruby-lang.org/issues/5698. Since that patch was only
to improve perf of REXML in Ruby 1.8 mode, I think it's better to
have slower and working than faster and broken. Reverting.
280e435
@headius headius Fix JRUBY-6248
I'm committing a cardinal sin here by using Thread#stop, but I
think in this case it's warranted (and probably safe enough). The
threads in question are doing nothing but spinning on two IO
streams, and even if they leave those streams unusable we're
walking away from them anyway.

This fix makes thread count stay stable for a tight backquote loop
as described in the bug.

I did try using our Executor, but the problem isn't reusing
threads, it's terminating them completely.
5f65879
@enebo enebo JRUBY-6708: NullPointerException instead of proper exceptions output c82a360
@BanzaiMan BanzaiMan Remove Enumerable#join. This resolves JRUBY-6703. 566ae5d
@alindeman alindeman Fixes JRUBY-6677
* Passes the `mode` argument correctly from Kernel#open to IO#popen, so
  if a non-default mode is specified (e.g., "w"), it is honored

Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
df5b336
@headius headius Fix JRUBY-6706
Collections.shuffle works incorreclty on Ruby arrays

We were not implementing List.set properly; it should return the
previous value at that index, and we were just returning the
given value.
ddc48ae
@enebo enebo Errant printf removed a62f4af
@yokolet yokolet Fix for JRUBY-6728. This fixes Nokogiri's 1.9 test error of PrettyPri…
…nt as well as sparklemotion/nokogiri#657.

The change didn't break test:mri19. However, oddly, test_condvar_wait_exception_handling of test/externals/ruby1.9/ruby/test_thread.rb has become failing often  on my macbook. But, not always. This change might have revealed threadding issue?
0324c3d
@headius headius Fix JRUBY-6745
ClassCastException calling Fixnum#== with a Java Integer

The problem here was that when given multiple overloads that are
all dispatchable with a given set of arguments, we just picked the
first one. This specific case had both compareTo(Object) and
compareTo(Integer), and we picked the Object version. Because the
coercion from Fixnum to Object produces a Long, we eventually hit
a ClassCastException when the Object version tried to call the
Integer version. By fixing Java method dispatch to narrow
overloads to the most specific version, we now will call the
Integer version and coerce properly.

I believe this was working in 1.6.7 because we had other bugs
relating to how we coerce versus how we dispatch that caused us
to coerce to Integer but still call the Object version.
44b8f31
@headius headius Fix for JRUBY-6725. See jruby/ruby repo for description. f9b745f
@headius headius Fix JRUBY-6724
NO_FILE_CLASS Profile does disables File but not FileStats that depends on File

Fix NO_FILE_CLASS profile to also disable FileStat.
400c73a
@headius headius Fix JRUBY-6722
Can't instantiate ThreadGroup subclass with arguments

We had defined 'new' on ThreadGroup, when what we needed was an
ObjectAllocator for the normal 'new' logic to work. With that fix,
subclass initialize methods work fine.
98724c3
@headius headius Fix JRUBY-6753
class_eval should inherit __name__ from the caller

class_eval/module_eval were still being framed, which caused them
to interfere with getting the actual caller name. Removed the
framing and the bug is fixed.
413edcb
@headius headius Fix JRUBY-6775: RubyRandom is not thread-safe
I opted to simply synchronize around the logic in question. The
primary issue is the potential for "left" to get decremented below
zero under concurrency while it is in use. I tried to figure out
an AtomicInteger-based algorithm for guaranteeing both that left
would never cause a <0 array dereference *and* would not produce
the same random number twice, but was unable to find a way due
to the multiple state changes that happen.

Synchronization does slow things down a bit, but perhaps not
enough to worry about:

Before (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  3.670000   0.130000   3.800000 (  3.317000)
  3.880000   0.150000   4.030000 (  3.761000)
  4.550000   0.150000   4.700000 (  4.495000)
  4.500000   0.160000   4.660000 (  4.460000)
  4.500000   0.150000   4.650000 (  4.452000)

After (single thread):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
  4.420000   0.130000   4.550000 (  4.031000)
  4.460000   0.170000   4.630000 (  4.353000)
  5.000000   0.160000   5.160000 (  4.954000)
  4.980000   0.160000   5.140000 (  4.924000)
  5.070000   0.160000   5.230000 (  5.020000)

After (100 threads, same workload):

system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(100).map {Thread.new{500000.times{rand}}}.map(&:join) } }'
 13.900000   5.400000  19.300000 (  8.227000)
 12.900000   5.250000  18.150000 (  7.938000)
 13.800000   5.680000  19.480000 (  8.903000)
 13.540000   5.750000  19.290000 (  8.627000)
 13.390000   5.560000  18.950000 (  8.331000)
684ce37
@donv donv * JRUBY-6785 Allow ScriptingContainer#callMethod to be called on a Ja…
…va object

* Allow ScriptingContainer#callMethod to be called using nil as a receiver
a90bb11
@donv donv * JRUBY-6785 Make the receiver for callMethod more consistent
* Allow nil as a receiver
* Allow Java objects as receivers.  These will be wrapped before use.
7c8dba1
@headius headius Fix JRUBY-6731
Can't load class files in WARs compiled by warbler using jruby 1.7.0.preview1

When requiring a file with an explicit .class extension, assume
the .class filename is the only filename to be searched.

Because a require of .class almost always will indicate that the
user wants to load a specific precompiled Ruby file, there's no
good reason (I know of) to search for anything else.

The original logic did not work at all; require 'foo.class' would
search for 'foo.class.rb' and 'foo.class.class' (in that order)
but never 'foo.class'.

The claim is that this worked in 1.6.7.2, but I could not
reproduce that with a simple case. The logic involved does not
appear to have changed in a substantial way since well before
1.6.7.2, and in any case I believe the fix here is the right way
for it to work.
86ef07e
@headius headius Fix JRUBY-6395: Coerce Fixnum to Number
Simple enough fix: added Number (and Serializable) to the map of
types to converters in JavaUtil. Because our Numeric types were
already reporting that their "natural" classes were Number
subclasses, they were being selected for Number-receiving methods.
But the coercion logic did not know about Number, so it failed.
b8d0ca6
@headius headius Fix JRUBY-6514
java.lang.NullPointerException when using ScriptingCotainer from an Applet

Properties retrieved in a restricted environment may be null. We
must handle that case and not blindly try to getBytes().
8115b8c
@headius
Owner

Ok, time to broach the issue of the long delay in releases since 1.6.7, I guess.

We are not opposed to merging fixes into the 1.6 branch, but because we're busy with 1.7 release cycle now it's unlikely that we will have the cycles necessary to do a 1.6.8 release.

However...a few folks have proposed that we allow community-managed releases of earlier releases. We would not be responsible for them and would not make any guarantees about them, but if someone else is willing to go through the process of validating and building new releases, maybe it's worth exploring.

I'll talk to @enebo about it today.

@headius
Owner

Ok...so after talking with @enebo, we both agree it's worth doing a 1.6.8 release if we can get community help merging fixes over. This means you and others.

I'm going to merge this request in, and we'll happily accept more for as many fixes as re easily portable to 1.6.8. I would probably recommend avoiding fixes for 1.9 issues unless they're small and easy to port over, since master has changed substantially since 1.6.7 and many fixes will require much more complicated and error-prone merging. But fixes that are small or for 1.8.7 logic are open game, as are fixes to subsystems that have not changed significantly like Java integration.

@jrubyci jrubyci merged commit 8115b8c into jruby:jruby-1_6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 3, 2012
  1. @headius @dekellum

    Add rdoc to gems we install for dist, since hoe appears to depend on …

    headius authored dekellum committed
    …it now.
  2. @headius @dekellum

    Fix JRUBY-6489

    headius authored dekellum committed
    FileLock.release() throwing Java::JavaLang::NullPointerException
    
    Strategy works like this...we may want a more conservative change
    for jruby-1_6 branch:
    
    * Check if lock state would change as a result. If not, return success
    * Synchronize on FileChannel
    * Check again
    * Perform operation
    
    This should reduce the cost of repeated locks and unlocks to
    nearly zero, make concurrent locking/unlocking safe, and localize
    synchronization to only the code that needs it.
  3. @BanzaiMan @dekellum

    Fix JRUBY-6538: 'obj !~ thing' is not the same as obj.send(:!~, thing)

    BanzaiMan authored dekellum committed
    Credit goes to headius.
  4. @donv @dekellum

    * JRUBY-6560 Fix File.expand_path for jar:file:/ paths

    donv authored dekellum committed
  5. @BanzaiMan @dekellum

    Implement Kernel#Hash. Fixes JRUBY-6496.

    BanzaiMan authored dekellum committed
  6. @nahi @dekellum

    JRUBY-6559: GzipReader#rewind crashes when the input stream is serial…

    nahi authored dekellum committed
    …ly-multiplexed like RubyGems
    
    This commit adds regression spec.  The actual fix is at
    539f45c.
  7. @BanzaiMan @dekellum
  8. @headius @dekellum

    Fix JRUBY-5863: Named captures cause crash when there is no match

    headius authored dekellum committed
    Compiler calls match2AndUpdateScope[19], which did not pass through
    the match result and instead fell back to logic that used the
    scope's match and assumed it was non-nill.
  9. @headius @dekellum

    Fix JRUBY-6570: autoload :Time, 'time' doesn't work

    headius authored dekellum committed
    Instead of always using Object when calling autoload from an
    instance method, we use self's real class (skipping singletons)
    as the autoload location. This isn't exactly how MRI does it
    (they use cbase, which I think is euivalent to 1.8's ruby_class),
    but it should be correct for the majority of cases.
  10. @headius @dekellum

    Regression test for JRUBY-5863.

    headius authored dekellum committed
  11. @headius @dekellum

    Mitigation for JRUBY-6576

    headius authored dekellum committed
    Because in JRuby, concurrent selects can trigger blocking mode
    exceptions, I've modified IO.select logic to raise a concurrency
    error when multiple threads try to select on the same sockets at
    the same time (and fail to do so cleanly). At present, there's
    no way to do this since we use these channels in both blocking and
    nonblocking mode; we can't synchronize the blocking mode changes
    with multiple threads all trying to do select operations.
  12. @BanzaiMan @dekellum

    Add spec for JRUBY-6571

    BanzaiMan authored dekellum committed
  13. @BanzaiMan @dekellum
  14. @donv @dekellum

    * JRUBY-6624 Strip jar: prefix when searching for files in a JAR

    donv authored dekellum committed
    Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
  15. @donv @dekellum

    * JRUBY-6626 Ensure thread is set on finalize

    donv authored dekellum committed
  16. @headius @dekellum

    Regression spec for JRUBY-6634.

    headius authored dekellum committed
  17. @jabley @dekellum

    JRUBY-6611: Fix resource leak in LoadServiceResource

    jabley authored dekellum committed
  18. @bbrowning @dekellum

    Check if all throwables are Java exceptions in interpreted mode (JRUB…

    bbrowning authored dekellum committed
    …Y-6641)
    
    This fixes a simple typo in the RuntimeHelpers.isJavaExceptionHandled logic.
  19. @dekellum

    Remove -mimpure-text from CFLAGS. Fixes JRUBY-6633.

    Wayne Meissner authored dekellum committed
  20. @enebo @dekellum

    JRUBY-6464: Use "mvn versions:set -DnewVersion=<whatever>" instead of…

    enebo authored dekellum committed
    … our Ruby rake code for updating poms
  21. @headius @dekellum

    Fix JRUBY-6642.

    headius authored dekellum committed
    RubyException.printBacktrace will now print all lines by default,
    and has a separate form for skipping some number of lines. All
    internal uses that expected the first line to be skipped use the
    second form.
  22. @headius @dekellum

    Fix JRUBY-6517

    headius authored dekellum committed
    This seems to be a flaw in the patch I provided for
    http://bugs.ruby-lang.org/issues/5698. Since that patch was only
    to improve perf of REXML in Ruby 1.8 mode, I think it's better to
    have slower and working than faster and broken. Reverting.
  23. @headius @dekellum

    Fix JRUBY-6248

    headius authored dekellum committed
    I'm committing a cardinal sin here by using Thread#stop, but I
    think in this case it's warranted (and probably safe enough). The
    threads in question are doing nothing but spinning on two IO
    streams, and even if they leave those streams unusable we're
    walking away from them anyway.
    
    This fix makes thread count stay stable for a tight backquote loop
    as described in the bug.
    
    I did try using our Executor, but the problem isn't reusing
    threads, it's terminating them completely.
  24. @enebo @dekellum
  25. @BanzaiMan @dekellum

    Remove Enumerable#join. This resolves JRUBY-6703.

    BanzaiMan authored dekellum committed
  26. @alindeman @dekellum

    Fixes JRUBY-6677

    alindeman authored dekellum committed
    * Passes the `mode` argument correctly from Kernel#open to IO#popen, so
      if a non-default mode is specified (e.g., "w"), it is honored
    
    Signed-off-by: Hiro Asari <asari.ruby@gmail.com>
  27. @headius @dekellum

    Fix JRUBY-6706

    headius authored dekellum committed
    Collections.shuffle works incorreclty on Ruby arrays
    
    We were not implementing List.set properly; it should return the
    previous value at that index, and we were just returning the
    given value.
  28. @enebo @dekellum

    Errant printf removed

    enebo authored dekellum committed
  29. @yokolet @dekellum

    Fix for JRUBY-6728. This fixes Nokogiri's 1.9 test error of PrettyPri…

    yokolet authored dekellum committed
    …nt as well as sparklemotion/nokogiri#657.
    
    The change didn't break test:mri19. However, oddly, test_condvar_wait_exception_handling of test/externals/ruby1.9/ruby/test_thread.rb has become failing often  on my macbook. But, not always. This change might have revealed threadding issue?
  30. @headius @dekellum

    Fix JRUBY-6745

    headius authored dekellum committed
    ClassCastException calling Fixnum#== with a Java Integer
    
    The problem here was that when given multiple overloads that are
    all dispatchable with a given set of arguments, we just picked the
    first one. This specific case had both compareTo(Object) and
    compareTo(Integer), and we picked the Object version. Because the
    coercion from Fixnum to Object produces a Long, we eventually hit
    a ClassCastException when the Object version tried to call the
    Integer version. By fixing Java method dispatch to narrow
    overloads to the most specific version, we now will call the
    Integer version and coerce properly.
    
    I believe this was working in 1.6.7 because we had other bugs
    relating to how we coerce versus how we dispatch that caused us
    to coerce to Integer but still call the Object version.
  31. @headius @dekellum

    Fix for JRUBY-6725. See jruby/ruby repo for description.

    headius authored dekellum committed
  32. @headius @dekellum

    Fix JRUBY-6724

    headius authored dekellum committed
    NO_FILE_CLASS Profile does disables File but not FileStats that depends on File
    
    Fix NO_FILE_CLASS profile to also disable FileStat.
  33. @headius @dekellum

    Fix JRUBY-6722

    headius authored dekellum committed
    Can't instantiate ThreadGroup subclass with arguments
    
    We had defined 'new' on ThreadGroup, when what we needed was an
    ObjectAllocator for the normal 'new' logic to work. With that fix,
    subclass initialize methods work fine.
  34. @headius @dekellum

    Fix JRUBY-6753

    headius authored dekellum committed
    class_eval should inherit __name__ from the caller
    
    class_eval/module_eval were still being framed, which caused them
    to interfere with getting the actual caller name. Removed the
    framing and the bug is fixed.
  35. @headius @dekellum

    Fix JRUBY-6775: RubyRandom is not thread-safe

    headius authored dekellum committed
    I opted to simply synchronize around the logic in question. The
    primary issue is the potential for "left" to get decremented below
    zero under concurrency while it is in use. I tried to figure out
    an AtomicInteger-based algorithm for guaranteeing both that left
    would never cause a <0 array dereference *and* would not produce
    the same random number twice, but was unable to find a way due
    to the multiple state changes that happen.
    
    Synchronization does slow things down a bit, but perhaps not
    enough to worry about:
    
    Before (single thread):
    
    system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
      3.670000   0.130000   3.800000 (  3.317000)
      3.880000   0.150000   4.030000 (  3.761000)
      4.550000   0.150000   4.700000 (  4.495000)
      4.500000   0.160000   4.660000 (  4.460000)
      4.500000   0.150000   4.650000 (  4.452000)
    
    After (single thread):
    
    system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(1).map {Thread.new{50000000.times{rand}}}.map(&:join) } }'
      4.420000   0.130000   4.550000 (  4.031000)
      4.460000   0.170000   4.630000 (  4.353000)
      5.000000   0.160000   5.160000 (  4.954000)
      4.980000   0.160000   5.140000 (  4.924000)
      5.070000   0.160000   5.230000 (  5.020000)
    
    After (100 threads, same workload):
    
    system ~/projects/jruby $ jruby -rbenchmark -e '5.times { puts Benchmark.measure { Array.new(100).map {Thread.new{500000.times{rand}}}.map(&:join) } }'
     13.900000   5.400000  19.300000 (  8.227000)
     12.900000   5.250000  18.150000 (  7.938000)
     13.800000   5.680000  19.480000 (  8.903000)
     13.540000   5.750000  19.290000 (  8.627000)
     13.390000   5.560000  18.950000 (  8.331000)
  36. @donv @dekellum

    * JRUBY-6785 Allow ScriptingContainer#callMethod to be called on a Ja…

    donv authored dekellum committed
    …va object
    
    * Allow ScriptingContainer#callMethod to be called using nil as a receiver
  37. @donv @dekellum

    * JRUBY-6785 Make the receiver for callMethod more consistent

    donv authored dekellum committed
    * Allow nil as a receiver
    * Allow Java objects as receivers.  These will be wrapped before use.
  38. @headius @dekellum

    Fix JRUBY-6731

    headius authored dekellum committed
    Can't load class files in WARs compiled by warbler using jruby 1.7.0.preview1
    
    When requiring a file with an explicit .class extension, assume
    the .class filename is the only filename to be searched.
    
    Because a require of .class almost always will indicate that the
    user wants to load a specific precompiled Ruby file, there's no
    good reason (I know of) to search for anything else.
    
    The original logic did not work at all; require 'foo.class' would
    search for 'foo.class.rb' and 'foo.class.class' (in that order)
    but never 'foo.class'.
    
    The claim is that this worked in 1.6.7.2, but I could not
    reproduce that with a simple case. The logic involved does not
    appear to have changed in a substantial way since well before
    1.6.7.2, and in any case I believe the fix here is the right way
    for it to work.
  39. @headius @dekellum

    Fix JRUBY-6395: Coerce Fixnum to Number

    headius authored dekellum committed
    Simple enough fix: added Number (and Serializable) to the map of
    types to converters in JavaUtil. Because our Numeric types were
    already reporting that their "natural" classes were Number
    subclasses, they were being selected for Number-receiving methods.
    But the coercion logic did not know about Number, so it failed.
  40. @headius @dekellum

    Fix JRUBY-6514

    headius authored dekellum committed
    java.lang.NullPointerException when using ScriptingCotainer from an Applet
    
    Properties retrieved in a restricted environment may be null. We
    must handle that case and not blindly try to getBytes().
Something went wrong with that request. Please try again.