Fix ClassCastException when using StringBuilder/Buffer #496

[shelches]

Fixes a ClassCastException in the flatten() method, if a StringBuilder or StringBuffer was used to initialize a JavaScript variable.

[shelches]

I should add that compilation was successful, one unrelated test failed (the same test fails before I made my changes).

[rbri]

Thanks for your conyribution, can you please also add a test case.

[p-bakker]

Note that this PR fixes issue #496. Also see discussion #1206

Note that while this PR fixes a ClassCastException when concatenating strings and StringBuilder/Buffers together in JavaScript,
if the StringBuilder/Buffer gets modified after the concatenation in Javascript, an ArrayIndexOutOfBounds exception will be thrown, see #1206 (reply in thread). @rbri / @gbrail Any thoughs on this?

And on a sidenote: when doing string concatenation inside JS with StringBuilder/Buffers instantiated inside JS as well, the optimized path that uses ConString isn't getting hit, see #1206 (comment)

[shelches]

test

[shelches]

@rbri I created some unit tests in testsrc/org/mozilla/javascript/tests/Issue1206Test.java for testing String, StringBuffer, and StringBuilder. Without the patch, the tests fail where they should, with a java.lang.ClassCastException. With the patch, they all succeed.

The new unit test file (Issue1206Test.java) is in the fix-classcastexception branch on the Mozilla Rhino fork of my GitHub account. Hope I handled this correctly! (It's my first time, but hopefully not nearly my last, contributing to open source!)

Please let me know if everything looks OK. Thanks!

[gbrail]

I think that it's good to have this flexibility, but the fact that the contract for this class is confusing in that it will crash if you change a StringBuilder after putting it into this Object is not a great thing. I think that we should fix that but think carefully about where this class is actually used. Looks like there are two fixes:

1. Call "toString" in the constructor, rather than in the "flatten" method. That way, the whole thing is immutable, which is consistent with how strings work in JavaScript in general.

2. Take out the premature optimization of setting "length" in the constructor, and calculate in "flatten" instead. That means that a StringBuffer/Builder could be modified after this object is created.

I'd suggest that 2. is not intuitive to a typical JavaScript programmer who expects strings to be immutable, so I'd suggest thinking about 1. instead.

[gbrail]

In fact, based on how this object is probably typically used, we could even less-prematurely optimize with something that tests if either "str1" or "str2" is NOT just a String or ConsString in the constructor and then just flatten it right there, presuming that "toString" on anything that's not already a String is going to have some cost anyway. At any rate this is a complex set of operations and it's worth thinking more carefully about.

[shelches]

I think that it's good to have this flexibility, but the fact that the contract for this class is confusing in that it will crash if you change a StringBuilder after putting it into this Object is not a great thing. I think that we should fix that but think carefully about where this class is actually used. Looks like there are two fixes:

1. Call "toString" in the constructor, rather than in the "flatten" method. That way, the whole thing is immutable, which is consistent with how strings work in JavaScript in general.
2. Take out the premature optimization of setting "length" in the constructor, and calculate in "flatten" instead. That means that a StringBuffer/Builder could be modified after this object is created.

I'd suggest that #2 is not intuitive to a typical JavaScript programmer who expects strings to be immutable, so I'd suggest thinking about #1 instead.

The issue we have at $work is that not using StringBuilder results in 2x more CPU use and 7x more garbage collection, which unfortunately is a "show stopper" for us, although we recognize this is our problem, not Rhino's problem. We are still in the process of evaluating work arounds (a temporary solution to give us some breathing room is to supply our own mildly customized Rhino jar, but that's not a good long-term solution).

[gbrail]

I'd be OK with taking out the optimization that ConsString has now of pre-computing the length. I think that would fix the crash that @p-bakker identified while still giving you the performance benefit. That's what I was getting to in #1210 (comment) above.

[shelches]

I'd be OK with taking out the optimization that ConsString has now of pre-computing the length. I think that would fix the crash that @p-bakker identified while still giving you the performance benefit. That's what I was getting to in #1210 (comment) above.

Just a quick update: I have been looking into removing the code that pre-computes the length. A quick, minimal test was to change the length() method to this:

    public int length() {
        return isFlat ? left.length() : left.length() + right.length();
    }

Alas, this causes test failures. For example, testAppendManyStrings() in ConsStringTest.java fails with a java.lang.StackOverflowError. (Several other tests fail with the same exception.)

I'm digging further... but I think the ConsString.java refactoring (from years back) may have left it fundamentally incompatible with supporting java.lang.StringBuffer and java.lang.StringBuilder. I think it only truly supports java.lang.String.

However, if I come up with an unexpected solution, I'll be sure to comment here. My guess, though, is we'll need to accept that java.lang.StringBuffer and java.lang.StringBuilder are no longer supported.

[p-bakker]

Tnx for your continued investigation into this.

As for your minimal test: @gbrail suggested to call .toString() on str1/str2 if they are not instances of String or ConString (see comment #1210 (comment))

When computing the length inside the .length() method, the returned length could change over time if either of the values is a StringBuilder that gets modified.

[shelches]

It sounds like the change that would result in the most intuitive behavior for a JavaScript developer would be:

    public ConsString(CharSequence str1, CharSequence str2) {
        if (!(str1 instanceof String) && !(str1 instanceof ConsString)) {
            str1 = str1.toString();
        }
        if (!(str2 instanceof String) && !(str2 instanceof ConsString)) {
            str2 = str2.toString();
        }
        left = str1;
        right = str2;
        length = left.length() + right.length();
        isFlat = false;
    }

That way, if someone passes in a StringBuffer, StringBuilder, or any other kind of mutable CharSequence, and mutates it in Java code later, there would be no failures.

This means the original patch could be reverted, because the (String) cast would always work.

How does this look? @gbrail @p-bakker (with much thanks for all the help!)

[p-bakker]

Yeah, I think that is along the lines that @gbrail suggested and imho the best approach as well, as it won't cause any exceptions and gives predictable behavior.

[shelches]

Yeah, I think that is along the lines that @gbrail suggested and imho the best approach as well, as it won't cause any exceptions and gives predictable behavior.

Thank you! I believe I have successfully pushed the correct changes to my fork, and I think the pull request is still open for everyone's consideration.

All tests passed on my local system as well, and the original test I created also passes (and is in my fork as well).

[p-bakker]

I think all the tests currently just check whether the code doesn't throw an exception

Think it would be better if besides that, they'd also assert the result of cx.evaluateString to be as expected

[shelches]

Great idea! Those tests have been updated to assert the results of those cx.evaluateString calls.

The fork/branch has been updated. I think this might be ready for review by the Rhino team! Thanks!

[p-bakker]

Looks like there are some formatting issues in your PR, as the build fails on Spotless

[shelches]

Looks like there are some formatting issues in your PR, as the build fails on Spotless

Thank you, I believe I have resolved the code formatting issue.

[p-bakker]

LGTM

[gbrail]

This ends up being a very simple fix, which I like. Does it resolve your performance issues? If so I'll merge it!

[shelches]

This ends up being a very simple fix, which I like. Does it resolve your performance issues? If so I'll merge it!

Yes, it resolves our performance issues as well. Thank you!

[gbrail]

Great -- thanks for fixing this!

[shelches]

Great -- thanks for fixing this!

Thank you very much, and thanks to everyone for the help and input!

Does the Mozilla Rhino project have a release date for the next version? Just wondering if, at work, I should deliver a custom jar (with this change), or perhaps wait until the next Rhino release (depending on the schedule). Thanks!

[p-bakker]

There's no tentative date for the next Rhino release, so i'd go with a custom jar