Assigning null to a BeanShell variable in a JSR223 sampler does not work as expected

[asfimport]

Bernhard (Bug 56816):
vars.get() returns void in a java script if the variable is not existing instead of null (as described in it's documentation http://jmeter.apache.org/api/org/apache/jmeter/threads/JMeterVariables.html#get%28java.lang.String%29 )

This will lead to confusion and bugs if the user doesn't know this.

Severity: major
OS: All

[asfimport]

Sebb (migrated from Bugzilla):
(In reply to Bernhard from comment 0)

vars.get() returns void in a java script if the variable is not existing
instead of null (as described in it's documentation

Are you sure?
I cannot find any references that says that javascript can return void.

http://jmeter.apache.org/api/org/apache/jmeter/threads/JMeterVariables.
html#get%28java.lang.String%29 )

That Javadoc is correct; Java returns null (and does not have a concept of void as a return type)

This will lead to confusion and bugs if the user doesn't know this.

Possibly, but it's not clear that there is a problem here.

Please provide sample code that exhibits the problem.

[asfimport]

Bernhard (migrated from Bugzilla):
Sorry for the confusion, I meant java, not javascript.
I'll add a test plan to make it clear what's wrong.

In the test plan there is a JSR223 with Java which tries to get a nonexistent variable out of vars. Then there are 2 ifs which compare the nonexistent variable with null and void and log and add "null" or "void" to the ResponseMessage.

Created attachment bugtest2Void.jmx: testplan with bug

bugtest2Void.jmx

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.6" jmeter="2.11 r1554548">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1407302950000</longProp>
        <longProp name="ThreadGroup.end_time">1407302950000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>
      <hashTree>
        <JSR223Sampler guiclass="TestBeanGUI" testclass="JSR223Sampler" testname="JSR223 Sampler" enabled="true">
          <stringProp name="cacheKey"></stringProp>
          <stringProp name="filename"></stringProp>
          <stringProp name="parameters"></stringProp>
          <stringProp name="script">Object o = vars.get(&quot;nonexistent&quot;);
String ResponseMessage=&quot;&quot;;
if(o==null)
{
	ResponseMessage+=&quot;null&quot;;
	log.info(&quot;null&quot;);
}
if(o==void)
{
	ResponseMessage+=&quot;void&quot;;
	log.info(&quot;void&quot;);
}
SampleResult.setResponseMessage(ResponseMessage);</stringProp>
          <stringProp name="scriptLanguage">java</stringProp>
        </JSR223Sampler>
        <hashTree/>
      </hashTree>
      <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
        <boolProp name="ResultCollector.error_logging">false</boolProp>
        <objProp>
          <name>saveConfig</name>
          <value class="SampleSaveConfiguration">
            <time>true</time>
            <latency>true</latency>
            <timestamp>true</timestamp>
            <success>true</success>
            <label>true</label>
            <code>true</code>
            <message>false</message>
            <threadName>true</threadName>
            <dataType>false</dataType>
            <encoding>false</encoding>
            <assertions>false</assertions>
            <subresults>false</subresults>
            <responseData>false</responseData>
            <samplerData>false</samplerData>
            <xml>false</xml>
            <fieldNames>true</fieldNames>
            <responseHeaders>false</responseHeaders>
            <requestHeaders>false</requestHeaders>
            <responseDataOnError>false</responseDataOnError>
            <saveAssertionResultsFailureMessage>false</saveAssertionResultsFailureMessage>
            <assertionsResultsToSave>0</assertionsResultsToSave>
            <bytes>true</bytes>
            <hostname>true</hostname>
            <threadCounts>true</threadCounts>
            <sampleCount>true</sampleCount>
          </value>
        </objProp>
        <stringProp name="filename"></stringProp>
      </ResultCollector>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

[asfimport]

Sebb (migrated from Bugzilla):
Thanks for the sample JMX.

The code is not Java either - it is BeanShell.
Java does not support void as a value, only as a return type qualifier.
Whereas BeanShell uses void to indicate a non-existent variable.

==

It's not yet clear why assigning null to the variable o should result in it being void rather than null as expected.

It looks to be a side effect of using the JSR223 sampler rather than the BeanShell sampler, as the same code works as expected in the BeanShell sampler, and also works as expected in a Beanshell interpreter or console app.

It remains to be seen whether this is a feature or bug in the JSR223 BeanShell support in JMeter or in BeanShell.

Meanwhile the workround is to use the BeanShell test elements to run BeanShell code.

[asfimport]

@ham1 (migrated from Bugzilla):
In the latest trunk 3.2-SNAPSHOT all of the following match x == void but do NOT match x == null in the JSR223 sampler:

var x;
var x = null;
Object x = null;
String x = null;

Not sure if it's a bug, but it looks like expected BeanShell behaviour looking at the BeanShell docs (http://www.beanshell.org/manual/bshmanual.html):

unset
void unset ( String name )
"Undefine" the variable specifed by 'name' (So that it tests == void).
Note: there will be a better way to do this in the future. This is currently equivalent to doing namespace.setVariable(name, null);

Maybe we should document this in our JSR223 sampler as this is different to the BeanShell sampler which seems to display the opposite behaviour i.e. all the above tests only match == null not == void - this looks more like a bug to me, but is perhaps less intuitive when mixed with Java code.

[bugla]

Just ran into that same issue in the current JMeter 5.5.

It is definitely a bug in the included BeanShell library bsh-2.0b6.jar.
Fortunately it is already fixed in the current BeanShell library bsh-2.1.1.jar. I successfully tested this by locally replacing the library in JMeter and rerunnning the reproduction tests (see below).

As this old bug ticket got a bit convoluted in the comments, I added a concise problem description and easy reproducible example below.

A little clear statement first (as some IMHO strange ideas have been thrown around earlier in the ticket):
If you add a JSR223 sampler and select "language" as "java (<engine_names_and_versions>)", you expect (and can expect) that it interprets the text written in the section "Script" below as being written in Java.
That's the whole purpose of that language entry and that kind of handling Scripts is the whole purpose of JSR223 (see: JSR223 Spec: https://jcp.org/aboutJava/communityprocess/final/jsr223/index.html) and JMeter's JSR223 sampler (it's named that way for a reason).
So yes: It's a bug and there's no doubt about it, it's in an external library, and it's already fixed there.

Issue:
The Java implementation behind "java (BeanShell 2.0b6 / BeanShell Engine 1.0)" is catastrophically wrong concerning the handling of "null" and "void" (with void being new in Java 11 bsh mixed something up, about its usage).

Simplest reproduction (a comparison between openjdk 11 and bsh-2.0b6 in JMeter) (with the same openjdk 11 in the background):

Code and result from Java:
public class VoidTest {
public static void main(String[] args) {
System.out.println("Java TEST java.version: " + System.getProperty("java.version")); // 11
String varX3 = null;
System.out.println("Java TEST varX3: " + varX3); // null (as it should be)
System.out.println("Java TEST varX3 is null: " + (varX3 == null)); // true (as it should be)
// System.out.println("Java TEST varX3 is void: " + (varX3 == void)); // compile fail: illegal start of expression (as it should be)
}
}

(Put it in a file "VoidTest.java", run "javac VoidTest.java", run "java VoidTest".)
--> That result is fully compliant with the Java Language Specification, i.e. by definition openjdk11's implementation of that snippet is correct.

Code and result from "java (BeanShell 2.0b6 / BeanShell Engine 1.0)" in JMeter 5.5:
log.warn("BeanShell TEST java.version: " + System.getProperty("java.version")); // 11
String varX2 = null;
log.warn("BeanShell TEST varX2: " + varX2); // void (WRONG!)
log.warn("BeanShell TEST varX2 is null: " + (varX2 == null)); // false (WRONG!)
log.warn("BeanShell TEST varX2 is void: " + (varX2 == void)); // true (WRONG!)

--> That result fails to comply with the Java Language Specification, i.e. bsh-2.0b6 is wrong. And that bug is very prolematic as it breaks a lot of correct Java code.

Fix:
Fortunately this has already been fixed in the current version of the BeanShell library bsh-2.1.1.jar (see https://github.com/beanshell/beanshell/releases/tag/2.1.1), which I tested (same test as above, but with lib/bsh-2.0b6.jar replaced with lib/bsh-2.1.1.jar in JMeter5.5 gives the correct result, so no JMeter code needs to be changed, just an external library updated.

While trying to get a PR together, I found that unfortunately bsh-2.1.1.jar is not already published to Maven Central (https://mvnrepository.com/artifact/org.apache-extras.beanshell/bsh), so integrating it in JMeter without that would need some more effort as JMeter seems to only use dependencies from mvnrepository so far.

I did open a ticket in the bsh repository for publishing it in mvnrepository (beanshell/beanshell#739). Probably give this a thumbs-up, if you can, just in case it speeds up things.

Options:

• We wait for beanshell to publish bsh-2.1.1.jar on Maven Central (mvnrepository.com) and someone does a simple library replacement PR (I would have done it, but see above).
• A JMeter developer adds a way to retrieve the library directly from https://github.com/beanshell/beanshell .

Workaround:
Replace lib/bsh-2.0b6.jar with lib/bsh-2.1.1.jar in your installation of JMeter5.5.