Flaky-test: org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest.allSchemasShouldSupportNull and allSchemasShouldRoundtripInput

[eolivelli]

java.lang.NullPointerException: NPE when using schema null : null
	at org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest.allSchemasShouldRoundtripInput(PrimitiveSchemaTest.java:132)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

output:

Starting test [TestClass name=class org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest].allSchemasShouldRoundtripInput([{null=[00:18:23.823175, 22:18:23.823181], org.apache.pulsar.client.impl.schema.ByteSchema@2b4829aa=[-1, 0], org.apache.pulsar.client.impl.schema.DateSchema@32d22650=[2021-06-23, 2021-06-23], org.apache.pulsar.client.impl.schema.LocalDateSchema@33a71d23=[2021-06-23, 2021-06-21], org.apache.pulsar.client.impl.schema.BytesSchema@2fa879ed=[[B@9880a15], org.apache.pulsar.client.impl.schema.ShortSchema@794d28a3=[32767, -32768], org.apache.pulsar.client.impl.schema.TimestampSchema@4f6fd101=[2021-06-23 00:18:13.82, 2021-06-23 00:18:23.82], org.apache.pulsar.client.impl.schema.DoubleSchema@a99c42c=[5678567.12312, -5678567.12341], org.apache.pulsar.client.impl.schema.ByteBufferSchema@2b2b7e3c=[java.nio.HeapByteBuffer[pos=9 lim=10 cap=10]], org.apache.pulsar.client.impl.schema.StringSchema@aee05f4=[my string], org.apache.pulsar.client.impl.schema.TimeSchema@135064ea=[00:18:13, 00:18:23], org.apache.pulsar.client.impl.schema.InstantSchema@106459d9=[2021-06-23T00:18:23.820925Z, 2021-06-22T23:55:23.820928Z], org.apache.pulsar.client.impl.schema.FloatSchema@1a01ffff=[5678567.0, -5678567.0], org.apache.pulsar.client.impl.schema.BooleanSchema@2bcb1414=[false, true], org.apache.pulsar.client.impl.schema.IntSchema@a55e82a=[423412424, -41243432], org.apache.pulsar.client.impl.schema.LongSchema@70ee9d31=[922337203685477580, -922337203685477581], org.apache.pulsar.client.impl.schema.LocalDateTimeSchema@12166229=[2021-06-23T00:18:23.823191, 2021-06-21T00:18:23.823199, 2021-04-14T00:18:23.823209]}])-------
00:18:23.964 [TestNG-method=allSchemasShouldRoundtripInput-1:org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest@122] INFO org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest - Test schema null
00:18:23.964 [TestNG-method=allSchemasShouldRoundtripInput-1:org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest@124] INFO org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest - Encode : 00:18:23.823175

[eolivelli]

one key of the test data is null, there may be some weird issue with static initialization

[lhotari]

There's a similar flaky test issue that exists in master branch.

example: https://github.com/apache/pulsar/actions/runs/3759470154/jobs/6389256559#step:10:600

Error:  Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.439 s <<< FAILURE! - in org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest
  Error:  allSchemasShouldRoundtripInput(org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest)  Time elapsed: 0.102 s  <<< FAILURE!
  java.lang.RuntimeException: java.lang.AssertionError: expected object to not be null
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodNoCheckedException(MethodInvocationHelper.java:57)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:157)
  	at org.testng.internal.Parameters.handleParameters(Parameters.java:803)
  	at org.testng.internal.Parameters.handleParameters(Parameters.java:728)
  	at org.testng.internal.invokers.ParameterHandler.handleParameters(ParameterHandler.java:72)
  	at org.testng.internal.invokers.ParameterHandler.createParameters(ParameterHandler.java:51)
  	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:924)
  	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
  	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
  	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
  	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
  	at org.testng.TestRunner.privateRun(TestRunner.java:829)
  	at org.testng.TestRunner.run(TestRunner.java:[602](https://github.com/apache/pulsar/actions/runs/3759470154/jobs/6389256559#step:10:603))
  	at org.testng.SuiteRunner.runTest(SuiteRunner.java:437)
  	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431)
  	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391)
  	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
  	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
  	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
  	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
  	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
  	at org.testng.TestNG.runSuites(TestNG.java:1099)
  	at org.testng.TestNG.run(TestNG.java:1067)
  	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeLazy(TestNGDirectoryTestSuite.java:123)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:90)
  	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146)
  	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
  	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
  	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
  	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
  Caused by: java.lang.AssertionError: expected object to not be null
  	at org.testng.Assert.fail(Assert.java:110)
  	at org.testng.Assert.assertNotNull(Assert.java:1319)
  	at org.testng.Assert.assertNotNull(Assert.java:1303)
  	at org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest.schemas(PrimitiveSchemaTest.java:108)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodNoCheckedException(MethodInvocationHelper.java:53)
  	... 31 more
  
  Error:  allSchemasShouldSupportNull(org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest)  Time elapsed: 0.127 s  <<< FAILURE!
  java.lang.RuntimeException: java.lang.AssertionError: expected object to not be null
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodNoCheckedException(MethodInvocationHelper.java:57)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:157)
  	at org.testng.internal.Parameters.handleParameters(Parameters.java:803)
  	at org.testng.internal.Parameters.handleParameters(Parameters.java:728)
  	at org.testng.internal.invokers.ParameterHandler.handleParameters(ParameterHandler.java:72)
  	at org.testng.internal.invokers.ParameterHandler.createParameters(ParameterHandler.java:51)
  	at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:924)
  	at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194)
  	at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148)
  	at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
  	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
  	at org.testng.TestRunner.privateRun(TestRunner.java:829)
  	at org.testng.TestRunner.run(TestRunner.java:602)
  	at org.testng.SuiteRunner.runTest(SuiteRunner.java:437)
  	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:431)
  	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:391)
  	at org.testng.SuiteRunner.run(SuiteRunner.java:330)
  	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
  	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
  	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256)
  	at org.testng.TestNG.runSuitesLocally(TestNG.java:1176)
  	at org.testng.TestNG.runSuites(TestNG.java:1099)
  	at org.testng.TestNG.run(TestNG.java:1067)
  	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeLazy(TestNGDirectoryTestSuite.java:123)
  	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:90)
  	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146)
  	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
  	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
  	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
  	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
  Caused by: java.lang.AssertionError: expected object to not be null
  	at org.testng.Assert.fail(Assert.java:110)
  	at org.testng.Assert.assertNotNull(Assert.java:1319)
  	at org.testng.Assert.assertNotNull(Assert.java:1303)
  	at org.apache.pulsar.client.impl.schema.PrimitiveSchemaTest.schemas(PrimitiveSchemaTest.java:108)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
  	at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodNoCheckedException(MethodInvocationHelper.java:53)
  	... 31 more