[pulsar-sql] Handle schema not found

[codelipenghui]

Motivation

Currently, pulsar does not store byte[] schema in schema registry, while use send message with byte[], use pulsar sql will cause

Query 20190805_043001_00002_7yazb failed: Table has no columns: pulsar:PulsarTableHandle{connectorId=pulsar, schemaName=public/default, tableName=test, topicName=test}

Modifications

Handle schema not found exception and use default schema.

SchemaInfo.builder()
                .type(SchemaType.BYTES)
                .schema(new byte[0])
                .name(name)
                .build();

Verifying this change

Unit tests passed

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): (no)
• The public API: (no)
• The schema: (yes)
• The default values of configurations: (no)
• The wire protocol: (no)
• The rest endpoints: ( no)
• The admin cli options: (no)
• Anything that affects deployment: (/ no)

Documentation

• Does this pull request introduce a new feature? (no)

[sijie]

you can simplify with Schema.BYTES.getSchemaInfo(). This also avoids creating a schema info object again and again.

[codelipenghui]

fix it.

[jerrypeng]

"The value of the message with primitive schema type"

[jerrypeng]

Looks good. Minor spelling mistake. Possible to add any tests?

[codelipenghui]

@jerrypeng I have added a unit test for getSplit while topic have no schema, PLAT.

[codelipenghui]

run cpp tests

[codelipenghui]

run java8 tests

[sijie]

// org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?

run java8 tests

[mingfang]

I'm not sure it this is related but I'm seeing the "Table has no columns" error when running the Pulsar SQL tutorial here https://pulsar.apache.org/docs/en/sql-getting-started/.

Steps to recreate the problem
1-Generate some data by running this command for a minute and then kill it.
./bin/pulsar-admin sources localrun --name generator --destinationTopicName generator_test -a connectors/pulsar-io-data-generator-2.4.0.nar

2-Use Pulsar SQL to "consume" the data
select * from pulsar."public/default".generator_test;

3-Repeat steps 1 and 2. In step 2 I would see this error
Query 20190823_033331_00067_sw7tk failed: Table has no columns: pulsar:PulsarTableHandle{connectorId=pulsar, schemaName=public/default, tableName=generator_test, topicName=generator_test}

I can reproduce this error 100% of the time.

[sijie]

@mingfang

3-Repeat steps 1 and 2. In step 2 I would see this error

Do you mean you stop the generator and rerun it again? Or did you just rerun step2?

[mingfang]

You have to have some data into to see the error therefore you have to repeat step 1, run the generator for a minute and then kill it before running step 2 again.

My wild guess is that the schema got deleted and is not getting recreated by the generator because the topic already exist when running after the first time. If I change the topic name then it works again.