Implement BaseConnectorTest in Pinot connector

[ebyhr]

1. AbstractPinotIntegrationSmokeTest should extends BaseConnectorSmokeTest
2. Add a new TestPinotConnectorTest extending BaseConnectorTest

[ntantri]

Hi, @ebyhr is this pertaining to just extending the BaseConnectorTest in https://github.com/trinodb/trino/blob/master/plugin/trino-pinot/src/test/java/io/trino/plugin/pinot/AbstractPinotIntegrationSmokeTest.java#L96 ?

Can you provide more description on the actual expectation of this?

[ebyhr]

@tan31989 Updated the description.

[ntantri]

Hi @ebyhr, I was trying to extend BaseConnectorSmokeTest for AbstractPinotIntegrationSmokeTest, but then realised that the SQL assertions in BaseConnectorSmokeTest are designed for a normal SQL queries, whereas, AbstractPinotIntegrationSmokeTest has tables/SQL's that require JSON specific queries. For instance, pinot specifically has Kafka that creates records in JSON and is expected to be fetched with JSON queries.

Is the intent to remove all those queries and freshly create new tables/data that match queries in BaseConnectorSmokeTest?

[ebyhr]

create new tables/data that match queries in BaseConnectorSmokeTest?

Correct, but please don't remove existing tests.

[ntantri]

@ebyhr have created one PR for (adding you as a reviewer), please do review it.

AbstractPinotIntegrationSmokeTest should extends BaseConnectorSmokeTest

Working on the second one as a separate PR to make sure it's easy to develop.

[ntantri]

@ebyhr , I have been scratching my head around a problem to write tests for TestPinotConnectorTest for a while now.

Need some assistance before I proceed further.

• BaseConnectorTest has a table orders, that has a column for date
• Pinot support date transformation using some functions, mentioned here and here

But,

• The schema builder used to ingest data is org.apache.avro.SchemaBuilder, example, see this SchemaBuilder usage
• This SchemaBuilder doesn't support a TimeStamp type that Pinot supports. Not sure how to make the tests for orders pass in this case.

Any input on what is acceptable in this case?

My thoughts, there are two ways, either:

• Figure out some other way to ingest data that allows timestamp column (might require me to change kafka.sendMessages to accept something other than Avro

or

• Ignore all tests for orders.

[ebyhr]

@xiangfu0 @elonazoulay Can you help to setup orders table?

[ntantri]

@xiangfu0 @elonazoulay any update on this?