Skip to content
Permalink
Browse files
DRILL-7350: Move RowSet related classes from test folder
  • Loading branch information
vvysotskyi committed Aug 19, 2019
1 parent 4f4e1af commit 9c62bf1a91f611bdefa6f3a99e9dfbdf9b622413
Show file tree
Hide file tree
Showing 217 changed files with 1,045 additions and 998 deletions.
@@ -27,8 +27,8 @@
import org.apache.drill.exec.store.dfs.FileSystemPlugin;
import org.apache.drill.test.ClusterTest;
import org.apache.drill.test.BaseDirTestWatcher;
import org.apache.drill.test.rowSet.RowSet;
import org.apache.drill.test.rowSet.RowSetBuilder;
import org.apache.drill.exec.physical.rowSet.RowSet;
import org.apache.drill.exec.physical.rowSet.RowSetBuilder;
import org.apache.drill.test.ClusterFixture;
import org.apache.drill.test.rowSet.RowSetComparison;
import org.apache.drill.exec.record.metadata.SchemaBuilder;
@@ -13,13 +13,13 @@ of the [RowSetComparison](../../exec/java-exec/src/test/java/org/apache/drill/te

## Creating A [BatchSchema](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/BatchSchema.java)

The [SchemaBuilder](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/schema/SchemaBuilder.java) class can be used
The [SchemaBuilder](../../exec/vector/src/main/java/org/apache/drill/exec/record/metadata/SchemaBuilder.java) class can be used
to create an instance [BatchSchema](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/BatchSchema.java). An example
of how to to this can be found the **secondTest()** method of [ExampleTest](../../exec/java-exec/src/test/java/org/apache/drill/test/ExampleTest.java).

**Note:** The [BatchSchema](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/BatchSchema.java) class has limited complex type support. When
possible use [TupleMetadata](../../exec/vector/src/main/java/org/apache/drill/exec/record/metadata/TupleMetadata.java) and
[TupleSchema](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/metadata/TupleSchema.java) instead.
[TupleSchema](../../exec/vector/src/main/java/org/apache/drill/exec/record/metadata/TupleSchema.java) instead.

## Creating [TupleMetadata](../../exec/vector/src/main/java/org/apache/drill/exec/record/metadata/TupleMetadata.java)

@@ -33,21 +33,21 @@ TupleMetadata schema = new SchemaBuilder()
## Creating Test [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java)

[VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java)s populated with data can be created with the
[RowSetBuilder](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetBuilder.java). In order to use it do the following:
[RowSetBuilder](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSetBuilder.java). In order to use it do the following:

1. Create an allocator
```
BufferAllocator allocator = operatorFixture.allocator();
```
1. Create the desired BatchSchema using the [SchemaBuilder](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/schema/SchemaBuilder.java).
1. Create the desired BatchSchema using the [SchemaBuilder](../../exec/vector/src/main/java/org/apache/drill/exec/record/metadata/SchemaBuilder.java).
```
TupleMetadata schema = new SchemaBuilder()
.add(...)
.add(...)
.buildSchema();
```
1. Create a [RowSetBuilder](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetBuilder.java) and add
records to it. Then build a [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java).
1. Create a [RowSetBuilder](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSetBuilder.java) and add
records to it. Then build a [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java).
```
RowSet rowSet = new RowSetBuilder(allocator, schema)
.addRow(110, "green", new floatArray(5.5f, 2.3f), strArray("1a", "1b"))
@@ -56,33 +56,34 @@ TupleMetadata schema = new SchemaBuilder()
.build();
```
1. Retrieve the [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) wrapped by the
[RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java).
[RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java).
```
VectorContainer container = rowSet.container();
```

## Creating A Mock Record Batch (Upstream Operator) With Data

Create a [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java) and then create a
[RowSetBatch](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetBatch)
Create a [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java) and then create a
[RowSetBatch](../../exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/MockRecordBatch.java)

```
RecordBatch batch = new RowSetBatch(rowSet);
MockRecordBatch.Builder rowSetBatchBuilder = new MockRecordBatch.Builder()
.sendData(rowSet)
```

## Comparison Of Results

### Compare Two [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java)s
### Compare Two [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java)s

Use [RowSetUtilities](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSetUtilities.java).

```
RowSetUtilities.verify(expectedRowSet, actualRowSet)
```

### Compare A [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) To A [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java)
### Compare A [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) To A [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java)

You can convert a [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) into a [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java)
You can convert a [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) into a [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java)
a few ways:

* If you are using an [OperatorFixture](OperatorFixture.md) the best way to do this is with:
@@ -104,9 +105,9 @@ a few ways:

After the [VectorContainer](../../exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java) is wrapped in a [RowSet](../.
./exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java) you can compare the two
[RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java)s as usual.
[RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java)s as usual.

## End To End Example

A good example of building a [RowSet](../../exec/java-exec/src/test/java/org/apache/drill/test/rowSet/RowSet.java) and comparing results can be found in the
**testInitialSchema()** test in [TestResultSetLoaderProtocol](../../exec/java-exec/src/test/java/org/apache/drill/exec/physical/rowSet/impl/TestResultSetLoaderProtocol.java).
A good example of building a [RowSet](../../exec/java-exec/src/main/java/org/apache/drill/exec/physical/rowSet/RowSet.java) and comparing results can be found in the
**testInitialSchema()** test in [TestResultSetLoaderProtocol](../../exec/java-exec/src/test/java/org/apache/drill/exec/physical/resultSet/impl/TestResultSetLoaderProtocol.java).
@@ -23,7 +23,7 @@
import org.apache.drill.exec.physical.impl.scan.project.ResolvedTuple;
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection.ScanProjectionParser;
import org.apache.drill.exec.physical.impl.scan.project.ScanSchemaOrchestrator;
import org.apache.drill.exec.physical.rowSet.ResultSetLoader;
import org.apache.drill.exec.physical.resultSet.ResultSetLoader;
import org.apache.drill.exec.record.MaterializedField;
import org.apache.drill.exec.record.metadata.TupleMetadata;
import org.apache.drill.shaded.guava.com.google.common.annotations.VisibleForTesting;
@@ -22,8 +22,8 @@
import org.apache.drill.exec.physical.impl.scan.project.ColumnProjection;
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection;
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection.ScanProjectionParser;
import org.apache.drill.exec.physical.rowSet.project.RequestedColumnImpl;
import org.apache.drill.exec.physical.rowSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.physical.resultSet.project.RequestedColumnImpl;
import org.apache.drill.exec.physical.resultSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.store.easy.text.reader.TextReader;
import org.apache.drill.shaded.guava.com.google.common.annotations.VisibleForTesting;

@@ -19,7 +19,7 @@

import org.apache.drill.exec.physical.impl.scan.project.ResolvedTableColumn;
import org.apache.drill.exec.physical.impl.scan.project.VectorSource;
import org.apache.drill.exec.physical.rowSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.physical.resultSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.record.MaterializedField;

public class ResolvedColumnsArrayColumn extends ResolvedTableColumn {
@@ -18,7 +18,7 @@
package org.apache.drill.exec.physical.impl.scan.columns;

import org.apache.drill.exec.physical.impl.scan.project.AbstractUnresolvedColumn;
import org.apache.drill.exec.physical.rowSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.physical.resultSet.project.RequestedTuple.RequestedColumn;

public class UnresolvedColumnsArrayColumn extends AbstractUnresolvedColumn {

@@ -25,7 +25,7 @@
import org.apache.drill.exec.physical.impl.scan.project.ColumnProjection;
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection;
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection.ScanProjectionParser;
import org.apache.drill.exec.physical.rowSet.project.RequestedTuple.RequestedColumn;
import org.apache.drill.exec.physical.resultSet.project.RequestedTuple.RequestedColumn;

/**
* Parses the implicit file metadata columns out of a project list,
@@ -31,7 +31,7 @@
import org.apache.drill.exec.physical.impl.scan.project.ScanLevelProjection.ScanProjectionParser;
import org.apache.drill.exec.physical.impl.scan.project.ReaderLevelProjection.ReaderProjectionResolver;
import org.apache.drill.exec.physical.impl.scan.project.VectorSource;
import org.apache.drill.exec.physical.rowSet.ResultVectorCache;
import org.apache.drill.exec.physical.resultSet.ResultVectorCache;
import org.apache.drill.exec.record.VectorContainer;
import org.apache.drill.exec.record.metadata.TupleMetadata;
import org.apache.drill.exec.server.options.OptionSet;
@@ -19,8 +19,8 @@

import org.apache.drill.common.exceptions.CustomErrorContext;
import org.apache.drill.exec.ops.OperatorContext;
import org.apache.drill.exec.physical.rowSet.ResultSetLoader;
import org.apache.drill.exec.physical.rowSet.RowSetLoader;
import org.apache.drill.exec.physical.resultSet.ResultSetLoader;
import org.apache.drill.exec.physical.resultSet.RowSetLoader;
import org.apache.drill.exec.record.metadata.TupleMetadata;

/**
@@ -19,7 +19,7 @@