Skip to content

Commit

Permalink
apacheGH-37187: [MATLAB] Re-implement tfeathermex.m tests in terms …
Browse files Browse the repository at this point in the history
…of new internal Feather Reader and Writer objects (apache#37189)

### Rationale for this change

Now that we have new internal Feather V1 `arrow.internal.io.feather.Reader` and `arrow.internal.io.feather.Writer` objects, we can re-implement the `tfeathermex.m` tests in terms of calls to these new internal APIs. As part of this change, we can also move these test cases into `matlab/test/arrow/io/feather/tRoundTrip.m` and delete the old `tfeathermex.m` file. Deleting the old Feather MEX tests will allow us to also delete the old Feather MEX code.

### What changes are included in this PR?

1. Moved test cases `NumericDatatypesNulls` and `InvalidMATLABTableVariableNames` from `test/tfeathermex.m` to `test/arrow/internal/io/feather/tRoundTrip.m`
2. Deleted `test/tfeathermex.m`
3. Deleted obsolete test utility `test/util/createVariablesAndMetadataStructs.m`
4. Deleted obsolete test utility `test/util/featherMEXRoundTrip.m`

### Are these changes tested?

Yes. All tests pass.

### Are there any user-facing changes?

No.

### Future Directions

1. Delete the MEX source code from the interface.
2. Move `test/util/createTable.m` and `test/util/featherRoundTrip.m` into package functions underneath `arrow.internal.test.*` (See apache#37188)
* Closes: apache#37187

Authored-by: Sarah Gilmore <sgilmore@mathworks.com>
Signed-off-by: Kevin Gurney <kgurney@mathworks.com>
  • Loading branch information
sgilmore10 authored and loicalleyne committed Nov 13, 2023
1 parent 7df8da5 commit 25a0fee
Show file tree
Hide file tree
Showing 4 changed files with 65 additions and 187 deletions.
66 changes: 65 additions & 1 deletion matlab/test/arrow/io/feather/tRoundTrip.m
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,70 @@ function Basic(testCase)

testCase.verifyEqual(tableWrite, tableRead);
end
end

function NumericDatatypesNulls(testCase)
% Verify integers with null values are roundtripped correctly.
import matlab.unittest.fixtures.TemporaryFolderFixture
import arrow.internal.io.feather.*
import arrow.tabular.RecordBatch

fixture = testCase.applyFixture(TemporaryFolderFixture);
filename = fullfile(fixture.Folder, "temp.feather");

uint8Array = arrow.array(uint8([1, 2, 3, 4, 5]), Valid=[1, 3, 4]);
int16Array = arrow.array(uint8([1, 2, 3, 4, 5]), Valid=[3, 5]);
inputRecordBatch = RecordBatch.fromArrays(uint8Array, int16Array);

writer = Writer(filename);
writer.write(inputRecordBatch);

reader = Reader(filename);
outputRecordBatch = reader.read();

testCase.verifyEqual(outputRecordBatch.NumColumns, inputRecordBatch.NumColumns);
testCase.verifyEqual(outputRecordBatch.ColumnNames, inputRecordBatch.ColumnNames);

outColumn1 = outputRecordBatch.column(1);
testCase.verifyEqual(outColumn1.Valid, uint8Array.Valid);
testCase.verifyEqual(toMATLAB(outColumn1), toMATLAB(uint8Array));

outColumn2 = outputRecordBatch.column(2);
testCase.verifyEqual(outColumn2.Valid, int16Array.Valid);
testCase.verifyEqual(toMATLAB(outColumn2), toMATLAB(int16Array));
end

function InvalidMATLABTableVariableNames(testCase)
import matlab.unittest.fixtures.TemporaryFolderFixture
import arrow.internal.io.feather.*
import arrow.tabular.RecordBatch

fixture = testCase.applyFixture(TemporaryFolderFixture);
filename = fullfile(fixture.Folder, "temp.feather");

doubleArray = arrow.array([1, 2, 3, 4, 5]);
stringArray = arrow.array(["A", "B", "C", "D", "E"]);

% Create a RecordBatch whose first column name is an invalid
% variable name for a MATLAB table.
inputRecordBatch = RecordBatch.fromArrays(doubleArray, stringArray, ...
ColumnNames=[":", "Valid"]);

writer = Writer(filename);
writer.write(inputRecordBatch);

% Verify the RecordBatch returned by read has the same
% ColumnNames as the original RecordBatch.
reader = Reader(filename);
outputRecordBatch = reader.read();
testCase.verifyEqual(outputRecordBatch.ColumnNames, inputRecordBatch.ColumnNames);

% Verify the VariableNames property in the table returned by
% featherread has been modified and that
% VariableDescriptions is set property.
outputTable = featherread(filename);
testCase.verifyEqual(outputTable.Properties.VariableNames, {':_1', 'Valid'});
testCase.verifyEqual(outputTable.Properties.VariableDescriptions, ...
{'Original variable name: '':''', ''});
end
end
end
67 changes: 0 additions & 67 deletions matlab/test/tfeathermex.m

This file was deleted.

97 changes: 0 additions & 97 deletions matlab/test/util/createVariablesAndMetadataStructs.m

This file was deleted.

22 changes: 0 additions & 22 deletions matlab/test/util/featherMEXRoundTrip.m

This file was deleted.

0 comments on commit 25a0fee

Please sign in to comment.