Skip to content

Commit

Permalink
GH-37045: [MATLAB] Implement featherwrite in terms of arrow.internal.…
Browse files Browse the repository at this point in the history
…io.feather.Writer (#37047)

### Rationale for this change

Now that #37043 is merged, we can re-implement `featherwrite` in terms of the new `arrow.internal.io.feather.Writer` class. Once this change is made, we can delete the legacy build infrastructure and featherwrite MEX code. 

### What changes are included in this PR?

1. Re-implemented `featherwrite` using `arrow.internal.io.feather.Writer`. 

### Are these changes tested?

1. Yes, the existing tests in `tfeather.m` cover these changes.
2. I had to update some of the expected error message IDs in `tfeather.m` because the new implementation throws errors with different IDs. 
3. `featherwrite` used to export the real part of MATLAB complex numeric arrays. The new version of `featherwrite` now errors if the input table contains complex data because feather/Arrow itself does not support complex numeric data. We think this is the right decision. Writing out only the real part is lossy.

### Are there any user-facing changes?

Yes, `featherwrite` no longer supports writing complex numeric arrays.

### Future Directions

1. Once this PR is merged, we will remove the legacy build infrastructure and MEX code. 
* Closes: #37045

Authored-by: Sarah Gilmore <sgilmore@mathworks.com>
Signed-off-by: Kevin Gurney <kgurney@mathworks.com>
  • Loading branch information
sgilmore10 committed Aug 7, 2023
1 parent f549bf5 commit e90c316
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 22 deletions.
22 changes: 6 additions & 16 deletions matlab/src/matlab/featherwrite.m
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,11 @@ function featherwrite(filename, t)
% specific language governing permissions and limitations
% under the License.

import arrow.util.table2mlarrow;
arguments
filename(1, 1) string {mustBeNonmissing, mustBeNonzeroLengthText}
t table
end

% Validate input arguments.
narginchk(2, 2);
filename = convertStringsToChars(filename);
if ~ischar(filename)
error('MATLAB:arrow:InvalidFilenameDatatype', ...
'Filename must be a character vector or string scalar.');
end
if ~istable(t)
error('MATLAB:arrow:InvalidInputTable', 't must be a table.');
end

[variables, metadata] = table2mlarrow(t);

% Write the table to a Feather file.
arrow.cpp.call('featherwrite', filename, variables, metadata);
writer = arrow.internal.io.feather.Writer(filename);
writer.write(t);
end
11 changes: 5 additions & 6 deletions matlab/test/tfeather.m
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ function ErrorIfInvalidFilenameDatatype(testCase)

t = createTable;

testCase.verifyError(@() featherwrite({filename}, t), 'MATLAB:arrow:InvalidFilenameDatatype');
testCase.verifyError(@() featherwrite({table}, t), 'MATLAB:validation:UnableToConvert');
testCase.verifyError(@() featherread({filename}), 'MATLAB:arrow:InvalidFilenameDatatype');
end

Expand All @@ -178,7 +178,7 @@ function ErrorIfTooManyInputs(testCase)
end

function ErrorIfTooFewInputs(testCase)
testCase.verifyError(@() featherwrite(), 'MATLAB:narginchk:notEnoughInputs');
testCase.verifyError(@() featherwrite(), 'MATLAB:minrhs');
testCase.verifyError(@() featherread(), 'MATLAB:narginchk:notEnoughInputs');
end

Expand All @@ -193,7 +193,7 @@ function ErrorIfMultiColVarExist(testCase)

t = table(age, smoker, height, weight, bloodPressure);

testCase.verifyError(@() featherwrite(filename, t), 'MATLAB:arrow:UnsupportedVariableType');
testCase.verifyError(@() featherwrite(filename, t), 'arrow:array:InvalidShape');
end

function UnsupportedMATLABDatatypes(testCase)
Expand All @@ -205,7 +205,7 @@ function UnsupportedMATLABDatatypes(testCase)
calendarDuration(5, 3, 2)];
actualTable = addvars(actualTable, calendarDurationVariable);

testCase.verifyError(@() featherwrite(filename, actualTable) ,'MATLAB:arrow:UnsupportedVariableType');
testCase.verifyError(@() featherwrite(filename, actualTable) ,'arrow:array:UnsupportedMATLABType');
end

function NumericComplexUnsupported(testCase)
Expand All @@ -216,8 +216,7 @@ function NumericComplexUnsupported(testCase)
actualTable.double(2) = exp(9) + 5i;
actualTable.int64(2) = 1.0418e+03;

expectedTable = featherRoundTrip(filename, actualTable);
testCase.verifyNotEqual(actualTable, expectedTable);
testCase.verifyError(@() featherwrite(filename, actualTable) ,'arrow:array:ComplexNumeric');
end
end
end

0 comments on commit e90c316

Please sign in to comment.