You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.
It is unclear to me why we would special case JSON
It complicates making JSON an optional feature
Describe the solution you'd like
Rather than maintaining parallel logic to interpret serde_json::Value as arrow arrays, I wonder if we can just remove the JsonEqual functionality and encourage conversion to arrow arrays. I'm unclear of a use-case that would require highly-performant comparison of arrow and JSON data
Describe alternatives you've considered
The text was updated successfully, but these errors were encountered:
At second glance, JsonEqual functionality is used by ArrowJson to compare with another RecordBatchReader. Underlying JSON record batch ArrowJsonBatch is compared with RecordBatch.
Although the stuffs like ArrowJson is under integration_util, and it is indeed used by integration test. But it seems to be only used as intermediate format to read JSON data and convert to arrow array. I don't see that JSON is compared directly with arrow arrays (I may miss it if it is deep).
Currently seems the JsonEqual functionality is only used in some IPC tests where JSON files are read and compare with arrow arrays without converting.
For that tests, I think we can follow integration test to convert JSON to arrow arrays before comparing.