Partition tools don't correctly interact with LocalDate type #2302
Description
Description
LocalDate instances can be used by PartitionedTables, or partitioned Tables (iceberg, parquet, etc), and may be displayed in the UI. The user will then expect to select a partition to display only data from that partition. However, the UI seems to be using dh.FilterValue.ofNumber(...) to try to create partition values
PartitionedTable
Steps to reproduce
- In the web IDE, create a PartitionedTable with a LocalDate column:
from deephaven import empty_table pt = empty_table(5).update_view(["I=i", "Date=java.time.LocalDate.ofEpochDay(ii)"]).partition_by("Date")
- View the PartitionedTable widget.
Expected results
A partition selector dropdown should be visible, and there should be no errors in the JS console.
Actual results
There is no way to select a partition (only to merge the table), and an error is present:
Uncaught (in promise) Error: java.lang.NumberFormatException: For input string: "1970-01-05"
at NumberFormatException.createError (dh-core.js:1472:10)
at NumberFormatException.initializeBackingError (dh-core.js:1498:46)
at NumberFormatException.Throwable_0 (dh-core.js:1429:8)
at NumberFormatException.Exception_1 (dh-core.js:1523:18)
at NumberFormatException.RuntimeException_1 (dh-core.js:1541:18)
at NumberFormatException.IllegalArgumentException_0 (dh-core.js:1555:25)
at new NumberFormatException (dh-core.js:41262:33)
at Object.__parseAndValidateDouble (dh-core.js:40739:16)
at Object.ofNumber (dh-core.js:24934:27)
at C.makeFilterRawValue (TableUtils.js:1399:27)
at C.makeNullableEqFilter (TableUtils.js:1412:36)
at IrisGridPartitionSelector.js:220:34
at Array.map (<anonymous>)
at IrisGridPartitionSelector.js:218:57
at Generator.next (<anonymous>)
at ri (IrisGridPartitionSelector.js:3:102)
at o (IrisGridPartitionSelector.js:4:194)
at IrisGridPartitionSelector.js:4:364
at new Promise (<anonymous>)
at IrisGridPartitionSelector.js:4:97
at _o.updatePartitionOptions (IrisGridPartitionSelector.js:234:6)
at _o.componentDidUpdate (IrisGridPartitionSelector.js:75:12)
at het (react-dom.production.min.js:219:502)
at $et (react-dom.production.min.js:259:160)
at e.unstable_runWithPriority (scheduler.production.min.js:18:343)
at ag (react-dom.production.min.js:122:325)
at vv (react-dom.production.min.js:252:279)
at gF (react-dom.production.min.js:243:371)
at react-dom.production.min.js:123:115
at e.unstable_runWithPriority (scheduler.production.min.js:18:343)
at ag (react-dom.production.min.js:122:325)
at C5e (react-dom.production.min.js:123:61)
at Tp (react-dom.production.min.js:122:428)
at Vm (react-dom.production.min.js:237:203)
at Object.enqueueSetState (react-dom.production.min.js:133:211)
at w3.setState (react.production.min.js:12:369)
at IrisGrid.js:1446:18
at Object.$lambda$3 (dh-core.js:12276:5)
at Function.onInvoke_127 (dh-core.js:24649:17)
at lambda (dh-core.js:195:22)
at Array.forEach (<anonymous>)
at Object.$fireEvent (dh-core.js:12255:15)
at Object.$refire (dh-core.js:29373:106)
at TableViewportSubscription.notifyUpdate_0 [as notifyUpdate] (dh-core.js:29472:11)
at Object.$onDataChanged (dh-core.js:28706:15)
at AbstractTableSubscription$1methodref$onDataChanged$Type.onDataChanged (dh-core.js:28850:11)
at WebBarrageSubscription$ViewportImpl.applyUpdates_1 [as applyUpdates] (dh-core.js:22270:27)
at Object.$onFlightData (dh-core.js:28723:48)
at Function.apply_157 (dh-core.js:28877:11)
at lambda (dh-core.js:195:22)
at dh-internal.js:1:350285
at Array.forEach (<anonymous>)
at dh-internal.js:1:350264
at dh-internal.js:1:11539
at Array.forEach (<anonymous>)
at e.rawOnMessage (dh-internal.js:1:11501)
at dh-internal.js:1:9314
at Array.forEach (<anonymous>)
at e.onTransportChunk (dh-internal.js:1:9189)
at Object.$onMessage (dh-core.js:25125:35)
at MultiplexedWebsocketTransport$3methodref$onMessage$Type.handleEvent_2 [as handleEvent] (dh-core.js:25268:10)
Table with LocalDate partitions
Steps to reproduce
- In the web IDE, create a PartitionedTable with a LocalDate column, write it to disk as a parquet table in partitions, and read it back as a single partition. Note that this will work, ddeasdfe
from deephaven.parquet import write_partitioned, read, ParquetFileLayout
from deephaven import empty_table
from deephaven.column import col_def, ColumnType
from deephaven import dtypes
pt = empty_table(5).update_view(["I=i", "Date=java.time.LocalDate.ofEpochDay(ii)"]).partition_by("Date")
write_partitioned(pt, destination_dir='/tmp/table_with_partitions')
table_with_partitions = read('/tmp/table_with_partitions', table_definition=[
col_def('I', data_type=dtypes.int32),
col_def("Date", data_type=dtypes.LocalDate, column_type=ColumnType.PARTITIONING)
])
- View table
table_with_partitions.
Expected results
A partition selector dropdown should be visible, and there should be no errors in the JS console.
Actual results
Instead of seeing the table and the partition selector, we see an error message:
Unable to open table. Error: Error displaying table: Error: java.lang.NumberFormatException: For input string: "1970-01-05"
This is the same error as the PartitionedTable case, but hiding the table itself.
In both cases the web UI seems to be trying to serialize the LocalDate via Filters
- for the second case this makes sense, since a Table must/should be filtered by its partitions. There is no JS call to create a LocalDateWrapper, nor a FilterValue factory method to wrap them. See JS API support for filtering other supported types deephaven-core#6436
- for the first, case, we "just" need the ability to support LocalDate instances in
dh.IdeSession.newTable(...). See Web UI CSV Upload followup deephaven-core#1041 via JS Flight/Barrage Refactor follow up deephaven-core#5927.