feat: Create UI to Display Partitioned Tables#1663
Conversation
…o 1143-partitioned-table-ui
…o 1143-partitioned-table-ui
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1663 +/- ##
==========================================
- Coverage 46.48% 46.05% -0.44%
==========================================
Files 617 621 +4
Lines 37289 37430 +141
Branches 9378 9399 +21
==========================================
- Hits 17335 17237 -98
- Misses 19900 20139 +239
Partials 54 54
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
dsmmcken
left a comment
dsmmcken
left a comment
There was a problem hiding this comment.
Boolean and Char columns don't seem to work in the UI as parition keys, even though they are supported by the engine:
Currently breaks when partition columns are not in order, looking into it Also seems to not handle |
…o 1143-partitioned-table-ui
dsmmcken
left a comment
dsmmcken
left a comment
There was a problem hiding this comment.
Something awful is happening with the performance here. I can only replicate it with the parition viewer, but it might be something in core. I don't know.
Steps to reproduce:
- run this, time_table, creating new a parition on each tick
from deephaven import empty_table, time_table
simple_ticking = time_table("PT00:00:01").update([
"MyString=new String(`a`+i)",
"MyInt=new Integer(i)",
"MyLong=new Long(i)",
"MyDouble=new Double(i+i/10)",
"MyFloat=new Float(i+i/10)",
"MyBoolean=new Boolean(i%2==0)",
"MyChar= new Character((char) ((i%26)+97))",
"MyShort=new Short(Integer.toString(i%32767))",
"MyByte= new java.lang.Byte(Integer.toString(i%127))"])
p = simple_ticking.partition_by("MyString")
- Wait like a minute or two then take a performance snapshot. You'll get a million little nano-second locks too the point of it breaking everything.
|
The key columns passed in by the js api has incorrect order, going to fix that issue in core first. This should be the cause of dropdowns not showing any options for certain column orderings independent of type. There might still more issues when dealing with null or empty values and some type-dependent issues unrelated to this. Update: fix opened at deephaven/deephaven-core#4931 |
- Use EMPTY_ARRAY instead of returning a new array each time from the EmptyGridModel - Reduces re-renders - Save the current viewport in IrisGridProxyModel, and apply it to the new model if the columns haven't changed - Clean up hydration/dehydration - Needed to dehydrate the value correctly
- Return the partitioned tables columns instead of no columns, so rehydration will succeed - Fix char type partitions - needed to create the raw filter value better
| function makeModel( | ||
| columns = irisGridTestUtils.makeColumns() | ||
| ): PartitionedGridModel { | ||
| const model = { |
There was a problem hiding this comment.
Not worth changing, but createMockProxy could make this a bit cleaner
bmingles
left a comment
bmingles
left a comment
There was a problem hiding this comment.
Left 2 minor comments about missing async
- Add async to methods for consistency
…le (#5009) - Memoizing it is inconsistent with other methods like `WorkerConnection.getTable`, which always returns a new table - Was unclear who the "owner" of the returned table was - Memoization can happen at the client level if they need it - Tested with deephaven/web-client-ui#1663
dsmmcken
left a comment
dsmmcken
left a comment
There was a problem hiding this comment.
String column not working. Boolean column not working.
And I get this error occasionally:
Tried testing using dx.stocks and mostdatatypes queries.
- Was referencing the columns from the original table instead of the actual table - Was causing a mismatch when fetching the viewport, where it would just stall instead of throwing an error
|
@dsmmcken should be fixed. Also, you'll need to make sure you're on the latest |
|
Don's found two errors that are still occurring, both of them are because it is a ticking table starting with no partition. Using this snippet you can reproduce both issues: from deephaven import empty_table, time_table
simple_ticking = time_table("PT00:00:01").update([
"MyString=new String(`a`+i)",
"MyInt=new Integer(i)",
"MyLong=new Long(i)",
"MyDouble=new Double(i+i/10)",
"MyFloat=new Float(i+i/10)",
"MyBoolean=new Boolean(i%2==0)",
"MyChar= new Character((char) ((i%26)+97))",
"MyShort=new Short(Integer.toString(i%32767))",
"MyByte= new java.lang.Byte(Integer.toString(i%127))"])
p = simple_ticking.partition_by(["MyString", "MyInt", "MyBoolean"])
Workaround for both issues is to re-open the table. |
- Wait for a key to be added to the keys table if one does not initially exist - If the reloaded partition config is invalid, will reload the initial partition config
dsmmcken
left a comment
dsmmcken
left a comment
There was a problem hiding this comment.
Looks good, all working now. Changed a couple color variables to theme better.
4 participants
PartitionedTableobjects can now be opened and displayed with a new UITesting Instructions
PartitionedTable
ptParquet Tables