CSV download optimizations for larger files (formerly: "*doesn't work in release candidate")

[kosak]

I install Deephaven per the instructions (except with VERSION = edge)

In the Python window, I type

from deephaven.TableTools import *
t = emptyTable(10)

In the TableOptions menu, I select "Download CSV", keep the defaults and select "Download".
It pauses for a moment and then says "Download Canceled" (see screenshot).

[niloc132]

@mofojed can you look at this? The example in the description has no columns, so it is the most boring csv file ever, but even adding a column I am seeing a 404 at /ide/download/table-2021-12-09-142136.csv instead of a real download.

[mofojed]

@kosak did you test this with a table with any data? E.g:

from deephaven.TableTools import *
t = emptyTable(10).updateView("I=i")

Also what browser are you using?
There seems to be a problem exporting an empty table for sure, but that doesn't seem like a release blocker.
I can't reproduce in Firefox or Chromium, but Colin has an issue in Firefox, would like to know what you have so we can decide what to investigate. If you can test in another browser as well that would be great.

[niloc132]

FF94 failed for this, but ff95 (latest, released 3 days ago) does not for me.

It is definitely a bug to not handle a table with no columns correctly, but not a critical one.

[niloc132]

Okay, I reproduced this with Corey on a call, interestingly it does manifest in the same "Download Canceled" way as the no-column case. It can be seen with any large table (note that if it is too large you get a different error from the server), for example:

t = emptyTable(1000000).updateView("I=i")

The download prompt will appear, but the download will fail.

The issue is that JS's TableViewportSubscription.snapshot(rows, columns) doesnt actually use the rows and columns it is given, but returns the entire table. This seems to break the client due to a timeout of some kind.

With a much larger table, you get a different error, the server refuses to send payloads that big (could be in the grpc server, could be in envoy, I didn't investigate). This will also be fixed by fixing TableViewportSubscription.snapshot.

Options to fix:

• Fix JS API Table snapshot should support specific rows, columns #1039, by first fixing (Viewport) Support DoExchange equivalent of ConstructSnapshot over Flight #925 on the server and in the barrage spec
• Fix JS API Table snapshot should support specific rows, columns #1039 by using something like Table.slice and dropColumns to pick the data that the client will get.
• Change the JS UI to slide the viewport along the csv file. This may have a negative impact on the speed of

[niloc132]

Another consideration is that the UI should indicate up front the total row x col set that will be interesting, and snapshot that rather than the entire table.

[pete-petey]

Relates to CSV download being limited in size in web. A "gradual" approach with optimization is appropriate -- API and JS work.