Upload inputs.json to FireCloud fails

[bradtaylor]

User Matt Bookman reported:
When I go into FireCloud and go through the steps to "Populate with a json file" (select the file, preview the file and click "Upload"), it just spins indicating "Uploading file..." but never returns.

I'm trying to use the json file that is packaged with the WDL file in github. I have made no modifications to it.

In the Chrome "Developers Tools", in the FireCloud window, I see:

newrelic.js:2 Uncaught SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at Function.broadfcui.utils.parse_json_string.cljs$core$IFn$_invoke$arity$3 (base-deps.bundle.js?t=1534445958973:84)
    at Function.broadfcui.utils.parse_json_string.cljs$core$IFn$_invoke$arity$2 (base-deps.bundle.js?t=1534445958973:84)
    at base-deps.bundle.js?t=1534445958973:84
    at Function.<anonymous> (base-deps.bundle.js?t=1534445958973:84)
    at Function.cljs.core.apply_to_simple.cljs$core$IFn$_invoke$arity$3 (base-deps.bundle.js?t=1534445958973:84)
    at Function.cljs.core.apply.cljs$core$IFn$_invoke$arity$3 (base-deps.bundle.js?t=1534445958973:84)
    at Function.dmohs.react.core.call_fn.cljs$core$IFn$_invoke$arity$variadic (base-deps.bundle.js?t=1534445958973:84)
    at dmohs.react.core.call_fn (base-deps.bundle.js?t=1534445958973:84)
    at c.e (base-deps.bundle.js?t=1534445958973:84)
broadfcui.utils.parse_json_string.cljs$core$IFn$_invoke$arity$3 @ base-deps.bundle.js?t=1534445958973:84
broadfcui.utils.parse_json_string.cljs$core$IFn$_invoke$arity$2 @ base-deps.bundle.js?t=1534445958973:84
(anonymous) @ base-deps.bundle.js?t=1534445958973:84
(anonymous) @ base-deps.bundle.js?t=1534445958973:84
cljs.core.apply_to_simple.cljs$core$IFn$_invoke$arity$3 @ base-deps.bundle.js?t=1534445958973:84
cljs.core.apply.cljs$core$IFn$_invoke$arity$3 @ base-deps.bundle.js?t=1534445958973:84
dmohs.react.core.call_fn.cljs$core$IFn$_invoke$arity$variadic @ base-deps.bundle.js?t=1534445958973:84
dmohs.react.core.call_fn @ base-deps.bundle.js?t=1534445958973:84
e @ base-deps.bundle.js?t=1534445958973:84
s @ base-deps.bundle.js?t=1534445958973:84
dmohs.react.core.call.cljs$core$IFn$_invoke$arity$variadic @ base-deps.bundle.js?t=1534445958973:84
dmohs.react.core.call @ base-deps.bundle.js?t=1534445958973:84
e.cljs$core$IFn$_invoke$arity$1 @ base-deps.bundle.js?t=1534445958973:84
(anonymous) @ base-deps.bundle.js?t=1534445958973:84
(anonymous) @ base-deps.bundle.js?t=1534445958973:60
invokeGuardedCallback @ base-deps.bundle.js?t=1534445958973:60
invokeGuardedCallbackAndCatchFirstError @ base-deps.bundle.js?t=1534445958973:60
b @ base-deps.bundle.js?t=1534445958973:60
E @ base-deps.bundle.js?t=1534445958973:60
T @ base-deps.bundle.js?t=1534445958973:60
D @ base-deps.bundle.js?t=1534445958973:60
N @ base-deps.bundle.js?t=1534445958973:60
O @ base-deps.bundle.js?t=1534445958973:60
dr @ base-deps.bundle.js?t=1534445958973:60
batchedUpdates @ base-deps.bundle.js?t=1534445958973:60
Ve @ base-deps.bundle.js?t=1534445958973:60
hr @ base-deps.bundle.js?t=1534445958973:60
interactiveUpdates @ base-deps.bundle.js?t=1534445958973:60
mr @ base-deps.bundle.js?t=1534445958973:60
nrWrapper @ newrelic.js:2

I believe it has to do with JSON file size. I started some Fred Flintstone debugging, and just cut the file smaller and smaller. It worked fine when the file was under 4097 characters, but when it was 4098 or above it would generate this error.

It seems suspiciously like it is a failure at 4k or 4096 bytes (the upload probably trims the EOF character). So it seems like the upload is probably truncating at 4k, which generates a JSON parser error.

[davidangb]

The json upload feature relies on the import_data.cljs component. That component, internally, uses JavaScript's FileReader to read the contents of the file in question.

However, the code very specifically only reads the first slice of the file:
https://github.com/broadinstitute/firecloud-ui/blob/develop/src/cljs/main/broadfcui/page/workspace/data/import_data.cljs#L44

and the slice size is set to 4096:
https://github.com/broadinstitute/firecloud-ui/blob/develop/src/cljs/main/broadfcui/page/workspace/data/import_data.cljs#L15

originally, import_data.cljs was only used for previewing files, so the slicing was a valid choice. It was repurposed to handle json upload, but the json upload needs a way to read the whole file, not just the first slice.

[mbookman]

Is there a PR to attach to this with a code change? (#193 is release notes only)

[jmthibault79]

@mbookman Here is the code change: broadinstitute/firecloud-ui#1413

[mbookman]

Thanks!