You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When inserting or upserting a jsonlines file, no progressbar is shown. Expected behavior is that, just like with .csv/.tsv files, also for a jsonlines file (--nl), unless --silent is provided, a progressbar is shown.
sqlite-utils insert /tmp/t3.db t /tmp/big.json
[####################################] 100%
This is actually not doing the right thing. The problem is that sqlite-utils doesn't include a streaming JSON parser, so it instead reads that entire JSON file into memory first (exhausting the progress bar to 100% instantly) and then does the rest of the work in-memory while the bar sticks at 100%.
I decided to land this anyway. If a streaming JSON parser is added later it will start to work.
When inserting or upserting a jsonlines file, no progressbar is shown. Expected behavior is that, just like with .csv/.tsv files, also for a jsonlines file (--nl), unless --silent is provided, a progressbar is shown.
Currently
file_progress
is only called within the tsv/csv logic, however I think it can be safely wrapped around all the all the input formats that usedecoded
: https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/cli.py#L963The text was updated successfully, but these errors were encountered: