Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert with --multi and --dry-run flag does not work #415

Closed
dotcs opened this issue Mar 16, 2022 · 2 comments
Closed

Convert with --multi and --dry-run flag does not work #415

dotcs opened this issue Mar 16, 2022 · 2 comments
Labels
bug Something isn't working

Comments

@dotcs
Copy link

dotcs commented Mar 16, 2022

It's not possible to combine --multi and --dry-run flag in the convert command.

Let's first create a simple database from JSON string

$ echo '[{"foo": "abc"}]' | sqlite-utils insert demo.db demo -
$ sqlite-utils query demo.db "SELECT * FROM demo"             
[{"foo": "abc"}]

and then try to convert the "foo" column with a static value "bar" (see docs Converting a column into multiple columns)

$ sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi --dry-run
Traceback (most recent call last):
  File "/home/dotcs/anaconda3/envs/tools/bin/sqlite-utils", line 8, in <module>
    sys.exit(cli())
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/sqlite_utils/cli.py", line 2686, in convert
    for row in db.conn.execute(sql, where_args).fetchall():
sqlite3.OperationalError: user-defined function raised exception

But without the --dry-run flag it does work as expected:

$ sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi
$ sqlite-utils query demo.db "SELECT * FROM demo"               
[{"foo": "bar"}]
$ sqlite-utils --version
sqlite-utils, version 3.25.1
@simonw simonw added the bug Something isn't working label Mar 21, 2022
@simonw
Copy link
Owner

simonw commented Mar 21, 2022

Thanks for the really clear steps to reproduce!

@simonw
Copy link
Owner

simonw commented Mar 21, 2022

I can fix this like so:

% sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi --dry-run
abc
 --- becomes:
{"foo": "bar"}

Would affect 1 row

Diff is this:

diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py
index 0cf0468..b2a0440 100644
--- a/sqlite_utils/cli.py
+++ b/sqlite_utils/cli.py
@@ -2676,7 +2676,10 @@ def convert(
         raise click.ClickException(str(e))
     if dry_run:
         # Pull first 20 values for first column and preview them
-        db.conn.create_function("preview_transform", 1, lambda v: fn(v) if v else v)
+        preview = lambda v: fn(v) if v else v
+        if multi:
+            preview = lambda v: json.dumps(fn(v), default=repr) if v else v
+        db.conn.create_function("preview_transform", 1, preview)
         sql = """
             select
                 [{column}] as value,

@simonw simonw closed this as completed in 751ab20 Mar 21, 2022
simonw added a commit that referenced this issue Mar 21, 2022
simonw added a commit that referenced this issue Apr 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants