'sqlite3.OperationalError: unable to open database file' error with a fresh install

[MohHeader]

Hi,

I just installed llm

pip install llm

then when I use it for anything it would output the result then append it with the following error message

Traceback (most recent call last):
  File "/usr/local/bin/llm", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llm/cli.py", line 257, in prompt
    db = sqlite_utils.Database(log_path)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlite_utils/db.py", line 335, in __init__
    self.conn = sqlite3.connect(str(filename_or_conn))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file

[simonw]

That's surprising - what operating system is that?

You may be able to get a better idea of what's going on by running this:

python -i $(which llm) models list

This should drop you into a Python interactive prompt when the program hits an error. Then run:

>>> import pdb; pdb.pm()

And then:

(pdb) str(filename_or_conn)`

To see the value of that variable.

[MohHeader]

@simonw, Thanks for your response

OS: macOS Monterey
Intel Processor

mohheader@Mohammeds-MacBook-Pro ~ % llm 'Hi there'
Hello! How can I assist you today?
Traceback (most recent call last):
  File "/usr/local/bin/llm", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/llm/cli.py", line 257, in prompt
    db = sqlite_utils.Database(log_path)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlite_utils/db.py", line 335, in __init__
    self.conn = sqlite3.connect(str(filename_or_conn))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: unable to open database file

mohheader@Mohammeds-MacBook-Pro ~ % python -i $(which llm) models list
OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k)
OpenAI Chat: gpt-4 (aliases: 4, gpt4)
OpenAI Chat: gpt-4-32k (aliases: 4-32k)
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1065, in main
    ctx.exit()
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 687, in exit
    raise Exit(code)
click.exceptions.Exit: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/llm", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1083, in main
    sys.exit(e.exit_code)
SystemExit: 0

>>> import pdb; pdb.pm()
> /usr/local/lib/python3.11/site-packages/click/core.py(1083)main()
-> sys.exit(e.exit_code)

(Pdb) str(filename_or_conn)
*** NameError: name 'filename_or_conn' is not defined
(Pdb) 

[MohHeader]

Also I want to confirm something

as I noticed it is listing gpt-4-32k in the models list, although I don't have access to it yet.

[dsummersl]

I’m also seeing an error like this (on osx 13.4.1).

llm models list
OpenAI Chat: gpt-3.5-turbo (aliases: 3.5, chatgpt)
OpenAI Chat: gpt-3.5-turbo-16k (aliases: chatgpt-16k, 3.5-16k)
OpenAI Chat: gpt-4 (aliases: 4, gpt4)
OpenAI Chat: gpt-4-32k (aliases: 4-32k)

llm logs
Error: No log database found at /Users/danesummers/Library/Application Support/io.datasette.llm/logs.db

# I tried to fix it myself :P
mkdir ~/Library/Application\ Support/io.datasette.llm
sqlite3 ~/Library/Application\ Support/io.datasette.llm/logs.db "vacuum;"
llm logs

…
…
  File "~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/llm/cli.py", line 431, in logs_list
    migrate(db)
  File "~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/llm/migrations.py", line 14, in migrate
    fn(db)
  File "~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/llm/migrations.py", line 61, in m003_chat_id_foreign_key
    db["log"].add_foreign_key("chat_id", "log", "id")
  File "~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/sqlite_utils/db.py", line 2194, in add_foreign_key
    self.db.add_foreign_keys([(self.name, column, other_table, other_column)])
  File "~/.pyenv/versions/3.10.2/lib/python3.10/site-packages/sqlite_utils/db.py", line 1143, in add_foreign_keys
    cursor.execute(
sqlite3.OperationalError: table sqlite_master may not be modified

Not sure how to debug this further but happy to help…

[jasonrsmith]

I'm seeing the same thing and was able to workaround it by commenting out the two add_foreign_key statements in llm/migrations.py. Not sure if this has any negative impact, but I am seeing logs now.

I also needed to manually create the logs directory as mentioned above:

mkdir ~/Library/Application\ Support/io.datasette.llm

Here's the diff for llm/migrations.py:

61c61
<     db["log"].add_foreign_key("chat_id", "log", "id")
---
>     #db["log"].add_foreign_key("chat_id", "log", "id")
139c139,140
<     db["logs"].add_foreign_key("reply_to_id", "logs", "id")
---
>     #db["logs"].add_foreign_key("reply_to_id", "logs", "id")
>     pass
 

[dsummersl]

This thread was useful to me: simonw/sqlite-utils#235

Seems to be a common problem, which can be address by:

pip3 install sqlean.py
pip3 install sqlite-dump

# now works!
llm logs list

[MohHeader]

Hello, I appreciate everyone's input.
By implementing the following two suggested solutions, I successfully resolved the issue. Thank you all.

@dsummersl

pip3 install sqlean.py
pip3 install sqlite-dump

@jasonrsmith

mkdir ~/Library/Application\ Support/io.datasette.llm

For future reference, I also uninstalled both sqlean.py and sqlite-dump, and everything continues to function correctly. So, I'm not entirely certain if solely the 'mkdir' solution would have been sufficient to fix the issue.

[ryanau]

Just running the following line works for me 👍

mkdir ~/Library/Application\ Support/io.datasette.llm

[tranqy]

Same problem here, fresh install. I did not have a .config folder. Once I added both .config and then io.datasette.llm inside that things worked as expected.

[amit]

I encountered the same issue today on WSL on Windows 10 (Ubuntu 20.04). It looks like the user_dir() (~/.config/io.datasette.llm) does not get created during the pip installation. I manually created that directory and after that the error went away!

[simonw]

Dupe of:

• Create io.datasette.llm in Application Support on install #193

Fixed in b30f689