"Undefined column" after upgrading linkding to v1.27.0 using postgres DB backend

[soerenschneider]

Thanks for your effort on linkding, I love it!

After upgrading from v1.26.0 to v1.27.0, I only get HTTP 500 errors. As I'm using PostgreSQL as DB backend, I guess some columns are missing. Going back to v1.26.0 solved the issue temporarily. Here's an excerpt of the log:

2024-04-05 06:56:57,958 ERROR Internal Server Error: /bookmarks
Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 506, in get
rel_obj = self.related.get_cached_value(instance)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/fields/mixins.py", line 15, in get_cached_value
return instance._state.fields_cache[cache_name]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'profile'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column bookmarks_userprofile.enable_automatic_html_snapshots does not exist
LINE 1: ...s", "bookmarks_userprofile"."search_preferences", "bookmarks...
^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/utils/deprecation.py", line 133, in call
response = self.process_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/contrib/auth/middleware.py", line 94, in process_request
auth.login(request, user)
File "/opt/venv/lib/python3.11/site-packages/django/contrib/auth/init.py", line 152, in login
user_logged_in.send(sender=user.class, request=request, user=user)
File "/opt/venv/lib/python3.11/site-packages/django/dispatch/dispatcher.py", line 189, in send
response = receiver(signal=self, sender=sender, **named)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/etc/linkding/bookmarks/signals.py", line 11, in user_logged_in
tasks.schedule_bookmarks_without_snapshots(user)
File "/etc/linkding/bookmarks/services/tasks.py", line 113, in schedule_bookmarks_without_snapshots
if is_web_archive_integration_active(user):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/etc/linkding/bookmarks/services/tasks.py", line 24, in is_web_archive_integration_active
user.profile.web_archive_integration
^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 514, in get
rel_obj = self.get_queryset(instance=instance).get(**filter_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/query.py", line 645, in get
num = len(clone)
^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/query.py", line 382, in len
self._fetch_all()
File "/opt/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1928, in _fetch_all
self._result_cache = list(self._iterable_class(self))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/query.py", line 91, in iter
results = compiler.execute_sql(
^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
cursor.execute(sql, params)
File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/opt/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: column bookmarks_userprofile.enable_automatic_html_snapshots does not exist
LINE 1: ...s", "bookmarks_userprofile"."search_preferences", "bookmarks...
^

[sissbruecker]

django.db.utils.ProgrammingError: column bookmarks_userprofile.enable_automatic_html_snapshots

Looks like the latest migrations were not run. Could you please share the Docker logs from starting a fresh container? It should contain something like this:

  Applying bookmarks.0029_bookmark_list_actions_toast... OK
  Applying bookmarks.0030_bookmarkasset... OK
  Applying bookmarks.0031_userprofile_enable_automatic_html_snapshots... OK
  Applying bookmarks.0032_html_snapshots_hint_toast... OK

There might be some errors in that area.

[sissbruecker]

Something else to try would be to open a shell in the container and then run:

python manage.py migrate bookmarks

And then see what the output of that is.

[leepeuker]

I had the same issue. My database migrations could not be executed because the LD_CSRF_TRUSTED_ORIGINS environment variable value did not have a leading http(s)://

$ docker exec -it linkding bash
root@a024777586fe:/etc/linkding# python manage.py migrate bookmarks
SystemCheckError: System check identified some issues:

ERRORS:
?: (4_0.E001) As of Django 4.0, the values in the CSRF_TRUSTED_ORIGINS setting must start with a scheme (usually http:// or https://) but found linkding.leepeuker.dev. See the release notes for details.

After adjusting the environment variable value it worked fine.

[eli-yip]

Something else to try would be to open a shell in the container and then run:

python manage.py migrate bookmarks

And then see what the output of that is.

Using linkding1.28.0 and postgres as database backend, I got such output:

python manage.py migrate bookmarks
Operations to perform:
  Apply all migrations: bookmarks
Running migrations:
  Applying bookmarks.0030_bookmarkasset...Traceback (most recent call last):
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.InsufficientPrivilege: permission denied for schema public
LINE 1: CREATE TABLE "bookmarks_bookmarkasset" ("id" integer NOT NUL...
                     ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/etc/linkding/manage.py", line 21, in <module>
    main()
  File "/etc/linkding/manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/venv/lib/python3.11/site-packages/django/db/migrations/operations/models.py", line 96, in database_forwards
    schema_editor.create_model(model)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 488, in create_model
    self.execute(sql, params or None)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 45, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 201, in execute
    cursor.execute(sql, params)
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: permission denied for schema public
LINE 1: CREATE TABLE "bookmarks_bookmarkasset" ("id" integer NOT NUL...
                     ^