Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ProgrammingError exception when upgrading to v4.0 #15605

Closed
opericgithub opened this issue Apr 3, 2024 · 24 comments
Closed

ProgrammingError exception when upgrading to v4.0 #15605

opericgithub opened this issue Apr 3, 2024 · 24 comments
Assignees
@jeremystretch
Labels
beta Concerns a bug/feature in a beta release severity: medium Results in substantial degraded or broken functionality for specfic workflows status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application

Comments

@opericgithub
Copy link

opericgithub commented Apr 3, 2024 •
edited
Loading

Deployment Type

Self-hosted

NetBox Version

v3.7.4 and v3.6.9

Python Version

3.10

Steps to Reproduce

I'm not sure whether this is the right place, but please correct me if I should post this issue in some other place.
I tried to do the upgrade from v3.7.4 as well as from v3.6.9, to the new v4.0.x (x = either feature, or beta).

First I tried upgrading to feature branch few days ago, and now I tried upgrading to v4.0-beta1.
I was using the following commands:

cd /opt/netbox
sudo git pull origin v4.0-beta1
sudo ./upgrade.sh

Then I get the following error messages:

Installing collected packages: pyasn1, pyasn1-modules, python-ldap, django-auth-ldap
Successfully installed django-auth-ldap-4.7.0 pyasn1-0.6.0 pyasn1-modules-0.4.0 python-ldap-3.4.4
Applying database migrations (python3 netbox/manage.py migrate)...
Operations to perform:
  Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying circuits.0043_circuittype_color... OK
  Applying core.0006_datasource_type_remove_choices... OK
  Applying core.0007_job_add_error_field... OK
  Applying core.0008_contenttype_proxy... OK
  Applying core.0009_configrevision... OK
  Applying core.0010_gfk_indexes... OK
  Applying dcim.0183_devicetype_exclude_from_utilization... OK
  Applying dcim.0184_protect_child_interfaces... OK
  Applying dcim.0185_gfk_indexes... OK
  Applying dcim.0186_location_facility... OK
  Applying extras.0099_cachedvalue_ordering... OK
  Applying extras.0100_customfield_ui_attrs... OK
  Applying extras.0101_eventrule... OK
  Applying extras.0102_move_configrevision... OK
  Applying extras.0103_gfk_indexes... OK
  Applying extras.0104_stagedchange_remove_change_logging... OK
  Applying extras.0105_customfield_min_max_values... OK
  Applying extras.0106_bookmark_user_cascade_deletion... OK
  Applying extras.0107_cachedvalue_extras_cachedvalue_object... OK
  Applying extras.0108_convert_reports_to_scripts... OK
  Applying extras.0109_script_model... OK
  Applying extras.0110_remove_eventrule_action_parameters... OK
  Applying extras.0111_rename_content_types...Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedTable: relation "extras_customfield_content_types_id_seq" does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/operations/special.py", line 106, in database_forwards
    self._run_sql(schema_editor, self.sql)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/operations/special.py", line 133, in _run_sql
    schema_editor.execute(statement, params=None)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/postgresql/schema.py", line 45, in execute
    return super().execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 201, in execute
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: relation "extras_customfield_content_types_id_seq" does not exist

Am I doing something wrong? Is there anything else I should check for?
I am also afraid that I won't be able to do the upgrade to the final version v4.0 in few weeks.

Expected Behavior

I expect to upgrade to the new (beta) version, as I was able to do this in the past using the same commands.

Observed Behavior

I get the error:
django.db.utils.ProgrammingError: relation "extras_customfield_content_types_id_seq" does not exist
Detailed log messages are given above.
@opericgithub opericgithub added status: needs triage This issue is awaiting triage by a maintainer type: bug A confirmed report of unexpected behavior in the application labels Apr 3, 2024
@jeremystretch jeremystretch added status: under review Further discussion is needed to determine this issue's scope and/or implementation and removed status: needs triage This issue is awaiting triage by a maintainer labels Apr 3, 2024
@jeremystretch
Copy link
Member

I tried to do the upgrade from v3.7.4 as well as from v3.6.9, to the new v4.0.x

You cannot upgrade from v3.6 directly to v4.0: Per the documentation you'll need to first upgrade to v3.7 to ensure all migrations are applied prior to being squashed.

NetBox can generally be upgraded directly to any newer release with no interim steps, with the one exception being incrementing major versions. This can be done only from the most recent minor release of the major version.

However, it should work fine when upgrading from v3.7.x. On a clean install of NetBox v3.7.4 (starting with an empty database), I have extras_customfield_content_types_id_seq:

netbox=> \d extras_customfield_content_types_id_seq
          Sequence "public.extras_customfield_content_types_id_seq"
  Type  | Start | Minimum |       Maximum       | Increment | Cycles? | Cache 
--------+-------+---------+---------------------+-----------+---------+-------
 bigint |     1 |       1 | 9223372036854775807 |         1 | no      |     1
Sequence for identity column: public.extras_customfield_content_types.id

And after switching to the v4.0-beta code base, all migrations apply successfully:

$ ./manage.py migrate
Operations to perform:
  Apply all migrations: account, admin, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying dcim.0186_location_facility... OK
  Applying extras.0108_convert_reports_to_scripts... OK
  Applying extras.0109_script_model... OK
  Applying extras.0110_remove_eventrule_action_parameters... OK
  Applying extras.0111_rename_content_types... OK
  Applying extras.0112_tag_update_object_types... OK
  Applying extras.0113_customfield_rename_object_type... OK
  Applying tenancy.0015_contactassignment_rename_content_type... OK
  Applying users.0005_alter_user_table... OK
  Applying users.0006_custom_group_model... OK
  Applying users.0007_objectpermission_update_object_types... OK
  Applying users.0008_flip_objectpermission_assignments... OK
  Applying vpn.0005_rename_indexes... OK

With your original v3.7.4 database in place, do you see extras_customfield_content_types_id_seq defined? (You can use manage.py dbshell to enter the PostgreSQL command prompt and type \d to list all tables.)

@opericgithub
Copy link
Author

With this:

cd /opt/netbox/netbox
python3 manage.py dbshell

I get the following:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 58, in __getitem__
    return getattr(self._connections, alias)
  File "/usr/local/lib/python3.10/dist-packages/asgiref/local.py", line 105, in __getattr__
    raise AttributeError(f"{self!r} object has no attribute {key!r}")
AttributeError: <asgiref.local.Local object at 0x7f0a43267610> object has no attribute 'default'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/dbshell.py", line 28, in handle
    connection = connections[options["database"]]
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 60, in __getitem__
    if alias not in self.settings:
  File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 45, in settings
    self._settings = self.configure_settings(self._settings)
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 148, in configure_settings
    databases = super().configure_settings(databases)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 50, in configure_settings
    settings = getattr(django_settings, self.settings_name)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 87, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 74, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 183, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/netbox/netbox/netbox/settings.py", line 22, in <module>
    from netbox.config import PARAMS
  File "/opt/netbox/netbox/netbox/config/__init__.py", line 9, in <module>
    from .parameters import PARAMS
  File "/opt/netbox/netbox/netbox/config/parameters.py", line 2, in <module>
    from django.contrib.postgres.forms import SimpleArrayField
  File "/usr/local/lib/python3.10/dist-packages/django/contrib/postgres/forms/__init__.py", line 3, in <module>
    from .ranges import *  # NOQA
  File "/usr/local/lib/python3.10/dist-packages/django/contrib/postgres/forms/ranges.py", line 1, in <module>
    from psycopg2.extras import DateRange, DateTimeTZRange, NumericRange
ModuleNotFoundError: No module named 'psycopg2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 427, in run_from_argv
    connections.close_all()
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 212, in close_all
    for alias in self:
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 73, in __iter__
    return iter(self.settings)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 49, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 45, in settings
    self._settings = self.configure_settings(self._settings)
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 148, in configure_settings
    databases = super().configure_settings(databases)
  File "/usr/local/lib/python3.10/dist-packages/django/utils/connection.py", line 50, in configure_settings
    settings = getattr(django_settings, self.settings_name)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 87, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 74, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python3.10/dist-packages/django/conf/__init__.py", line 183, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/netbox/netbox/netbox/settings.py", line 22, in <module>
    from netbox.config import PARAMS
  File "/opt/netbox/netbox/netbox/config/__init__.py", line 9, in <module>
    from .parameters import PARAMS
  File "/opt/netbox/netbox/netbox/config/parameters.py", line 2, in <module>
    from django.contrib.postgres.forms import SimpleArrayField
  File "/usr/local/lib/python3.10/dist-packages/django/contrib/postgres/forms/__init__.py", line 3, in <module>
    from .ranges import *  # NOQA
  File "/usr/local/lib/python3.10/dist-packages/django/contrib/postgres/forms/ranges.py", line 1, in <module>
    from psycopg2.extras import DateRange, DateTimeTZRange, NumericRange
ModuleNotFoundError: No module named 'psycopg2'

@jeremystretch
Copy link
Member

ModuleNotFoundError: No module named 'psycopg2'

You're missing at least one required library, which indicates that the installation/upgrade instructions were not followed. Please go back and ensure that your v3.7 deployment is fully functional before attempting to upgrade to v4.0.

@opericgithub
Copy link
Author

Thanks, I beleive I now installed all django packages.
Then I tried this:

netbox=> \d extras_customfield_content_types_id_seq
Did not find any relation named "extras_customfield_content_types_id_seq".

Here are all the tables (347 rows):

netbox=> \d
                              List of relations
 Schema |                      Name                      |   Type   | Owner  
--------+------------------------------------------------+----------+--------
 public | auth_group                                     | table    | netbox
 public | auth_group_id_seq                              | sequence | netbox
 public | auth_group_permissions                         | table    | netbox
 public | auth_group_permissions_id_seq                  | sequence | netbox
 public | auth_permission                                | table    | netbox
 public | auth_permission_id_seq                         | sequence | netbox
 public | auth_user                                      | table    | netbox
 public | auth_user_groups                               | table    | netbox
 public | auth_user_groups_id_seq                        | sequence | netbox
 public | auth_user_id_seq                               | sequence | netbox
 public | auth_user_user_permissions                     | table    | netbox
 public | auth_user_user_permissions_id_seq              | sequence | netbox
 public | circuits_circuit                               | table    | netbox
 public | circuits_circuit_id_seq                        | sequence | netbox
 public | circuits_circuittermination                    | table    | netbox
 public | circuits_circuittermination_id_seq             | sequence | netbox
 public | circuits_circuittype                           | table    | netbox
 public | circuits_circuittype_id_seq                    | sequence | netbox
 public | circuits_provider                              | table    | netbox
 public | circuits_provider_asns                         | table    | netbox
 public | circuits_provider_asns_id_seq                  | sequence | netbox
 public | circuits_provider_id_seq                       | sequence | netbox
 public | circuits_provideraccount                       | table    | netbox
 public | circuits_provideraccount_id_seq                | sequence | netbox
 public | circuits_providernetwork                       | table    | netbox
 public | circuits_providernetwork_id_seq                | sequence | netbox
 public | core_autosyncrecord                            | table    | netbox
 public | core_autosyncrecord_id_seq                     | sequence | netbox
 public | core_configrevision                            | table    | netbox
 public | core_datafile                                  | table    | netbox
 public | core_datafile_id_seq                           | sequence | netbox
 public | core_datasource                                | table    | netbox
 public | core_datasource_id_seq                         | sequence | netbox
 public | core_job                                       | table    | netbox
 public | core_job_id_seq                                | sequence | netbox
 public | core_managedfile                               | table    | netbox
 public | core_managedfile_id_seq                        | sequence | netbox
 public | dcim_cable                                     | table    | netbox
 public | dcim_cable_id_seq                              | sequence | netbox
 public | dcim_cablepath                                 | table    | netbox
 public | dcim_cablepath_id_seq                          | sequence | netbox
 public | dcim_cabletermination                          | table    | netbox
 public | dcim_cabletermination_id_seq                   | sequence | netbox
 public | dcim_consoleport                               | table    | netbox
 public | dcim_consoleport_id_seq                        | sequence | netbox
 public | dcim_consoleporttemplate                       | table    | netbox
 public | dcim_consoleporttemplate_id_seq                | sequence | netbox
 public | dcim_consoleserverport                         | table    | netbox
 public | dcim_consoleserverport_id_seq                  | sequence | netbox
 public | dcim_consoleserverporttemplate                 | table    | netbox
 public | dcim_consoleserverporttemplate_id_seq          | sequence | netbox
 public | dcim_device                                    | table    | netbox
 public | dcim_device_id_seq                             | sequence | netbox
 public | dcim_devicebay                                 | table    | netbox
 public | dcim_devicebay_id_seq                          | sequence | netbox
 public | dcim_devicebaytemplate                         | table    | netbox
 public | dcim_devicebaytemplate_id_seq                  | sequence | netbox
 public | dcim_devicerole                                | table    | netbox
 public | dcim_devicerole_id_seq                         | sequence | netbox
 public | dcim_devicetype                                | table    | netbox
 public | dcim_devicetype_id_seq                         | sequence | netbox
 public | dcim_frontport                                 | table    | netbox
 public | dcim_frontport_id_seq                          | sequence | netbox
 public | dcim_frontporttemplate                         | table    | netbox
 public | dcim_frontporttemplate_id_seq                  | sequence | netbox
 public | dcim_interface                                 | table    | netbox
 public | dcim_interface_id_seq                          | sequence | netbox
 public | dcim_interface_tagged_vlans                    | table    | netbox
 public | dcim_interface_tagged_vlans_id_seq             | sequence | netbox
 public | dcim_interface_vdcs                            | table    | netbox
 public | dcim_interface_vdcs_id_seq                     | sequence | netbox
 public | dcim_interface_wireless_lans                   | table    | netbox
 public | dcim_interface_wireless_lans_id_seq            | sequence | netbox
 public | dcim_interfacetemplate                         | table    | netbox
 public | dcim_interfacetemplate_id_seq                  | sequence | netbox
 public | dcim_inventoryitem                             | table    | netbox
 public | dcim_inventoryitem_id_seq                      | sequence | netbox
 public | dcim_inventoryitemrole                         | table    | netbox
 public | dcim_inventoryitemrole_id_seq                  | sequence | netbox
 public | dcim_inventoryitemtemplate                     | table    | netbox
 public | dcim_inventoryitemtemplate_id_seq              | sequence | netbox
 public | dcim_location                                  | table    | netbox
 public | dcim_manufacturer                              | table    | netbox
 public | dcim_manufacturer_id_seq                       | sequence | netbox
 public | dcim_module                                    | table    | netbox
 public | dcim_module_id_seq                             | sequence | netbox
 public | dcim_module_id_seq1                            | sequence | netbox
 public | dcim_modulebay                                 | table    | netbox
 public | dcim_modulebay_id_seq                          | sequence | netbox
 public | dcim_modulebaytemplate                         | table    | netbox
 public | dcim_modulebaytemplate_id_seq                  | sequence | netbox
 public | dcim_moduletype                                | table    | netbox
 public | dcim_moduletype_id_seq                         | sequence | netbox
 public | dcim_platform                                  | table    | netbox
 public | dcim_platform_id_seq                           | sequence | netbox
 public | dcim_powerfeed                                 | table    | netbox
 public | dcim_powerfeed_id_seq                          | sequence | netbox
 public | dcim_poweroutlet                               | table    | netbox
 public | dcim_poweroutlet_id_seq                        | sequence | netbox
 public | dcim_poweroutlettemplate                       | table    | netbox
 public | dcim_poweroutlettemplate_id_seq                | sequence | netbox
 public | dcim_powerpanel                                | table    | netbox
 public | dcim_powerpanel_id_seq                         | sequence | netbox
 public | dcim_powerport                                 | table    | netbox
 public | dcim_powerport_id_seq                          | sequence | netbox
 public | dcim_powerporttemplate                         | table    | netbox
 public | dcim_powerporttemplate_id_seq                  | sequence | netbox
 public | dcim_rack                                      | table    | netbox
 public | dcim_rack_id_seq                               | sequence | netbox
 public | dcim_rackgroup_id_seq                          | sequence | netbox
 public | dcim_rackreservation                           | table    | netbox
 public | dcim_rackreservation_id_seq                    | sequence | netbox
 public | dcim_rackrole                                  | table    | netbox
 public | dcim_rackrole_id_seq                           | sequence | netbox
 public | dcim_rearport                                  | table    | netbox
 public | dcim_rearport_id_seq                           | sequence | netbox
 public | dcim_rearporttemplate                          | table    | netbox
 public | dcim_rearporttemplate_id_seq                   | sequence | netbox
 public | dcim_region                                    | table    | netbox
 public | dcim_region_id_seq                             | sequence | netbox
 public | dcim_site                                      | table    | netbox
 public | dcim_site_asns                                 | table    | netbox
 public | dcim_site_asns_id_seq                          | sequence | netbox
 public | dcim_site_id_seq                               | sequence | netbox
 public | dcim_sitegroup                                 | table    | netbox
 public | dcim_sitegroup_id_seq                          | sequence | netbox
 public | dcim_virtualchassis                            | table    | netbox
 public | dcim_virtualchassis_id_seq                     | sequence | netbox
 public | dcim_virtualdevicecontext                      | table    | netbox
 public | dcim_virtualdevicecontext_id_seq               | sequence | netbox
 public | django_admin_log                               | table    | netbox
 public | django_admin_log_id_seq                        | sequence | netbox
 public | django_content_type                            | table    | netbox
 public | django_content_type_id_seq                     | sequence | netbox
 public | django_migrations                              | table    | netbox
 public | django_migrations_id_seq                       | sequence | netbox
 public | django_session                                 | table    | netbox
 public | extras_bookmark                                | table    | netbox
 public | extras_bookmark_id_seq                         | sequence | netbox
 public | extras_branch                                  | table    | netbox
 public | extras_branch_id_seq                           | sequence | netbox
 public | extras_cachedvalue                             | table    | netbox
 public | extras_configcontext                           | table    | netbox
 public | extras_configcontext_cluster_groups            | table    | netbox
 public | extras_configcontext_cluster_groups_id_seq     | sequence | netbox
 public | extras_configcontext_cluster_types             | table    | netbox
 public | extras_configcontext_cluster_types_id_seq      | sequence | netbox
 public | extras_configcontext_clusters                  | table    | netbox
 public | extras_configcontext_clusters_id_seq           | sequence | netbox
 public | extras_configcontext_device_types              | table    | netbox
 public | extras_configcontext_device_types_id_seq       | sequence | netbox
 public | extras_configcontext_id_seq                    | sequence | netbox
 public | extras_configcontext_locations                 | table    | netbox
 public | extras_configcontext_locations_id_seq          | sequence | netbox
 public | extras_configcontext_platforms                 | table    | netbox
 public | extras_configcontext_platforms_id_seq          | sequence | netbox
 public | extras_configcontext_regions                   | table    | netbox
 public | extras_configcontext_regions_id_seq            | sequence | netbox
 public | extras_configcontext_roles                     | table    | netbox
 public | extras_configcontext_roles_id_seq              | sequence | netbox
 public | extras_configcontext_site_groups               | table    | netbox
 public | extras_configcontext_site_groups_id_seq        | sequence | netbox
 public | extras_configcontext_sites                     | table    | netbox
 public | extras_configcontext_sites_id_seq              | sequence | netbox
 public | extras_configcontext_tags                      | table    | netbox
 public | extras_configcontext_tags_id_seq               | sequence | netbox
 public | extras_configcontext_tenant_groups             | table    | netbox
 public | extras_configcontext_tenant_groups_id_seq      | sequence | netbox
 public | extras_configcontext_tenants                   | table    | netbox
 public | extras_configcontext_tenants_id_seq            | sequence | netbox
 public | extras_configrevision_id_seq                   | sequence | netbox
 public | extras_configtemplate                          | table    | netbox
 public | extras_configtemplate_id_seq                   | sequence | netbox
 public | extras_customfield                             | table    | netbox
 public | extras_customfield_content_types               | table    | netbox
 public | extras_customfield_id_seq                      | sequence | netbox
 public | extras_customfield_obj_type_id_seq             | sequence | netbox
 public | extras_customfieldchoiceset                    | table    | netbox
 public | extras_customfieldchoiceset_id_seq             | sequence | netbox
 public | extras_customlink                              | table    | netbox
 public | extras_customlink_content_types                | table    | netbox
 public | extras_customlink_content_types_id_seq         | sequence | netbox
 public | extras_customlink_id_seq                       | sequence | netbox
 public | extras_dashboard                               | table    | netbox
 public | extras_dashboard_id_seq                        | sequence | netbox
 public | extras_eventrule                               | table    | netbox
 public | extras_eventrule_content_types                 | table    | netbox
 public | extras_eventrule_content_types_id_seq          | sequence | netbox
 public | extras_eventrule_id_seq                        | sequence | netbox
 public | extras_exporttemplate                          | table    | netbox
 public | extras_exporttemplate_content_types            | table    | netbox
 public | extras_exporttemplate_content_types_id_seq     | sequence | netbox
 public | extras_exporttemplate_id_seq                   | sequence | netbox
 public | extras_imageattachment                         | table    | netbox
 public | extras_imageattachment_id_seq                  | sequence | netbox
 public | extras_journalentry                            | table    | netbox
 public | extras_journalentry_id_seq                     | sequence | netbox
 public | extras_objectchange                            | table    | netbox
 public | extras_objectchange_id_seq                     | sequence | netbox
 public | extras_savedfilter                             | table    | netbox
 public | extras_savedfilter_content_types               | table    | netbox
 public | extras_savedfilter_content_types_id_seq        | sequence | netbox
 public | extras_savedfilter_id_seq                      | sequence | netbox
 public | extras_stagedchange                            | table    | netbox
 public | extras_stagedchange_id_seq                     | sequence | netbox
 public | extras_tag                                     | table    | netbox
 public | extras_tag_id_seq                              | sequence | netbox
 public | extras_tag_object_types                        | table    | netbox
 public | extras_tag_object_types_id_seq                 | sequence | netbox
 public | extras_taggeditem                              | table    | netbox
 public | extras_taggeditem_id_seq                       | sequence | netbox
 public | extras_webhook                                 | table    | netbox
 public | extras_webhook_id_seq                          | sequence | netbox
 public | ipam_aggregate                                 | table    | netbox
 public | ipam_aggregate_id_seq                          | sequence | netbox
 public | ipam_asn                                       | table    | netbox
 public | ipam_asn_id_seq                                | sequence | netbox
 public | ipam_asnrange                                  | table    | netbox
 public | ipam_asnrange_id_seq                           | sequence | netbox
 public | ipam_fhrpgroup                                 | table    | netbox
 public | ipam_fhrpgroup_id_seq                          | sequence | netbox
 public | ipam_fhrpgroupassignment                       | table    | netbox
 public | ipam_fhrpgroupassignment_id_seq                | sequence | netbox
 public | ipam_ipaddress                                 | table    | netbox
 public | ipam_ipaddress_id_seq                          | sequence | netbox
 public | ipam_iprange                                   | table    | netbox
 public | ipam_iprange_id_seq                            | sequence | netbox
 public | ipam_l2vpn_export_targets_id_seq               | sequence | netbox
 public | ipam_l2vpn_id_seq                              | sequence | netbox
 public | ipam_l2vpn_import_targets_id_seq               | sequence | netbox
 public | ipam_l2vpntermination_id_seq                   | sequence | netbox
 public | ipam_prefix                                    | table    | netbox
 public | ipam_prefix_id_seq                             | sequence | netbox
 public | ipam_rir                                       | table    | netbox
 public | ipam_rir_id_seq                                | sequence | netbox
 public | ipam_role                                      | table    | netbox
 public | ipam_role_id_seq                               | sequence | netbox
 public | ipam_routetarget                               | table    | netbox
 public | ipam_routetarget_id_seq                        | sequence | netbox
 public | ipam_service                                   | table    | netbox
 public | ipam_service_id_seq                            | sequence | netbox
 public | ipam_service_ipaddresses                       | table    | netbox
 public | ipam_service_ipaddresses_id_seq                | sequence | netbox
 public | ipam_servicetemplate                           | table    | netbox
 public | ipam_servicetemplate_id_seq                    | sequence | netbox
 public | ipam_vlan                                      | table    | netbox
 public | ipam_vlan_id_seq                               | sequence | netbox
 public | ipam_vlangroup                                 | table    | netbox
 public | ipam_vlangroup_id_seq                          | sequence | netbox
 public | ipam_vrf                                       | table    | netbox
 public | ipam_vrf_export_targets                        | table    | netbox
 public | ipam_vrf_export_targets_id_seq                 | sequence | netbox
 public | ipam_vrf_id_seq                                | sequence | netbox
 public | ipam_vrf_import_targets                        | table    | netbox
 public | ipam_vrf_import_targets_id_seq                 | sequence | netbox
 public | nextbox_ui_plugin_savedtopology                | table    | netbox
 public | nextbox_ui_plugin_savedtopology_id_seq         | sequence | netbox
 public | secrets_secret                                 | table    | netbox
 public | secrets_secret_id_seq                          | sequence | netbox
 public | secrets_secretrole                             | table    | netbox
 public | secrets_secretrole_id_seq                      | sequence | netbox
 public | secrets_sessionkey                             | table    | netbox
 public | secrets_sessionkey_id_seq                      | sequence | netbox
 public | secrets_userkey                                | table    | netbox
 public | secrets_userkey_id_seq                         | sequence | netbox
 public | social_auth_association                        | table    | netbox
 public | social_auth_association_id_seq                 | sequence | netbox
 public | social_auth_code                               | table    | netbox
 public | social_auth_code_id_seq                        | sequence | netbox
 public | social_auth_nonce                              | table    | netbox
 public | social_auth_nonce_id_seq                       | sequence | netbox
 public | social_auth_partial                            | table    | netbox
 public | social_auth_partial_id_seq                     | sequence | netbox
 public | social_auth_usersocialauth                     | table    | netbox
 public | social_auth_usersocialauth_id_seq              | sequence | netbox
 public | taggit_tag                                     | table    | netbox
 public | taggit_tag_id_seq                              | sequence | netbox
 public | taggit_taggeditem                              | table    | netbox
 public | taggit_taggeditem_id_seq                       | sequence | netbox
 public | tenancy_contact                                | table    | netbox
 public | tenancy_contact_id_seq                         | sequence | netbox
 public | tenancy_contactassignment                      | table    | netbox
 public | tenancy_contactassignment_id_seq               | sequence | netbox
 public | tenancy_contactgroup                           | table    | netbox
 public | tenancy_contactgroup_id_seq                    | sequence | netbox
 public | tenancy_contactrole                            | table    | netbox
 public | tenancy_contactrole_id_seq                     | sequence | netbox
 public | tenancy_tenant                                 | table    | netbox
 public | tenancy_tenant_id_seq                          | sequence | netbox
 public | tenancy_tenantgroup                            | table    | netbox
 public | tenancy_tenantgroup_id_seq                     | sequence | netbox
 public | users_objectpermission                         | table    | netbox
 public | users_objectpermission_groups                  | table    | netbox
 public | users_objectpermission_groups_id_seq           | sequence | netbox
 public | users_objectpermission_id_seq                  | sequence | netbox
 public | users_objectpermission_object_types            | table    | netbox
 public | users_objectpermission_object_types_id_seq     | sequence | netbox
 public | users_objectpermission_users                   | table    | netbox
 public | users_objectpermission_users_id_seq            | sequence | netbox
 public | users_token                                    | table    | netbox
 public | users_token_id_seq                             | sequence | netbox
 public | users_userconfig                               | table    | netbox
 public | users_userconfig_id_seq                        | sequence | netbox
 public | virtualization_cluster                         | table    | netbox
 public | virtualization_cluster_id_seq                  | sequence | netbox
 public | virtualization_clustergroup                    | table    | netbox
 public | virtualization_clustergroup_id_seq             | sequence | netbox
 public | virtualization_clustertype                     | table    | netbox
 public | virtualization_clustertype_id_seq              | sequence | netbox
 public | virtualization_virtualdisk                     | table    | netbox
 public | virtualization_virtualdisk_id_seq              | sequence | netbox
 public | virtualization_virtualmachine                  | table    | netbox
 public | virtualization_virtualmachine_id_seq           | sequence | netbox
 public | virtualization_vminterface                     | table    | netbox
 public | virtualization_vminterface_id_seq              | sequence | netbox
 public | virtualization_vminterface_tagged_vlans        | table    | netbox
 public | virtualization_vminterface_tagged_vlans_id_seq | sequence | netbox
 public | vpn_ikepolicy                                  | table    | netbox
 public | vpn_ikepolicy_id_seq                           | sequence | netbox
 public | vpn_ikepolicy_proposals                        | table    | netbox
 public | vpn_ikepolicy_proposals_id_seq                 | sequence | netbox
 public | vpn_ikeproposal                                | table    | netbox
 public | vpn_ikeproposal_id_seq                         | sequence | netbox
 public | vpn_ipsecpolicy                                | table    | netbox
 public | vpn_ipsecpolicy_id_seq                         | sequence | netbox
 public | vpn_ipsecpolicy_proposals                      | table    | netbox
 public | vpn_ipsecpolicy_proposals_id_seq               | sequence | netbox
 public | vpn_ipsecprofile                               | table    | netbox
 public | vpn_ipsecprofile_id_seq                        | sequence | netbox
 public | vpn_ipsecproposal                              | table    | netbox
 public | vpn_ipsecproposal_id_seq                       | sequence | netbox
 public | vpn_l2vpn                                      | table    | netbox
 public | vpn_l2vpn_export_targets                       | table    | netbox
 public | vpn_l2vpn_import_targets                       | table    | netbox
 public | vpn_l2vpntermination                           | table    | netbox
 public | vpn_tunnel                                     | table    | netbox
 public | vpn_tunnel_id_seq                              | sequence | netbox
 public | vpn_tunnelgroup                                | table    | netbox
 public | vpn_tunnelgroup_id_seq                         | sequence | netbox
 public | vpn_tunneltermination                          | table    | netbox
 public | vpn_tunneltermination_id_seq                   | sequence | netbox
 public | wireless_wirelesslan                           | table    | netbox
 public | wireless_wirelesslan_id_seq                    | sequence | netbox
 public | wireless_wirelesslangroup                      | table    | netbox
 public | wireless_wirelesslangroup_id_seq               | sequence | netbox
 public | wireless_wirelesslink                          | table    | netbox
 public | wireless_wirelesslink_id_seq                   | sequence | netbox
(347 rows)

@candlerb
Copy link
Contributor

candlerb commented Apr 3, 2024 •
edited
Loading

Thanks, I beleive I now installed all django packages.

You should not have to install anything.

You should allow Netbox to create a virtualenv and populate it; this is all done using the ./upgrade.sh script. If you have a problem with running upgrade.sh, please show it.

If you find you have to install packages manually then there is definitely some other problem: you need to stop and fix that problem, before continuing.

EDIT: if you did this (as you say):

cd /opt/netbox/netbox
python3 manage.py dbshell

and you got that error, it means you probably hadn't activated the virtualenv before running the command. You definitely should not install django packages globally to workaround that issue.

Actually, there's a safer way to run it:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py dbshell

This will activate the virtualenv for you (regardless of whether it's currently activated or not)

@jeremystretch jeremystretch added the beta Concerns a bug/feature in a beta release label Apr 3, 2024
@opericgithub
Copy link
Author

opericgithub commented Apr 3, 2024 •
edited
Loading

Actually, there's a safer way to run it:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py dbshell

This will activate the virtualenv for you (regardless of whether it's currently activated or not)

Yes, after I run this command, I can immediately list all tables with "\d" command, as Jeremy asked.

You should allow Netbox to create a virtualenv and populate it; this is all done using the ./upgrade.sh script. If you have a problem with running upgrade.sh, please show it.

I can run upgrade.sh without problems:

cd /opt/netbox
sudo ./upgrade.sh

...
Removing old virtual environment...
Creating a new virtual environment at /opt/netbox/venv...
Updating pip (pip install --upgrade pip)...
Requirement already satisfied: pip in ./venv/lib/python3.10/site-packages (22.0.2)
...  
Installing local dependencies (pip install -r local_requirements.txt)...
Collecting django-auth-ldap (from -r local_requirements.txt (line 1))
  Using cached django_auth_ldap-4.7.0-py3-none-any.whl.metadata (7.2 kB)
Requirement already satisfied: Django>=3.2 in ./venv/lib/python3.10/site-packages (from django-auth-ldap->-r local_requirements.txt (line 1)) (4.2.11)
...
  wireless.wirelesslink... No objects found.
Completed. Total entries: 36437
Removing expired user sessions (python3 netbox/manage.py clearsessions)...
Upgrade complete! Don't forget to restart the NetBox services:

Even after successful execution of this script on the stable version 3.7.4, I am not able to upgrade to beta.

I mean, first I do this:
sudo ./upgrade.sh
Everything is OK.
Then this:
sudo git pull origin v4.0-beta1
then again this:
sudo ./upgrade.sh
and I get stucked with the message from the first post (relation "extras_customfield_content_types_id_seq" does not exist).

And really, when one looks at the tables I posted above, there is a missing row with sequence regarding the custom field content type.
As far as I understand there should be sequence for every table (vpntunnel, vpntunnel_id_seq, dcim_site, dcim_site_id_seq etc).

@tomasz-c
Copy link

tomasz-c commented Apr 4, 2024 •
edited
Loading

I have this problem too.
./upgrade.sh on version 3.7.4 goes correctly, but there is no extras_customfield_content_types_id_seq in the database

netbox=> \d extras_customfield_content_types_id_seq
Did not find any relation named "extras_customfield_content_types_id_seq".

Trying to force migration to extras on version 3.7.4 reports no changes:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py migrate extras
Operations to perform:
  Apply all migrations: extras
Running migrations:
  No migrations to apply.

So I have the same error when trying to update to v4.0-beta1.
I have been updating the netbox for a long time, maybe this table was created only for new installations and the update omitted it?

@candlerb
Copy link
Contributor

candlerb commented Apr 4, 2024

I mean, first I do this: sudo ./upgrade.sh Everything is OK. Then this: sudo git pull origin v4.0-beta1 then again this: sudo ./upgrade.sh and I get stucked with the message from the first post (relation "extras_customfield_content_types_id_seq" does not exist).

It's broken for me too; some of the sequences and indexes in my system (and your system, and some others it seems) have different names to the ones that the migration scripts are expecting, and some constraints are missing.

I was able to make the migrations complete with some temporary patches:
#15601 (reply in thread)

But there's some underlying issue about how machines which have a long history of upgrades having different constraint names than those currently expected.

@opericgithub
Copy link
Author

opericgithub commented Apr 4, 2024 •
edited
Loading

Regarding the long history of upgrades, which other users have also discussed, it might be worth mentioning that we initially had version 2.8, then progressed subsequently to 2.11, followed by 3.0, and subsequently to each minor version (3.1, 3.2, etc.).

At one moment we did clean install of 3.x (don't remember the particular version) and did migrate all data from the same version from another server.

@PaulR282

This comment was marked as off-topic.

Sign in to view
@jhammond-git
Copy link

To add to the data set, I had the same issues as documented in #15601 (reply in thread).

The VM I installed onto is a brand new ubuntu 22.04 machine with a fresh install of 3.7.4, but the database that was imported was upgraded over the years from 1.9.5, or at least, that's the first version I know we had on our production box.

@ingvaldlorentzen

This comment was marked as off-topic.

Sign in to view
@v0tti

This comment was marked as off-topic.

Sign in to view
@v0tti
Copy link
Contributor

v0tti commented Apr 5, 2024 

FileNotFoundError: [Errno 2] No such file or directory: '/opt/netbox-4.0-beta1/netbox/reports/devices.py'

After fixing this by manually creating the mentioned files I am also running into the django.db.utils.ProgrammingError: relation "extras_customfield_content_types_id_seq" does not exist problem.

@jeremystretch
Copy link
Member

I've hidden several comments in the thread which are not relevant to the reported behavior above. Please remember to open a separate bug report for unrelated issues.

@jeremystretch jeremystretch changed the title unable to upgrade to 4.0.x ProgrammingError exception when upgrading to v4.0 Apr 8, 2024
@jeremystretch
Copy link
Member

Could someone who is experiencing this issue please post the list of tables which exist in their database prior to attempting the upgrade to v4.0? This can be done by running manage.py dbshell and issuing the \d console command.)

I'm specifically interested in determining how extras_customfield_content_types_id_seq is named in older databases.

@a084ed22
Copy link

a084ed22 commented Apr 8, 2024

I have a setup from the 2.11 era, gradually updated, currently on 3.7.5. I don't have a sequence for extras_customfield_content_types_id_seq.

@tobiasge
Copy link
Member

tobiasge commented Apr 8, 2024 •
edited
Loading

Could someone who is experiencing this issue please post the list of tables which exist in their database prior to attempting the upgrade to v4.0? This can be done by running manage.py dbshell and issuing the \d console command.)

I'm specifically interested in determining how extras_customfield_content_types_id_seq is named in older databases.

This is the list from our Netbox 3.7.5 (first installation on this DB was from 2.5 or 2.4):

netbox-prod=# \d
                                List of relations
 Schema |                      Name                      |   Type   |    Owner
--------+------------------------------------------------+----------+-------------
 public | auth_group                                     | table    | netbox-prod
 public | auth_group_id_seq                              | sequence | netbox-prod
 public | auth_group_permissions                         | table    | netbox-prod
 public | auth_group_permissions_id_seq                  | sequence | netbox-prod
 public | auth_permission                                | table    | netbox-prod
 public | auth_permission_id_seq                         | sequence | netbox-prod
 public | auth_user                                      | table    | netbox-prod
 public | auth_user_groups                               | table    | netbox-prod
 public | auth_user_groups_id_seq                        | sequence | netbox-prod
 public | auth_user_id_seq                               | sequence | netbox-prod
 public | auth_user_user_permissions                     | table    | netbox-prod
 public | auth_user_user_permissions_id_seq              | sequence | netbox-prod
 public | circuits_circuit                               | table    | netbox-prod
 public | circuits_circuit_id_seq                        | sequence | netbox-prod
 public | circuits_circuittermination                    | table    | netbox-prod
 public | circuits_circuittermination_id_seq             | sequence | netbox-prod
 public | circuits_circuittype                           | table    | netbox-prod
 public | circuits_circuittype_id_seq                    | sequence | netbox-prod
 public | circuits_provider                              | table    | netbox-prod
 public | circuits_provider_asns                         | table    | netbox-prod
 public | circuits_provider_asns_id_seq                  | sequence | netbox-prod
 public | circuits_provider_id_seq                       | sequence | netbox-prod
 public | circuits_provideraccount                       | table    | netbox-prod
 public | circuits_provideraccount_id_seq                | sequence | netbox-prod
 public | circuits_providernetwork                       | table    | netbox-prod
 public | circuits_providernetwork_id_seq                | sequence | netbox-prod
 public | core_autosyncrecord                            | table    | netbox-prod
 public | core_autosyncrecord_id_seq                     | sequence | netbox-prod
 public | core_configrevision                            | table    | netbox-prod
 public | core_datafile                                  | table    | netbox-prod
 public | core_datafile_id_seq                           | sequence | netbox-prod
 public | core_datasource                                | table    | netbox-prod
 public | core_datasource_id_seq                         | sequence | netbox-prod
 public | core_job                                       | table    | netbox-prod
 public | core_job_id_seq                                | sequence | netbox-prod
 public | core_managedfile                               | table    | netbox-prod
 public | core_managedfile_id_seq                        | sequence | netbox-prod
 public | dcim_cable                                     | table    | netbox-prod
 public | dcim_cable_id_seq                              | sequence | netbox-prod
 public | dcim_cablepath                                 | table    | netbox-prod
 public | dcim_cablepath_id_seq                          | sequence | netbox-prod
 public | dcim_cabletermination                          | table    | netbox-prod
 public | dcim_cabletermination_id_seq                   | sequence | netbox-prod
 public | dcim_consoleport                               | table    | netbox-prod
 public | dcim_consoleport_id_seq                        | sequence | netbox-prod
 public | dcim_consoleporttemplate                       | table    | netbox-prod
 public | dcim_consoleporttemplate_id_seq                | sequence | netbox-prod
 public | dcim_consoleserverport                         | table    | netbox-prod
 public | dcim_consoleserverport_id_seq                  | sequence | netbox-prod
 public | dcim_consoleserverporttemplate                 | table    | netbox-prod
 public | dcim_consoleserverporttemplate_id_seq          | sequence | netbox-prod
 public | dcim_device                                    | table    | netbox-prod
 public | dcim_device_id_seq                             | sequence | netbox-prod
 public | dcim_devicebay                                 | table    | netbox-prod
 public | dcim_devicebay_id_seq                          | sequence | netbox-prod
 public | dcim_devicebaytemplate                         | table    | netbox-prod
 public | dcim_devicebaytemplate_id_seq                  | sequence | netbox-prod
 public | dcim_devicerole                                | table    | netbox-prod
 public | dcim_devicerole_id_seq                         | sequence | netbox-prod
 public | dcim_devicetype                                | table    | netbox-prod
 public | dcim_devicetype_id_seq                         | sequence | netbox-prod
 public | dcim_frontport                                 | table    | netbox-prod
 public | dcim_frontport_id_seq                          | sequence | netbox-prod
 public | dcim_frontporttemplate                         | table    | netbox-prod
 public | dcim_frontporttemplate_id_seq                  | sequence | netbox-prod
 public | dcim_interface                                 | table    | netbox-prod
 public | dcim_interface_id_seq                          | sequence | netbox-prod
 public | dcim_interface_tagged_vlans                    | table    | netbox-prod
 public | dcim_interface_tagged_vlans_id_seq             | sequence | netbox-prod
 public | dcim_interface_vdcs                            | table    | netbox-prod
 public | dcim_interface_vdcs_id_seq                     | sequence | netbox-prod
 public | dcim_interface_wireless_lans                   | table    | netbox-prod
 public | dcim_interface_wireless_lans_id_seq            | sequence | netbox-prod
 public | dcim_interfacetemplate                         | table    | netbox-prod
 public | dcim_interfacetemplate_id_seq                  | sequence | netbox-prod
 public | dcim_inventoryitem                             | table    | netbox-prod
 public | dcim_inventoryitem_id_seq                      | sequence | netbox-prod
 public | dcim_inventoryitemrole                         | table    | netbox-prod
 public | dcim_inventoryitemrole_id_seq                  | sequence | netbox-prod
 public | dcim_inventoryitemtemplate                     | table    | netbox-prod
 public | dcim_inventoryitemtemplate_id_seq              | sequence | netbox-prod
 public | dcim_location                                  | table    | netbox-prod
 public | dcim_manufacturer                              | table    | netbox-prod
 public | dcim_manufacturer_id_seq                       | sequence | netbox-prod
 public | dcim_module                                    | table    | netbox-prod
 public | dcim_module_id_seq                             | sequence | netbox-prod
 public | dcim_module_id_seq1                            | sequence | netbox-prod
 public | dcim_modulebay                                 | table    | netbox-prod
 public | dcim_modulebay_id_seq                          | sequence | netbox-prod
 public | dcim_modulebaytemplate                         | table    | netbox-prod
 public | dcim_modulebaytemplate_id_seq                  | sequence | netbox-prod
 public | dcim_moduletype                                | table    | netbox-prod
 public | dcim_moduletype_id_seq                         | sequence | netbox-prod
 public | dcim_platform                                  | table    | netbox-prod
 public | dcim_platform_id_seq                           | sequence | netbox-prod
 public | dcim_powerfeed                                 | table    | netbox-prod
 public | dcim_powerfeed_id_seq                          | sequence | netbox-prod
 public | dcim_poweroutlet                               | table    | netbox-prod
 public | dcim_poweroutlet_id_seq                        | sequence | netbox-prod
 public | dcim_poweroutlettemplate                       | table    | netbox-prod
 public | dcim_poweroutlettemplate_id_seq                | sequence | netbox-prod
 public | dcim_powerpanel                                | table    | netbox-prod
 public | dcim_powerpanel_id_seq                         | sequence | netbox-prod
 public | dcim_powerport                                 | table    | netbox-prod
 public | dcim_powerport_id_seq                          | sequence | netbox-prod
 public | dcim_powerporttemplate                         | table    | netbox-prod
 public | dcim_powerporttemplate_id_seq                  | sequence | netbox-prod
 public | dcim_rack                                      | table    | netbox-prod
 public | dcim_rack_id_seq                               | sequence | netbox-prod
 public | dcim_rackgroup_id_seq                          | sequence | netbox-prod
 public | dcim_rackreservation                           | table    | netbox-prod
 public | dcim_rackreservation_id_seq                    | sequence | netbox-prod
 public | dcim_rackrole                                  | table    | netbox-prod
 public | dcim_rackrole_id_seq                           | sequence | netbox-prod
 public | dcim_rearport                                  | table    | netbox-prod
 public | dcim_rearport_id_seq                           | sequence | netbox-prod
 public | dcim_rearporttemplate                          | table    | netbox-prod
 public | dcim_rearporttemplate_id_seq                   | sequence | netbox-prod
 public | dcim_region                                    | table    | netbox-prod
 public | dcim_region_id_seq                             | sequence | netbox-prod
 public | dcim_site                                      | table    | netbox-prod
 public | dcim_site_asns                                 | table    | netbox-prod
 public | dcim_site_asns_id_seq                          | sequence | netbox-prod
 public | dcim_site_id_seq                               | sequence | netbox-prod
 public | dcim_sitegroup                                 | table    | netbox-prod
 public | dcim_sitegroup_id_seq                          | sequence | netbox-prod
 public | dcim_virtualchassis                            | table    | netbox-prod
 public | dcim_virtualchassis_id_seq                     | sequence | netbox-prod
 public | dcim_virtualdevicecontext                      | table    | netbox-prod
 public | dcim_virtualdevicecontext_id_seq               | sequence | netbox-prod
 public | django_admin_log                               | table    | netbox-prod
 public | django_admin_log_id_seq                        | sequence | netbox-prod
 public | django_content_type                            | table    | netbox-prod
 public | django_content_type_id_seq                     | sequence | netbox-prod
 public | django_migrations                              | table    | netbox-prod
 public | django_migrations_id_seq                       | sequence | netbox-prod
 public | django_session                                 | table    | netbox-prod
 public | extras_bookmark                                | table    | netbox-prod
 public | extras_bookmark_id_seq                         | sequence | netbox-prod
 public | extras_branch                                  | table    | netbox-prod
 public | extras_branch_id_seq                           | sequence | netbox-prod
 public | extras_cachedvalue                             | table    | netbox-prod
 public | extras_configcontext                           | table    | netbox-prod
 public | extras_configcontext_cluster_groups            | table    | netbox-prod
 public | extras_configcontext_cluster_groups_id_seq     | sequence | netbox-prod
 public | extras_configcontext_cluster_types             | table    | netbox-prod
 public | extras_configcontext_cluster_types_id_seq      | sequence | netbox-prod
 public | extras_configcontext_clusters                  | table    | netbox-prod
 public | extras_configcontext_clusters_id_seq           | sequence | netbox-prod
 public | extras_configcontext_device_types              | table    | netbox-prod
 public | extras_configcontext_device_types_id_seq       | sequence | netbox-prod
 public | extras_configcontext_id_seq                    | sequence | netbox-prod
 public | extras_configcontext_locations                 | table    | netbox-prod
 public | extras_configcontext_locations_id_seq          | sequence | netbox-prod
 public | extras_configcontext_platforms                 | table    | netbox-prod
 public | extras_configcontext_platforms_id_seq          | sequence | netbox-prod
 public | extras_configcontext_regions                   | table    | netbox-prod
 public | extras_configcontext_regions_id_seq            | sequence | netbox-prod
 public | extras_configcontext_roles                     | table    | netbox-prod
 public | extras_configcontext_roles_id_seq              | sequence | netbox-prod
 public | extras_configcontext_site_groups               | table    | netbox-prod
 public | extras_configcontext_site_groups_id_seq        | sequence | netbox-prod
 public | extras_configcontext_sites                     | table    | netbox-prod
 public | extras_configcontext_sites_id_seq              | sequence | netbox-prod
 public | extras_configcontext_tags                      | table    | netbox-prod
 public | extras_configcontext_tags_id_seq               | sequence | netbox-prod
 public | extras_configcontext_tenant_groups             | table    | netbox-prod
 public | extras_configcontext_tenant_groups_id_seq      | sequence | netbox-prod
 public | extras_configcontext_tenants                   | table    | netbox-prod
 public | extras_configcontext_tenants_id_seq            | sequence | netbox-prod
 public | extras_configrevision_id_seq                   | sequence | netbox-prod
 public | extras_configtemplate                          | table    | netbox-prod
 public | extras_configtemplate_id_seq                   | sequence | netbox-prod
 public | extras_customfield                             | table    | netbox-prod
 public | extras_customfield_content_types               | table    | netbox-prod
 public | extras_customfield_id_seq                      | sequence | netbox-prod
 public | extras_customfield_obj_type_id_seq             | sequence | netbox-prod
 public | extras_customfieldchoiceset                    | table    | netbox-prod
 public | extras_customfieldchoiceset_id_seq             | sequence | netbox-prod
 public | extras_customlink                              | table    | netbox-prod
 public | extras_customlink_content_types                | table    | netbox-prod
 public | extras_customlink_content_types_id_seq         | sequence | netbox-prod
 public | extras_customlink_id_seq                       | sequence | netbox-prod
 public | extras_dashboard                               | table    | netbox-prod
 public | extras_dashboard_id_seq                        | sequence | netbox-prod
 public | extras_eventrule                               | table    | netbox-prod
 public | extras_eventrule_content_types                 | table    | netbox-prod
 public | extras_eventrule_content_types_id_seq          | sequence | netbox-prod
 public | extras_eventrule_id_seq                        | sequence | netbox-prod
 public | extras_exporttemplate                          | table    | netbox-prod
 public | extras_exporttemplate_content_types            | table    | netbox-prod
 public | extras_exporttemplate_content_types_id_seq     | sequence | netbox-prod
 public | extras_exporttemplate_id_seq                   | sequence | netbox-prod
 public | extras_imageattachment                         | table    | netbox-prod
 public | extras_imageattachment_id_seq                  | sequence | netbox-prod
 public | extras_journalentry                            | table    | netbox-prod
 public | extras_journalentry_id_seq                     | sequence | netbox-prod
 public | extras_objectchange                            | table    | netbox-prod
 public | extras_objectchange_id_seq                     | sequence | netbox-prod
 public | extras_savedfilter                             | table    | netbox-prod
 public | extras_savedfilter_content_types               | table    | netbox-prod
 public | extras_savedfilter_content_types_id_seq        | sequence | netbox-prod
 public | extras_savedfilter_id_seq                      | sequence | netbox-prod
 public | extras_stagedchange                            | table    | netbox-prod
 public | extras_stagedchange_id_seq                     | sequence | netbox-prod
 public | extras_tag                                     | table    | netbox-prod
 public | extras_tag_id_seq                              | sequence | netbox-prod
 public | extras_tag_object_types                        | table    | netbox-prod
 public | extras_tag_object_types_id_seq                 | sequence | netbox-prod
 public | extras_taggeditem                              | table    | netbox-prod
 public | extras_taggeditem_id_seq                       | sequence | netbox-prod
 public | extras_webhook                                 | table    | netbox-prod
 public | extras_webhook_id_seq                          | sequence | netbox-prod
 public | ipam_aggregate                                 | table    | netbox-prod
 public | ipam_aggregate_id_seq                          | sequence | netbox-prod
 public | ipam_asn                                       | table    | netbox-prod
 public | ipam_asn_id_seq                                | sequence | netbox-prod
 public | ipam_asnrange                                  | table    | netbox-prod
 public | ipam_asnrange_id_seq                           | sequence | netbox-prod
 public | ipam_fhrpgroup                                 | table    | netbox-prod
 public | ipam_fhrpgroup_id_seq                          | sequence | netbox-prod
 public | ipam_fhrpgroupassignment                       | table    | netbox-prod
 public | ipam_fhrpgroupassignment_id_seq                | sequence | netbox-prod
 public | ipam_ipaddress                                 | table    | netbox-prod
 public | ipam_ipaddress_id_seq                          | sequence | netbox-prod
 public | ipam_iprange                                   | table    | netbox-prod
 public | ipam_iprange_id_seq                            | sequence | netbox-prod
 public | ipam_l2vpn_export_targets_id_seq               | sequence | netbox-prod
 public | ipam_l2vpn_id_seq                              | sequence | netbox-prod
 public | ipam_l2vpn_import_targets_id_seq               | sequence | netbox-prod
 public | ipam_l2vpntermination_id_seq                   | sequence | netbox-prod
 public | ipam_prefix                                    | table    | netbox-prod
 public | ipam_prefix_id_seq                             | sequence | netbox-prod
 public | ipam_rir                                       | table    | netbox-prod
 public | ipam_rir_id_seq                                | sequence | netbox-prod
 public | ipam_role                                      | table    | netbox-prod
 public | ipam_role_id_seq                               | sequence | netbox-prod
 public | ipam_routetarget                               | table    | netbox-prod
 public | ipam_routetarget_id_seq                        | sequence | netbox-prod
 public | ipam_service                                   | table    | netbox-prod
 public | ipam_service_id_seq                            | sequence | netbox-prod
 public | ipam_service_ipaddresses                       | table    | netbox-prod
 public | ipam_service_ipaddresses_id_seq                | sequence | netbox-prod
 public | ipam_servicetemplate                           | table    | netbox-prod
 public | ipam_servicetemplate_id_seq                    | sequence | netbox-prod
 public | ipam_vlan                                      | table    | netbox-prod
 public | ipam_vlan_id_seq                               | sequence | netbox-prod
 public | ipam_vlangroup                                 | table    | netbox-prod
 public | ipam_vlangroup_id_seq                          | sequence | netbox-prod
 public | ipam_vrf                                       | table    | netbox-prod
 public | ipam_vrf_export_targets                        | table    | netbox-prod
 public | ipam_vrf_export_targets_id_seq                 | sequence | netbox-prod
 public | ipam_vrf_id_seq                                | sequence | netbox-prod
 public | ipam_vrf_import_targets                        | table    | netbox-prod
 public | ipam_vrf_import_targets_id_seq                 | sequence | netbox-prod
 public | secrets_secret                                 | table    | netbox-prod
 public | secrets_secret_id_seq                          | sequence | netbox-prod
 public | secrets_secretrole                             | table    | netbox-prod
 public | secrets_secretrole_id_seq                      | sequence | netbox-prod
 public | secrets_sessionkey                             | table    | netbox-prod
 public | secrets_sessionkey_id_seq                      | sequence | netbox-prod
 public | secrets_userkey                                | table    | netbox-prod
 public | secrets_userkey_id_seq                         | sequence | netbox-prod
 public | social_auth_association                        | table    | netbox-prod
 public | social_auth_association_id_seq                 | sequence | netbox-prod
 public | social_auth_code                               | table    | netbox-prod
 public | social_auth_code_id_seq                        | sequence | netbox-prod
 public | social_auth_nonce                              | table    | netbox-prod
 public | social_auth_nonce_id_seq                       | sequence | netbox-prod
 public | social_auth_partial                            | table    | netbox-prod
 public | social_auth_partial_id_seq                     | sequence | netbox-prod
 public | social_auth_usersocialauth                     | table    | netbox-prod
 public | social_auth_usersocialauth_id_seq              | sequence | netbox-prod
 public | taggit_tag                                     | table    | netbox-prod
 public | taggit_tag_id_seq                              | sequence | netbox-prod
 public | taggit_taggeditem                              | table    | netbox-prod
 public | taggit_taggeditem_id_seq                       | sequence | netbox-prod
 public | tenancy_contact                                | table    | netbox-prod
 public | tenancy_contact_id_seq                         | sequence | netbox-prod
 public | tenancy_contactassignment                      | table    | netbox-prod
 public | tenancy_contactassignment_id_seq               | sequence | netbox-prod
 public | tenancy_contactgroup                           | table    | netbox-prod
 public | tenancy_contactgroup_id_seq                    | sequence | netbox-prod
 public | tenancy_contactrole                            | table    | netbox-prod
 public | tenancy_contactrole_id_seq                     | sequence | netbox-prod
 public | tenancy_tenant                                 | table    | netbox-prod
 public | tenancy_tenant_id_seq                          | sequence | netbox-prod
 public | tenancy_tenantgroup                            | table    | netbox-prod
 public | tenancy_tenantgroup_id_seq                     | sequence | netbox-prod
 public | users_objectpermission                         | table    | netbox-prod
 public | users_objectpermission_groups                  | table    | netbox-prod
 public | users_objectpermission_groups_id_seq           | sequence | netbox-prod
 public | users_objectpermission_id_seq                  | sequence | netbox-prod
 public | users_objectpermission_object_types            | table    | netbox-prod
 public | users_objectpermission_object_types_id_seq     | sequence | netbox-prod
 public | users_objectpermission_users                   | table    | netbox-prod
 public | users_objectpermission_users_id_seq            | sequence | netbox-prod
 public | users_token                                    | table    | netbox-prod
 public | users_token_id_seq                             | sequence | netbox-prod
 public | users_userconfig                               | table    | netbox-prod
 public | users_userconfig_id_seq                        | sequence | netbox-prod
 public | virtualization_cluster                         | table    | netbox-prod
 public | virtualization_cluster_id_seq                  | sequence | netbox-prod
 public | virtualization_clustergroup                    | table    | netbox-prod
 public | virtualization_clustergroup_id_seq             | sequence | netbox-prod
 public | virtualization_clustertype                     | table    | netbox-prod
 public | virtualization_clustertype_id_seq              | sequence | netbox-prod
 public | virtualization_virtualdisk                     | table    | netbox-prod
 public | virtualization_virtualdisk_id_seq              | sequence | netbox-prod
 public | virtualization_virtualmachine                  | table    | netbox-prod
 public | virtualization_virtualmachine_id_seq           | sequence | netbox-prod
 public | virtualization_vminterface                     | table    | netbox-prod
 public | virtualization_vminterface_id_seq              | sequence | netbox-prod
 public | virtualization_vminterface_tagged_vlans        | table    | netbox-prod
 public | virtualization_vminterface_tagged_vlans_id_seq | sequence | netbox-prod
 public | vpn_ikepolicy                                  | table    | netbox-prod
 public | vpn_ikepolicy_id_seq                           | sequence | netbox-prod
 public | vpn_ikepolicy_proposals                        | table    | netbox-prod
 public | vpn_ikepolicy_proposals_id_seq                 | sequence | netbox-prod
 public | vpn_ikeproposal                                | table    | netbox-prod
 public | vpn_ikeproposal_id_seq                         | sequence | netbox-prod
 public | vpn_ipsecpolicy                                | table    | netbox-prod
 public | vpn_ipsecpolicy_id_seq                         | sequence | netbox-prod
 public | vpn_ipsecpolicy_proposals                      | table    | netbox-prod
 public | vpn_ipsecpolicy_proposals_id_seq               | sequence | netbox-prod
 public | vpn_ipsecprofile                               | table    | netbox-prod
 public | vpn_ipsecprofile_id_seq                        | sequence | netbox-prod
 public | vpn_ipsecproposal                              | table    | netbox-prod
 public | vpn_ipsecproposal_id_seq                       | sequence | netbox-prod
 public | vpn_l2vpn                                      | table    | netbox-prod
 public | vpn_l2vpn_export_targets                       | table    | netbox-prod
 public | vpn_l2vpn_import_targets                       | table    | netbox-prod
 public | vpn_l2vpntermination                           | table    | netbox-prod
 public | vpn_tunnel                                     | table    | netbox-prod
 public | vpn_tunnel_id_seq                              | sequence | netbox-prod
 public | vpn_tunnelgroup                                | table    | netbox-prod
 public | vpn_tunnelgroup_id_seq                         | sequence | netbox-prod
 public | vpn_tunneltermination                          | table    | netbox-prod
 public | vpn_tunneltermination_id_seq                   | sequence | netbox-prod
 public | wireless_wirelesslan                           | table    | netbox-prod
 public | wireless_wirelesslan_id_seq                    | sequence | netbox-prod
 public | wireless_wirelesslangroup                      | table    | netbox-prod
 public | wireless_wirelesslangroup_id_seq               | sequence | netbox-prod
 public | wireless_wirelesslink                          | table    | netbox-prod
 public | wireless_wirelesslink_id_seq                   | sequence | netbox-prod
(345 rows)

I think it's named extras_customfield_obj_type_id_seq:

netbox-dev=# \d extras_customfield_content_types
                              Table "public.extras_customfield_content_types"
     Column     |  Type   | Collation | Nullable |                         Default
----------------+---------+-----------+----------+---------------------------------------------------------
 id             | integer |           | not null | nextval('extras_customfield_obj_type_id_seq'::regclass)

@jeremystretch
Copy link
Member

I think it's named extras_customfield_obj_type_id_seq

Yep, that seems to be the issue. This was from the old obj_type field on the CustomField model from before custom fields were completely reimplemented way back in NetBox v2.10. Unfortunately the sequence was never renamed accordingly for upgraded deployments.

We can tweak the v4.0 migration to check for either sequence name and rename it accordingly.

@jeremystretch jeremystretch added status: accepted This issue has been accepted for implementation and removed status: under review Further discussion is needed to determine this issue's scope and/or implementation labels Apr 8, 2024
@jeremystretch jeremystretch added the severity: medium Results in substantial degraded or broken functionality for specfic workflows label Apr 8, 2024
jeremystretch added a commit that referenced this issue Apr 8, 2024
@jeremystretch
Fixes #15605: Account for older sequence name in migration
27ca053
@jeremystretch jeremystretch mentioned this issue Apr 8, 2024
Merged
@jeremystretch
Copy link
Member

@opericgithub @a084ed22 @tobiasge (and anyone else experiencing this issue), could you try checking out the 15605-migrate-old-sequence branch and attempting the migration again? Should be fixed.

@tobiasge
Copy link
Member

tobiasge commented Apr 8, 2024

The content types are now working for out database.

Applying extras.0111_rename_content_types... OK

But now I'm seeing a new error:

Applying users.0006_custom_group_model...
psycopg.errors.ForeignKeyViolation: update or delete on table "auth_group" violates foreign key constraint "auth_group_permissions_group_id_b120cbf9_fk_auth_group_id" on table "auth_group_permissions"
DETAIL:  Key (id)=(10) is still referenced from table "auth_group_permissions".

@a084ed22
Copy link

a084ed22 commented Apr 8, 2024 •
edited
Loading

I can confirm that the fix allows the specific migration to work, however I'm later stuck on this:

Running migrations:
  Applying users.0008_flip_objectpermission_assignments...Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
psycopg.errors.UndefinedObject: constraint "users_objectpermissi_objectpermission_id_2f7cc117_fk_users_obj" for table "users_group_object_permissions" does not exist

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/operations/special.py", line 37, in database_forwards
    database_operation.database_forwards(
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/operations/special.py", line 106, in database_forwards
    self._run_sql(schema_editor, self.sql)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/migrations/operations/special.py", line 133, in _run_sql
    schema_editor.execute(statement, params=None)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 45, in execute
    return super().execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 201, in execute
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in _execute
    return self.cursor.execute(sql)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/psycopg/cursor.py", line 732, in execute
    raise ex.with_traceback(None)
django.db.utils.ProgrammingError: constraint "users_objectpermissi_objectpermission_id_2f7cc117_fk_users_obj" for table "users_group_object_permissions" does not exist

@opericgithub
Copy link
Author

@opericgithub @a084ed22 @tobiasge (and anyone else experiencing this issue), could you try checking out the 15605-migrate-old-sequence branch and attempting the migration again? Should be fixed.

Sure. I tried checking out to that branch, upgrade completes without errors, but I stay on the same version (3.7.5).
Maybe I'm doing something wrong?

cd /opt/netbox
sudo git pull origin 15605-migrate-old-sequence

remote: Enumerating objects: 11713, done.
remote: Counting objects: 100% (6519/6519), done.
remote: Compressing objects: 100% (1721/1721), done.
remote: Total 5115 (delta 4177), reused 4181 (delta 3374), pack-reused 0
Receiving objects: 100% (5115/5115), 8.24 MiB | 23.97 MiB/s, done.
Resolving deltas: 100% (4177/4177), completed with 527 local objects.
From https://github.com/netbox-community/netbox
 * branch                15605-migrate-old-sequence -> FETCH_HEAD

hint: You have divergent branches and need to specify how to reconcile them.
hint: You can do so by running one of the following commands sometime before
hint: your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
fatal: Need to specify how to reconcile divergent branches.

sudo ./upgrade.sh 

You are installing (or upgrading to) NetBox version 3.7.5
Using Python 3.10.12
Removing old virtual environment...
Creating a new virtual environment at /opt/netbox/venv...
Updating pip (pip install --upgrade pip)...
...
Successfully installed django-auth-ldap-4.8.0 pyasn1-0.6.0 pyasn1-modules-0.4.0 python-ldap-3.4.4
Applying database migrations (python3 netbox/manage.py migrate)...
Operations to perform:
  Apply all migrations: account, admin, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  No migrations to apply.
...
Completed. Total entries: 36437
Removing expired user sessions (python3 netbox/manage.py clearsessions)...
Upgrade complete! Don't forget to restart the NetBox services:
  > sudo systemctl restart netbox netbox-rq

@jeremystretch
Copy link
Member

I'm going to merge PR #15661 and consider this fixed as the above feedback indicates that the extras.0111_rename_content_types migration now completes successfully even for very old databases.

If you continue to experience different errors when attempting to upgrade, please submit a new bug report with details.

@opericgithub Try just running the migration command (manage.py migrate) after checking out the branch, instead of the whole upgrade script.

jeremystretch added a commit that referenced this issue Apr 10, 2024
@jeremystretch
Fixes #15605: Account for older sequence name in migration
89453a4
@jeremystretch jeremystretch mentioned this issue Apr 11, 2024
Closed
jeremystretch added a commit that referenced this issue Apr 11, 2024
@jeremystretch
Changelog for #15605, #15616, #15617, #15619, #15637, #15638
379fe7c
@SwaroopGiri SwaroopGiri mentioned this issue Jun 17, 2024
Open
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jeremystretch jeremystretch

Labels
beta Concerns a bug/feature in a beta release severity: medium Results in substantial degraded or broken functionality for specfic workflows status: accepted This issue has been accepted for implementation type: bug A confirmed report of unexpected behavior in the application
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests