Token-generating automation scripts fail in 4.5 (ok in 4.4.10)

[bevand10]

NetBox Edition

NetBox Community

NetBox Version

v4.5.4

Python Version

3.12

Steps to Reproduce

GIVEN
An automation script generates a user token (using the Netbox generator script) and assigns to a user using the NetBox manage.py shell script as follows:

set_netbox_api_token() {
  local username="$1"
  local token="$2"
  local mode="${3:-rw}"   # "rw" (default) or "ro"

  if [ -z "$username" ] || [ -z "$token" ]; then
    echo "Error: usage: set_netbox_api_token <username> <token> [rw|ro]" >&2
    return 1
  fi

  NB_USERNAME="$username" NB_TOKEN="$token" NB_MODE="$mode" \
  $PYTHON manage.py shell <<'PYCODE'
import os, sys
from django.contrib.auth import get_user_model
from users.models import Token

username = os.environ['NB_USERNAME']
desired_key = os.environ['NB_TOKEN']
mode = os.environ.get('NB_MODE', 'rw').lower()

if mode not in ('rw','ro'):
    print(f"❌ Error: NB_MODE must be 'rw' or 'ro' (got {mode!r})", file=sys.stderr)
    sys.exit(1)

write_enabled = (mode == 'rw')
User = get_user_model()

try:
    user = User.objects.get(username=username)
except User.DoesNotExist:
    print(f"❌ Error: User '{username}' does not exist.", file=sys.stderr)
    sys.exit(1)

# Guard against cross-user key collision
existing_other = Token.objects.filter(key=desired_key).exclude(user=user).first()
if existing_other:
    print(f"❌ Error: Token key already in use by user '{existing_other.user.username}'.", file=sys.stderr)
    sys.exit(2)

# Upsert-by-key for THIS user so multiple tokens can coexist
token_obj = Token.objects.filter(user=user, key=desired_key).first()
created = False
if not token_obj:
    token_obj = Token(user=user, key=desired_key)
    created = True

changed = created

# Ensure desired write mode
if getattr(token_obj, 'write_enabled', True) != write_enabled:
    token_obj.write_enabled = write_enabled
    changed = True

# Pin as "non-expiring"
if getattr(token_obj, 'expires', None) is not None:
    token_obj.expires = None
    changed = True

if changed:
    token_obj.save()
    status = "created" if created else "updated"
    print(f"✅ API token {status} for '{user.username}' (mode={'RW' if write_enabled else 'RO'}).", file=sys.stderr)
else:
    print(f"🔄 API token for '{user.username}' already matches desired state (mode={'RW' if write_enabled else 'RO'}).", file=sys.stderr)
PYCODE
}

WHEN
When used in a 4.4 series system, the script works well, no errors. The underying database schema at 4.4.10 is:

\d users_token
                                    Table "public.users_token"
    Column     |           Type           | Collation | Nullable |             Default
---------------+--------------------------+-----------+----------+----------------------------------
 id            | bigint                   |           | not null | generated by default as identity
 created       | timestamp with time zone |           | not null |
 expires       | timestamp with time zone |           |          |
 key           | character varying(40)    |           | not null |
 write_enabled | boolean                  |           | not null |
 description   | character varying(200)   |           | not null |
 user_id       | bigint                   |           | not null |
 allowed_ips   | cidr[]                   |           |          |
 last_used     | timestamp with time zone |           |          |
Indexes:
    "users_token_pkey" PRIMARY KEY, btree (id)
    "users_token_key_820deccd_like" btree (key varchar_pattern_ops)
    "users_token_key_key" UNIQUE CONSTRAINT, btree (key)
    "users_token_user_id_af964690" btree (user_id)
Foreign-key constraints:
    "users_token_user_id_af964690_fk_auth_user_id" FOREIGN KEY (user_id) REFERENCES users_user(id) DEFERRABLE INITIALLY DEFERRED
Referenced by:
    TABLE "netbox_proxbox_netboxendpoint" CONSTRAINT "netbox_proxbox_netbo_token_id_db9b4868_fk_users_tok" FOREIGN KEY (token_id) REFERENCES users_token(id) DEFERRABLE INITIALLY DEFERRED

However, when used in a 4.5.4 system, the script fails as shown below.

The schema for the users_token table has changed between 4.4 and 4.5. It is now:

netbox=> \d users_token
                                     Table "public.users_token"
    Column     |           Type           | Collation | Nullable |             Default              
---------------+--------------------------+-----------+----------+----------------------------------
 id            | bigint                   |           | not null | generated by default as identity
 created       | timestamp with time zone |           | not null | 
 expires       | timestamp with time zone |           |          | 
 plaintext     | character varying(40)    |           |          | 
 write_enabled | boolean                  |           | not null | 
 description   | character varying(200)   |           | not null | 
 user_id       | bigint                   |           | not null | 
 allowed_ips   | cidr[]                   |           |          | 
 last_used     | timestamp with time zone |           |          | 
 enabled       | boolean                  |           | not null | 
 version       | smallint                 |           | not null | 
 key           | character varying(12)    |           |          | 
 pepper_id     | smallint                 |           |          | 
 hmac_digest   | character varying(64)    |           |          | 
Indexes:
    "users_token_pkey" PRIMARY KEY, btree (id)
    "users_token_key_820deccd_like" btree (key varchar_pattern_ops)
    "users_token_key_key" UNIQUE CONSTRAINT, btree (key)
    "users_token_plaintext_46c6f315_like" btree (plaintext varchar_pattern_ops)
    "users_token_plaintext_key" UNIQUE CONSTRAINT, btree (plaintext)
    "users_token_user_id_af964690" btree (user_id)
Check constraints:
    "enforce_version_dependent_fields" CHECK (hmac_digest IS NULL AND key IS NULL AND pepper_id IS NULL AND plaintext IS NOT NULL AND version = 1 OR hmac_digest IS NOT NULL AND key IS NOT NULL AND pepper_id IS NOT NULL AND plaintext IS NULL AND version = 2)
    "users_token_pepper_id_check" CHECK (pepper_id >= 0)
    "users_token_version_check" CHECK (version >= 0)
Foreign-key constraints:
    "users_token_user_id_af964690_fk_auth_user_id" FOREIGN KEY (user_id) REFERENCES users_user(id) DEFERRABLE INITIALLY DEFERRED

netbox=>

THEN

Given I'm using the netbox models to do this work, is the error mine, or should the token model compensate accordingly?

Expected Behavior

User token successfully applied to the user.

Observed Behavior

These errors emit when running the same script:

Attempting to create or update user 'dbuser'...
161 objects imported automatically (use -v 2 for details).

🔄 User 'dbuser' updated (permissions & password reset).
161 objects imported automatically (use -v 2 for details).

Traceback (most recent call last):
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
psycopg.errors.StringDataRightTruncation: value too long for type character varying(12)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox-4.5.4/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/commands/shell.py", line 257, in handle
    exec(sys.stdin.read(), {**globals(), **self.get_namespace(**options)})
  File "<string>", line 48, in <module>
  File "/opt/netbox-4.5.4/netbox/users/models/tokens.py", line 246, in save
    return super().save(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 902, in save
    self.save_base(
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1169, in _save_table
    results = self._do_insert(
              ^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1210, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/query.py", line 1873, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
django.db.utils.DataError: value too long for type character varying(12)
161 objects imported automatically (use -v 2 for details).

Traceback (most recent call last):
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
psycopg.errors.StringDataRightTruncation: value too long for type character varying(12)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox-4.5.4/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/core/management/commands/shell.py", line 257, in handle
    exec(sys.stdin.read(), {**globals(), **self.get_namespace(**options)})
  File "<string>", line 48, in <module>
  File "/opt/netbox-4.5.4/netbox/users/models/tokens.py", line 246, in save
    return super().save(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 902, in save
    self.save_base(
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1008, in save_base
    updated = self._save_table(
              ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1169, in _save_table
    results = self._do_insert(
              ^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/base.py", line 1210, in _do_insert
    return manager._insert(
           ^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/query.py", line 1873, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/models/sql/compiler.py", line 1882, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib64/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib64/python3.12/site-packages/psycopg/cursor.py", line 117, in execute
    raise ex.with_traceback(None)
django.db.utils.DataError: value too long for type character varying(12)
161 objects imported automatically (use -v 2 for details).

[pheus]

@bevand10, thanks for the report. I’ve converted this to a discussion because the report appears to point to custom automation rather than a bug in NetBox itself.

Please keep in mind that minor NetBox releases may include breaking changes for API consumers and other integrations, so custom automation may need to be updated between releases. In this case, the v4.5.0 release notes call out the changes to Token.

I’d also be cautious about managing tokens by writing directly to the ORM from a shell script. For example:

if changed:
    token_obj.save()

Calling save() directly here is fragile because it skips the normal validation step, so problems may surface only as a database error.

If you need to automate token creation, I’d recommend using the REST API. The docs cover both token provisioning and creating tokens for other users, and pynetbox can help with that as well.