Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storing deleted messages fails because of "not null" constraint #324

Closed
SebastiaanZ opened this issue Feb 3, 2020 · 6 comments
Closed

Storing deleted messages fails because of "not null" constraint #324

SebastiaanZ opened this issue Feb 3, 2020 · 6 comments

Comments

@SebastiaanZ
Copy link
Member

@SebastiaanZ SebastiaanZ commented Feb 3, 2020

Currently, some of our moderation tools fail because our database rejects messages without a value for the attachment field. The issue is caused by the fact that null=True is not set for this field, while we do not always provide a value:

attachments = pgfields.ArrayField(
models.URLField(
max_length=512
),
blank=True,
help_text="Attachments attached to this message."
)

(Note that blank=True means that the field won't be required in form validation, but it does not mean the field can be blank in the database.)

This bug is critical as it causes some of our moderation tools to fail halfway through their execution; most notably, when our anti-spam feature issues a mute, it will mute the user but not alert moderators. Since there are quite a few false positives, we rely heavily on those alerts to inspect the trigger. The bug also means the deleted messages are not retained for inspection.


Full traceback:

Feb. 3, 2020, 10:06 a.m.	Website	Error	Internal Server Error: /bot/deleted-messages 
Traceback (most recent call last): 
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: null value in column "attachments" violates not-null constraint DETAIL: Failing row contains (673831740799844352, 607247579608121354, .savethedate, {}, 271315596405112833, 164, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last): 
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/viewsets.py", line 116, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/views.py", line 495, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/views.py", line 455, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/views.py", line 492, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/mixins.py", line 21, in create
    self.perform_create(serializer)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/mixins.py", line 26, in perform_create
    serializer.save()
  File "/usr/local/lib/python3.7/site-packages/rest_framework/serializers.py", line 214, in save
    self.instance = self.create(validated_data)
  File "/app/pydis_site/apps/api/serializers.py", line 84, in create
    **message
  File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 422, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 741, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 779, in save_base
    force_update, using, update_fields,
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 870, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 908, in _do_insert
    using=using, raw=raw)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1186, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1368, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execut
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__ 
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: null value in column "attachments" violates not-null constraint DETAIL: Failing row contains (673831740799844352, 607247579608121354, .savethedate, {}, 271315596405112833, 164, null). 
@Akarys42

This comment has been minimized.

Copy link
Member

@Akarys42 Akarys42 commented Feb 3, 2020

python-discord/bot#630 will solve this, the two PRs should have been merged at the same time.

@SebastiaanZ

This comment has been minimized.

Copy link
Member Author

@SebastiaanZ SebastiaanZ commented Feb 3, 2020

Okay, what's blocking the other PR?

@Akarys42

This comment has been minimized.

Copy link
Member

@Akarys42 Akarys42 commented Feb 3, 2020

Some small issues, but @MarkKoz worked on it in the meantime, and from my testing, the bot doesn’t seem to send an empty list when there is no attachments.

@SebastiaanZ

This comment has been minimized.

Copy link
Member Author

@SebastiaanZ SebastiaanZ commented Feb 3, 2020

This has been solved now, right?

@Akarys42

This comment has been minimized.

Copy link
Member

@Akarys42 Akarys42 commented Feb 3, 2020

@SebastiaanZ

This comment has been minimized.

Copy link
Member Author

@SebastiaanZ SebastiaanZ commented Feb 3, 2020

Thanks! I'll close this, then.

@SebastiaanZ SebastiaanZ closed this Feb 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.