Skip to content

Prevent signals in test db setup#9272

Merged
SpecLad merged 9 commits intodevelopfrom
zm/prevent-signals-in-test-db-setup
Mar 28, 2025
Merged

Prevent signals in test db setup#9272
SpecLad merged 9 commits intodevelopfrom
zm/prevent-signals-in-test-db-setup

Conversation

@zhiltsov-max
Copy link
Contributor

@zhiltsov-max zhiltsov-max commented Mar 27, 2025

Motivation and context

To setup test DB CVAT runs python manage.py loaddata .... This command populates the DB with objects from tests/python/shared/assets/cvat_db/data.json. This action also invokes signals on the server normally, as specified for models. In some cases it can lead to unexpected errors with duplicate keys in DB - because, for example, a parent object is loaded first, a signal runs to create a nested object, and then the nested object is created once again by loaddata when time comes. I'm surprised the current test startup is working without extra steps to disable signals.

References:

Another problem is that sometimes it's possible to get test suite startup check failed if the server check is invoked before the server is started.

Changes:

  • Custom signals that create nested objects now ignored when called from loaddata
  • Startup checks in tests now allow the server status check request to fail

How has this been tested?

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@zhiltsov-max zhiltsov-max requested a review from SpecLad as a code owner March 27, 2025 13:09

@receiver(post_save, sender=User, dispatch_uid=__name__ + ".save_user_handler")
def __save_user_handler(instance: User, **kwargs):
if kwargs.get("created") and kwargs.get("raw"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The created check doesn't seem necessary. If we skip the logic on raw creates, then why wouldn't we do that on raw updates?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem I faced was related to extra object creation, so I focused the fix only on this. I'm not sure why this may be called without object creation, but if you know such cases, I'm open to covering this as well.

Copy link
Contributor Author

@zhiltsov-max zhiltsov-max Mar 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, in pre_save we don't have created, so it makes sense to align the logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, doesn't seem to work for any raw - 4a84adb has a number of failed tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, curious...

I'm fine with keeping it as-is for most cases, but for webhooks we do need equivalent checks in the pre_save and post_save handlers; post_save depends on pre_save having run first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, updated. pre_save only adds extra fields to the passed instance to be used later in post_save, so removed all changes from pre_save.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do these changes come from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the source of these changes. I can reproduce it on the develop branch as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this seems to do is put the annotations in a different order. Since the order appears to be non-deterministic, I don't see much reason to commit this. I think we should update the dump script to sort the output before saving.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's revert this

@zhiltsov-max zhiltsov-max mentioned this pull request Mar 27, 2025
6 tasks
@zhiltsov-max zhiltsov-max force-pushed the zm/prevent-signals-in-test-db-setup branch from 3702a44 to 265dfba Compare March 27, 2025 16:48
@zhiltsov-max zhiltsov-max changed the title Prevent signals in test db setup [do not merge] Prevent signals in test db setup Mar 27, 2025
@codecov-commenter
Copy link

codecov-commenter commented Mar 27, 2025

Codecov Report

Attention: Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.

Project coverage is 74.16%. Comparing base (2054e90) to head (67129ee).
Report is 3 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9272      +/-   ##
===========================================
- Coverage    74.18%   74.16%   -0.03%     
===========================================
  Files          429      428       -1     
  Lines        44970    44964       -6     
  Branches      3918     3918              
===========================================
- Hits         33363    33348      -15     
- Misses       11607    11616       +9     
Components Coverage Δ
cvat-ui 77.21% <ø> (-0.05%) ⬇️
cvat-server 71.63% <82.35%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sonarqubecloud
Copy link

@zhiltsov-max zhiltsov-max changed the title [do not merge] Prevent signals in test db setup Prevent signals in test db setup Mar 28, 2025
@SpecLad SpecLad merged commit 21db1d4 into develop Mar 28, 2025
34 checks passed
@SpecLad SpecLad deleted the zm/prevent-signals-in-test-db-setup branch March 28, 2025 12:02
azhavoro added a commit that referenced this pull request Jun 27, 2025
<!-- Raise an issue to propose your change
(https://github.com/cvat-ai/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution guide](https://docs.cvat.ai/docs/contributing/).
-->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
Resolves #9382 
I have ran into the same issue, I have done some searching and saw that
the LDAP `create_user` was changed in #9272 for the LDAP implementation
to have two new arguments `created` and `raw` which is not needed in the
[Django LDAP
Authentication](https://github.com/django-auth-ldap/django-auth-ldap/blob/master/django_auth_ldap/backend.py),
I think the change was just meant to fall in line with the [django
post_save
signal](https://docs.djangoproject.com/en/5.1/ref/signals/#django.db.models.signals.post_save)

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->
I removed the lines and rebuilt and tried logging in with LDAP and it
succeed.

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
- [ ] ~~I have updated the documentation accordingly~~
- [ ] ~~I have added tests to cover my changes~~
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

---------

Co-authored-by: Kira Pearce <kira.pearce@digital-workbench.de>
Co-authored-by: Andrey Zhavoronkov <andrey@cvat.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments