Skip to content

Enhance queryset with prefetch for many-to-many fields#32

Closed
Mte90 wants to merge 2 commits into
AlexanderCollins:mainfrom
Mte90:patch-5
Closed

Enhance queryset with prefetch for many-to-many fields#32
Mte90 wants to merge 2 commits into
AlexanderCollins:mainfrom
Mte90:patch-5

Conversation

@Mte90
Copy link
Copy Markdown
Contributor

@Mte90 Mte90 commented Oct 17, 2025

Added prefetching of many-to-many fields in queryset.

Added prefetching of many-to-many fields in queryset.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the queryset optimization in get_queryset() to automatically detect and prefetch many-to-many fields alongside the existing select_related optimization for foreign keys. This prevents N+1 query problems when accessing many-to-many relationships through nested field notation.

Key Changes:

  • Added prefetch_fields list to collect ManyToManyField relationships
  • Updated field type detection logic to differentiate between ForeignKey/OneToOneField and ManyToManyField
  • Applied prefetch_related() to the queryset when many-to-many fields are detected

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread turbodrf/views.py
# This is a related field
base_field = field.split("__")[0]
if base_field not in select_related_fields:
model_field = self.model._meta.get_field(base_field)
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The models module is not imported at the module level. This will cause a NameError when this code executes. Add from django.db import models at the top of the file with the other imports.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot that's wrong because this is a class extending from another and as you can see it is internal as parameter not from that value

Comment thread turbodrf/views.py Outdated
Comment thread turbodrf/views.py
Comment on lines +344 to +345
if prefetch_fields:
queryset = queryset.prefetch_related(*prefetch_fields)
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new prefetch_related functionality for ManyToManyField relationships lacks test coverage. Consider adding tests that verify:

  1. ManyToMany fields are correctly identified and added to prefetch_fields
  2. The queryset actually has prefetch_related applied
  3. No N+1 query issues occur when accessing many-to-many relationships

You may need to add a ManyToManyField to one of the test models (e.g., in tests/test_app/models.py) to properly test this functionality.

Copilot uses AI. Check for mistakes.
Comment thread turbodrf/views.py Outdated
Comment on lines +335 to +339
model_field = self.model._meta.get_field(base_field)
if isinstance(model_field, (models.ForeignKey, models.OneToOneField)):
select_related_fields.append(base_field)
if isinstance(model_field, models.ManyToManyField):
prefetch_fields.append(base_field)
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to self.model._meta.get_field(base_field) can raise a FieldDoesNotExist exception if the base_field doesn't exist on the model. Consider wrapping this in a try-except block to handle invalid field names gracefully:

try:
    model_field = self.model._meta.get_field(base_field)
    if isinstance(model_field, (models.ForeignKey, models.OneToOneField)):
        # ...
except FieldDoesNotExist:
    continue  # Skip invalid field names

This pattern is used elsewhere in the codebase (e.g., in turbodrf/mixins.py line 153).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

base_field always exists, @copilot as you can see in the lines above

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@AlexanderCollins
Copy link
Copy Markdown
Owner

Thanks for this PR! M2M field prefetching has been implemented in v0.2.0 - the get_queryset() method now automatically detects and applies prefetch_related() for many-to-many relationships alongside the existing select_related() optimization for foreign keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants