Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ticket #34631 Optimize Column expression identity #16940

Closed
wants to merge 3 commits into from

Conversation

snuderl
Copy link

@snuderl snuderl commented Jun 4, 2023

https://code.djangoproject.com/ticket/34631#ticket

Based on some profiling I noticed a lot of time is spent doing reflection/inspect family of calls. The culprit looks like identity property, especially that on Col class. We can easily avoid reflection here.

Running a relatively simple query with a good number of columns in our codebase gives a nice boost.

qs = Asset.objects.filter(pk=3).select_related("asset_contract")[:21]
%timeit str(qs.query)

Before:
440us
After:
280us

@snuderl snuderl closed this Jun 4, 2023
@snuderl snuderl deleted the optimize-col-identity branch June 4, 2023 11:36
@snuderl snuderl restored the optimize-col-identity branch June 4, 2023 11:36
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello! Thank you for your contribution 💪

As it's your first contribution be sure to check out the patch review checklist.

If you're fixing a ticket from Trac make sure to set the "Has patch" flag and include a link to this PR in the ticket!

If you have any design or process questions then you can ask in the Django forum.

Welcome aboard ⛵️!

@snuderl snuderl reopened this Jun 4, 2023
@snuderl snuderl changed the title Optimize col identity Optimize Column expression identity Jun 4, 2023
@snuderl snuderl closed this Jun 4, 2023
@snuderl snuderl reopened this Jun 4, 2023
@smithdc1 smithdc1 added the benchmark Apply to have ASV benchmarks run on a PR label Jun 4, 2023
Copy link

@damjankuznar damjankuznar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@snuderl snuderl changed the title Optimize Column expression identity Ticket #34631 Optimize Column expression identity Jun 5, 2023
@@ -1142,6 +1142,10 @@ def __init__(self, alias, target, output_field=None):
super().__init__(output_field=output_field)
self.alias, self.target = alias, target

@cached_property
def identity(self):
return (self.__class__, self.alias, self.target, self.output_field)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to move this back to a property as testing on django 4.2.2 showed no perf benefit otherwise anymore. I initially tested on 4.2.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's suspicious. As far as I'm aware, we haven't done anything that could affect this optimization. Maybe it's not worth changing 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah look I agree with felix here, especially seeing it's only a couple of dot releases. I think there needs be a second party confirming the profiling and not just throw in optimisations in a somewhat random looking manner :)

Copy link
Author

@snuderl snuderl Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://code.djangoproject.com/ticket/34580 I think its related to this. Before the optimization in that PR we would always end up calling identity on these objects so the difference was much easily observed. Right now a very simple query without order_by will not call identity and thus there will be no difference.

When I add order_by to my query I see the difference. Let me provide some easy to reproduce examples.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's no longer worth if it's only micro-optimization.

Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
@felixxm
Copy link
Member

felixxm commented Jun 8, 2023

Closing per ticket.

@felixxm felixxm closed this Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark Apply to have ASV benchmarks run on a PR
Projects
None yet
5 participants