Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use data loaders to eliminate duplicate queries #5303

Merged
merged 4 commits into from Apr 24, 2020
Merged

Use data loaders to eliminate duplicate queries #5303

merged 4 commits into from Apr 24, 2020

Conversation

patrys
Copy link
Member

@patrys patrys commented Feb 27, 2020

⚠️ Work in progress!

This is an initial implementation with some basic data loaders. You can test them by fetching the pricing information of products and their variants.

Pull Request Checklist

  1. Privileged views and APIs are guarded by proper permission checks.
  2. All visible strings are translated with proper context.
  3. All data-formatting is locale-aware (dates, numbers, and so on).
  4. Database queries are optimized and the number of queries is constant.
  5. Database migration files are up to date.
  6. The changes are tested.
  7. GraphQL schema and type definitions are up to date.
  8. Changes are mentioned in the changelog.

@patrys patrys requested review from maarcingebala and fowczarek Feb 27, 2020
@patrys patrys changed the title Ininitial work on adding data loaders Initial work on adding data loaders Feb 27, 2020
Copy link

django-queries commented Feb 27, 2020

Here is the report for 355a37c83f3da39550460bb0ec5b0481e54bf326 (patrys/saleor @ introduce-data-loaders)
Base comparison is 079efe9.

**Found 7 differences!** (click me)

# api.benchmark category
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
+ category view                              	         23	         19	              1

# api.benchmark checkout
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
- add billing address to checkout            	         34	         36	             19
  add shipping to checkout                   	          7	          7	              0
  checkout payment charge                    	         10	         10	              0
  complete checkout                          	          8	          8	              0
  create checkout                            	          5	          5	              1

# api.benchmark collection
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
+ collection view                            	         21	         18	              0

# api.benchmark homepage
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
+ featured products list                     	         20	         14	              0
  retrieve main menu                         	          8	          8	              0
  retrieve product list                      	          6	          6	              0
  retrieve secondary menu                    	          8	          8	              0
  retrieve shop                              	          2	          2	              0
- user checkout details                      	         53	         57	             27

# api.benchmark product
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
  product details                            	         17	         17	              1
+ retrieve product attributes                	          9	          6	              0

# api.benchmark variant
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
  product variant bulk create                	         51	         51	              2
+ retrieve variant list                      	         23	         16	              3

# api product sorting attributes
  test name                                  	left count 	right count	duplicate count
  -------------------------------------------	-----------	-----------	---------------
  sort product not having attribute data     	         21	         21	              0

@codecov
Copy link

codecov bot commented Feb 27, 2020

Codecov Report

Merging #5303 into master will increase coverage by 0.01%.
The diff coverage is 92.24%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5303      +/-   ##
==========================================
+ Coverage   91.59%   91.61%   +0.01%     
==========================================
  Files         283      286       +3     
  Lines       18181    18285     +104     
  Branches     1581     1598      +17     
==========================================
+ Hits        16653    16751      +98     
- Misses       1105     1110       +5     
- Partials      423      424       +1
Impacted Files Coverage Δ
saleor/graphql/product/resolvers.py 83.33% <100%> (ø) ⬆️
saleor/discount/utils.py 100% <100%> (ø) ⬆️
saleor/graphql/product/dataloaders.py 100% <100%> (ø)
saleor/graphql/discount/dataloaders.py 100% <100%> (ø)
saleor/core/middleware.py 54.4% <100%> (+1.89%) ⬆️
saleor/graphql/product/types/products.py 90.74% <70.83%> (-0.62%) ⬇️
saleor/graphql/core/dataloaders.py 94.28% <94.28%> (ø)
saleor/graphql/core/mutations.py 95.3% <0%> (ø) ⬆️
saleor/graphql/shop/schema.py 100% <0%> (ø) ⬆️
saleor/graphql/webhook/types.py 100% <0%> (ø) ⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2c6d70...64890d9. Read the comment docs.

@codecov
Copy link

codecov bot commented Feb 27, 2020

Codecov Report

Merging #5303 into master will decrease coverage by 0.01%.
The diff coverage is 98.37%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #5303      +/-   ##
==========================================
- Coverage   91.51%   91.50%   -0.02%     
==========================================
  Files         296      302       +6     
  Lines       19251    19535     +284     
  Branches     1745     1812      +67     
==========================================
+ Hits        17618    17875     +257     
- Misses       1188     1215      +27     
  Partials      445      445              
Impacted Files Coverage Δ
saleor/graphql/account/deprecated/types.py 94.44% <ø> (-0.20%) ⬇️
saleor/graphql/app/types.py 94.00% <ø> (-0.24%) ⬇️
saleor/graphql/core/utils/__init__.py 100.00% <ø> (ø)
saleor/graphql/payment/mutations.py 98.34% <ø> (ø)
saleor/graphql/payment/types.py 84.52% <ø> (-0.71%) ⬇️
saleor/plugins/avatax/plugin.py 57.42% <ø> (ø)
saleor/plugins/base_plugin.py 96.46% <ø> (ø)
saleor/plugins/vatlayer/plugin.py 73.94% <ø> (ø)
saleor/graphql/product/types/digital_contents.py 90.90% <50.00%> (+2.33%) ⬆️
saleor/graphql/product/types/products.py 86.65% <91.17%> (-5.48%) ⬇️
... and 47 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9d147d...5aeb1f4. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Feb 27, 2020

This pull request introduces 1 alert when merging 64890d9edfc5fe8991cdf9597060ef4fea38f21c into 89ca7b6 - view on LGTM.com

new alerts:

  • 1 for Unused import

@lgtm-com
Copy link

lgtm-com bot commented Mar 4, 2020

This pull request introduces 3 alerts when merging 6ba105c6bec5ef6fb0ad8815d5639c4f1d6112b8 into d8fe58d - view on LGTM.com

new alerts:

  • 2 for Module imports itself
  • 1 for Unused import

@lgtm-com
Copy link

lgtm-com bot commented Mar 4, 2020

This pull request introduces 3 alerts when merging 01ff785e280cfb3a62798b2c056f988067429aeb into d8fe58d - view on LGTM.com

new alerts:

  • 2 for Module imports itself
  • 1 for Unused import

@lgtm-com
Copy link

lgtm-com bot commented Mar 6, 2020

This pull request introduces 3 alerts when merging 355a37c83f3da39550460bb0ec5b0481e54bf326 into 079efe9 - view on LGTM.com

new alerts:

  • 2 for Module imports itself
  • 1 for Unused import

@patrys patrys changed the title Initial work on adding data loaders Use data loaders to eliminate duplicate queries Apr 6, 2020
@db-queries
Copy link

db-queries bot commented Apr 16, 2020

Here is the report for 5aeb1f4 (patrys/saleor @ introduce-data-loaders)
Base comparison is c9d147d.

**Found 20 differences!** (click me)

# api.benchmark account
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
  delete staff members                                       	         32	         32	              0
  query staff user                                           	         21	         21	              4
  staff create                                               	         24	         24	              5
  staff update groups and permissions                        	         36	         36	              6

# api.benchmark category
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ category view                                              	         23	         18	              1

# api.benchmark checkout mutations
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ add billing address to checkout                            	         58	         52	             26
- add shipping to checkout                                   	         36	         40	             12
+ checkout email update                                      	         35	         29	             13
  checkout payment charge                                    	         23	         23	              5
- checkout shipping address update                           	         32	         35	              8
+ checkout voucher code                                      	         67	         59	             31
+ complete checkout                                          	         75	         74	             19
+ create checkout                                            	        184	        140	             75
+ update checkout lines                                      	        133	        101	             50

# api.benchmark collection
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ collection view                                            	         21	         17	              0

# api.benchmark homepage
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ featured products list                                     	         20	         14	              0
  retrieve main menu                                         	          8	          8	              0
  retrieve product list                                      	          5	          5	              0
  retrieve secondary menu                                    	          8	          8	              0
  retrieve shop                                              	          2	          2	              0
+ user checkout details                                      	         56	         52	             26

# api.benchmark order
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ user order details                                         	         18	         17	              2

# api.benchmark permission group
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
  permission group create                                    	         23	         23	              5
  permission group delete                                    	         21	         21	              4
+ permission group query                                     	          9	          8	              0
  permission group update                                    	         35	         35	              5
  permission group update remove users with manage staff     	         31	         31	              4

# api.benchmark product
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
+ product details                                            	         21	         19	              2
+ retrieve product attributes                                	          8	          7	              0

# api.benchmark variant
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
  product variant bulk create                                	         48	         48	              2
+ retrieve variant list                                      	         35	         23	              6

# api.benchmark variant stocks
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
- product variants stocks create                             	         21	         23	              5
- product variants stocks delete                             	         19	         20	              5
- product variants stocks update                             	         26	         28	              5

# api product sorting attributes
  test name                                                  	left count 	right count	duplicate count
  -----------------------------------------------------------	-----------	-----------	---------------
  sort product not having attribute data                     	         20	         20	              0

@lgtm-com
Copy link

lgtm-com bot commented Apr 16, 2020

This pull request introduces 1 alert when merging 858ea102a0d35e1e36b34b4a7456e3bef1a48f94 into 21af455 - view on LGTM.com

new alerts:

  • 1 for Unused import

@patrys patrys marked this pull request as ready for review Apr 23, 2020
@maarcingebala maarcingebala merged commit 46df272 into saleor:master Apr 24, 2020
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants