Benchmark tests after implementing order promotions. #15359

zedzior · 2024-02-05T19:03:20Z

Issue: #15350
The PR adds benchmark test after implementing order promotions.

Impact

New migrations
New/Updated API fields or mutations
Deprecated API fields or mutations
Removed API types, fields, or mutations

Docs

Link to documentation:

Pull Request Checklist

Privileged queries and mutations are either absent or guarded by proper permission checks
Database queries are optimized and the number of queries is constant
Database migrations are either absent or optimized for zero downtime
The changes are covered by test cases
All new fields/inputs/mutations have proper labels added (ADDED_IN_X, PREVIEW_FEATURE, etc.)
All migrations have proper dependencies
All indexes are added concurrently in migrations
All RunSql and RunPython migrations have revert option defined

db-queries · 2024-02-05T19:26:45Z

Here is the report for fdfbe98 (saleor:15350-benchmark-tests)
Missing base report (da257ae). The results couldn't be compared.

No differences were found.

# saleor.graphql.accountbenchmark account
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  addresses for federation query count                                    	          9	          9	              2
  customers query                                                         	         51	         51	              0
  delete staff members                                                    	         41	         41	              0
  query staff user                                                        	         22	         22	              3
  staff create                                                            	         23	         23	              3
  staff update groups and permissions                                     	         36	         36	              5
  users for federation query count                                        	          8	          8	              3

# saleor.graphql.accountbenchmark permission group
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  groups for federation query count                                       	          5	          5	              0
  permission group create                                                 	         25	         25	              0
  permission group delete                                                 	         30	         30	              3
  permission group query                                                  	         14	         14	              0
  permission group update                                                 	         44	         44	              2
  permission group update remove users with manage staff                  	         36	         36	              3

# saleor.graphql.appbenchmarks app extensions
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  app extensions                                                          	         15	         15	              0
  app extensions with filter[filter0]                                     	         14	         14	              0
  app extensions with filter[filter1]                                     	         10	         10	              0
  app extensions with filter[filter2]                                     	         14	         14	              0
  app extensions with filter[filter3]                                     	         10	         10	              0

# saleor.graphql.appbenchmarks apps
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  apps for federation query count                                         	          9	          9	              3
  apps with tokens and webhooks                                           	          9	          9	              0

# saleor.graphql.attributebenchmark attribute
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  attribute translation                                                   	          6	          6	              0
  attribute value translation                                             	         20	         20	              0
  query attribute                                                         	          6	          6	              0
  query attributes                                                        	          8	          8	              0

# saleor.graphql.channelbenchmark channel
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  channels query                                                          	          5	          5	              0

# saleor.graphql.channelmutations channel update
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  channel update mutation remove warehouse                                	         38	         38	              6

# saleor.graphql.checkoutbenchmark checkout mutations
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  add billing address to checkout                                         	         77	         77	             10
  add checkout lines                                                      	         86	         86	             13
  add checkout lines gift discount applies                                	        147	        147	             21
  add checkout lines order discount applies                               	        161	        161	             16
  add checkout lines with external shipping                               	        189	        189	             76
  add checkout lines with reservations                                    	        181	        181	             65
  add delivery to checkout                                                	         83	         83	             13
  add shipping to checkout                                                	         93	         93	             15
  checkout email update                                                   	         42	         42	              1
  checkout payment charge                                                 	         57	         57	             15
  checkout shipping address update                                        	         87	         87	             11
  checkout voucher code                                                   	        101	        101	             17
  complete checkout                                                       	        239	        239	             81
  complete checkout preorder                                              	        237	        237	             82
  complete checkout with digital line                                     	        280	        280	             95
  complete checkout with out of stock webhook                             	        241	        241	             81
  complete checkout with single line                                      	        241	        241	             81
  create checkout                                                         	         83	         83	              8
  create checkout with gift promotion                                     	        122	        122	             18
  create checkout with order promotion                                    	        104	        104	              8
  create checkout with reservations                                       	        151	        151	             25
  customer complete checkout                                              	        252	        252	             87
  customer complete checkout for cc                                       	        227	        227	             76
  update checkout lines                                                   	         79	         79	             13
  update checkout lines with reservations                                 	        187	        187	             76

# saleor.graphql.checkoutbenchmark checkouts
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  staff multiple checkouts                                                	          6	          6	              0

# saleor.graphql.checkoutbenchmark homepage
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  user checkout details                                                   	         60	         60	              5
  user checkout details with tax app                                      	         54	         54	              2

# saleor.graphql.discountbenchmark promotion create
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotion create                                                        	         52	         52	              3
  promotion create order promotion                                        	         41	         41	              7

# saleor.graphql.discountbenchmark promotion delete
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  gift promotion delete                                                   	         24	         24	              1
  promotion delete                                                        	         32	         32	              1

# saleor.graphql.discountbenchmark promotion rule create
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotion rule create                                                   	         36	         36	              0
  promotion rule create gift                                              	        146	        146	             12

# saleor.graphql.discountbenchmark promotion rule delete
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotion rule delete                                                   	         22	         22	              0

# saleor.graphql.discountbenchmark promotion rule update
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotion rule update                                                   	         41	         41	              1

# saleor.graphql.discountbenchmark promotion update
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotion update                                                        	         38	         38	              1

# saleor.graphql.discountbenchmark promotions
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  promotions querytest promotions query                                   	          9	          9	              0

# saleor.graphql.discountbenchmark sales
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  sales query with channel slug                                           	          9	          9	              0
  sales query without channel slug                                        	          7	          7	              0

# saleor.graphql.discountbenchmark voucher code bulk delete
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  voucher code bulk delete queries                                        	         22	         22	              2

# saleor.graphql.discountbenchmark vouchers
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  vouchers query with channel slug                                        	         20	         20	              0
  vouchers query withot channel slug                                      	         19	         19	              0

# saleor.graphql.giftcardbenchmark gift card mutations
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  create never expiry gift card                                           	         26	         26	              4
  gift card bulk activate by staff                                        	         16	         16	              3
  update gift card                                                        	         36	         36	              4

# saleor.graphql.giftcardbenchmark gift card queries
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  filter gift cards by products                                           	         14	         14	              3
  filter gift cards by tags                                               	         12	         12	              3
  filter gift cards by used by user                                       	         13	         13	              3
  query gift card details                                                 	         14	         14	              3
  query gift cards                                                        	         12	         12	              3

# saleor.graphql.orderbenchmark fulfillment
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  fulfillment query                                                       	          6	          6	              0

# saleor.graphql.orderbenchmark fulfillment refund and return products
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  fulfillment refund products order lines                                 	         60	         60	              0
  fulfillment return products order lines                                 	        102	        102	              7

# saleor.graphql.orderbenchmark order
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  staff multiple draft orders                                             	          7	          7	              0
  staff multiple orders                                                   	          7	          7	              0
  staff order details                                                     	         56	         56	              7
  user order details                                                      	         54	         54	              8

# saleor.graphql.orderbenchmark order bulk create
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  order bulk create                                                       	         21	         21	              2

# saleor.graphql.orderbenchmark order fulfill
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  order fulfill                                                           	         19	         19	              0
  order fulfill with gift cards                                           	         19	         19	              0

# saleor.graphql.orderbenchmark order lines create
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  order lines create                                                      	         94	         94	             17
  order lines create variants on promotion                                	        155	        155	             22

# saleor.graphql.pagebenchmark page type
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  page types for federation query count                                   	          3	          3	              0
  query page type                                                         	         18	         18	              6
  query page types                                                        	         25	         25	             12

# saleor.graphql.paymentbenchmark payment transactions
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  payment transactions                                                    	         54	         54	             20

# saleor.graphql.productbenchmark category
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  categories children                                                     	          3	          3	              0
  categories for federation query count                                   	          3	          3	              0
  category delete                                                         	         65	         65	              9
  category view                                                           	         33	         33	              4

# saleor.graphql.productbenchmark collection
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  collection add products                                                 	         46	         46	              8
  collection bulk delete                                                  	         54	         54	              6
  collection view                                                         	         11	         11	              3
  collections for federation query count                                  	          6	          6	              0
  create collection                                                       	         37	         37	              5
  delete collection                                                       	         51	         51	              7
  remove products from collection                                         	         43	         43	              8
  retrieve collection channel listings                                    	          5	          5	              0

# saleor.graphql.productbenchmark homepage
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  report product sales                                                    	         10	         10	              3
  retrieve product list                                                   	          4	          4	              0

# saleor.graphql.productbenchmark product
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  filter products by attributes                                           	         10	         10	              1
  filter products by boolean attributes                                   	         17	         17	              1
  filter products by gift card                                            	         14	         14	              2
  filter products by numeric attributes                                   	         16	         16	              1
  product create                                                          	         69	         69	              3
  product details                                                         	         46	         46	              4
  product translations                                                    	          8	          8	              1
  products for federation query count                                     	         12	         12	              4
  products media for federation query count                               	          5	          5	              0
  products types for federation query count                               	          2	          2	              0
  retrieve channel listings                                               	         20	         20	              0
  retrieve product attributes                                             	         11	         11	              1
  retrieve product images                                                 	          7	          7	              1
  retrieve product media                                                  	          7	          7	              1
  retrive products with product types and attributes                      	         10	         10	              1
  update product                                                          	         62	         62	              5

# saleor.graphql.productbenchmark product variant channel listing update
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  variant channel listing update                                          	         47	         47	              3

# saleor.graphql.productbenchmark variant
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  product variant bulk create                                             	         67	         67	              1
  product variant create                                                  	         71	         71	              3
  products variants for federation query count                            	         10	         10	              3
  retrieve variant list                                                   	         37	         37	              2
  update product variant                                                  	         69	         69	              5

# saleor.graphql.productbenchmark variant stocks
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  product variants stocks create                                          	         29	         29	              4
  product variants stocks create with single webhook called               	         28	         28	              4
  product variants stocks delete by id                                    	         30	         30	              4
  product variants stocks delete by sku                                   	         30	         30	              4
  product variants stocks delete with out of stock webhook many calls     	         26	         26	              3
  product variants stocks update by sku                                   	         28	         28	              4
  product variants stocks update byid                                     	         28	         28	              4
  query product variants stocks                                           	         10	         10	              0

# saleor.graphql.producttest product sorting attributes
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  sort product not having attribute data                                  	         20	         20	              0

# saleor.graphql.shippingbenchmark shipping methods
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  vouchers query with channel slug                                        	          7	          7	              0
  vouchers query without channel slug                                     	          7	          7	              0

# saleor.graphql.shippingbenchmark shipping zones
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  shipping zones query                                                    	          6	          6	              0

# saleor.graphql.shopbenchmark homepage
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  retrieve shop                                                           	          2	          2	              0

# saleor.graphql.warehousebenchmark stock bulk update
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  stocks bulk update queries count                                        	         33	         33	              5

# saleor.graphql.warehousebenchmark stocks
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  stocks query                                                            	          5	          5	              0

# saleor.graphql.webhookbenchmark webhook events
  test name                                                               	left count 	right count	duplicate count
  ------------------------------------------------------------------------	-----------	-----------	---------------
  webhooks                                                                	          6	          6	              0

codecov · 2024-02-05T19:31:46Z

Codecov Report

All modified and coverable lines are covered by tests ✅

❗ No coverage uploaded for pull request base (feature/gift-promotions@da257ae). Click here to learn what that means.

Additional details and impacted files

@@                    Coverage Diff                     @@
##             feature/gift-promotions   #15359   +/-   ##
==========================================================
  Coverage                           ?   93.92%           
==========================================================
  Files                              ?      929           
  Lines                              ?    70623           
  Branches                           ?     8197           
==========================================================
  Hits                               ?    66333           
  Misses                             ?     2755           
  Partials                           ?     1535

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

saleor/graphql/checkout/tests/benchmark/test_checkout_mutations.py

saleor/graphql/discount/tests/benchmark/test_promotion_create.py

Ashir40 · 2024-02-07T11:55:48Z

Unsubscribe

…

On Wed, 7 Feb 2024, 4:53 pm Piotr Zabieglik, ***@***.***> wrote: Merged #15359 <#15359> into feature/gift-promotions. — Reply to this email directly, view it on GitHub <#15359 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXKN7GDK7SFSHD447Z5H4RTYSNTK7AVCNFSM6AAAAABC2VR6G6VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJRG4ZTCMZRGYYDONY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

* Add tests for checkout mutations. * Add benchmarks for promotoin mutations. * Add checks if discount was created. * Appply review comments.

* Apply database changes for gift promotion rule (#15254) * Extend PromotionRule with gifts field * Extend CheckoutLine and OrderLine with is_gift field * Add GIFT RewardType * Add gift_promotion_rule fixture * Fix migrations * Fix fixture * Gift promotions API. (#15274) * Add 'gifts' and 'giftsLimit' fields to 'PromotionRule' type. * Add 'is_gift' field to 'OrderLine' and 'CheckoutLine' types. * Add 'gifts' to promotion rule input; validate gift promotions. * Make it working. * Add tests to promotion rule create. * Adjust promotion create mutation. * Add test for promotion create mutation. * Add tests to promotion rule update. * Adjust validators. * Add more tests to promotion rule update. * Add dataloader. * Fix tests. * Test 'isGift' field. * Rename 'gifts' to 'giftIds'. * Adjust mutations after renaming. * Restore previous name in case of inputs. * Add PREVIEW_FEATURE flags. * Validate gift lines. (#15326) * Do not allow updating 'OrderLine.quantity' when line is a gift. * Do not allow updating 'CheckoutLine.quantity' when line is a gift. * Do not allow deleting gift from checkout. * Do not allow deleting gifts from checkout. * Extend error with line ids. * Do not allow deleting gifts from order. * Fix test. * Add missing test. * Fix flaky test. * Do not allow to update gift lines at all, not only its quantity; change error codes. * Simplify validation. * Refactor flaky tests. * Include gift promotion in calculation (#15278) * Create discount object for gift reward * Create checkout line for gift reward * Update gift line creation * Handle race condition for gift line creation * Create CheckoutLineDiscount instead of CheckoutDiscount for gift reward * Gift reward calculation adjustment * Update complete_checkout for gift promotion * Add tests for checkout total calculations and CheckoutLinesAdd mutation * Add tests for webhooks * Add tests for checkout claculations * Add tests for avatax * Update create_discount_objects_for_catalogue_promotions * Update discount/utils.py * Add missing tests for _get_best_gift_reward * Clean duplicated isGift on CheckoutLine * Add more tests * Update gift line removal * Ensure that the gift line is created and returned when the gift reward is applicable during CheckoutCreate mutation * Rename invalidate_checkout_prices to invalidate_checkout * Ensure gift line is created in CheckoutCreateFromOrder mutation * Apply code review suggestions * Update giftreward calculations * Fix checkout calculations with catalogue and gift reward * Handle multiple gifts in _delete_gift_line * Move checkout creation logic to checkout mutation utils * Do not delete checkout line discounts obejcts in loop * Adjust handling gift reward in calculations * Set gifts limit per promotion rule (#15353) * Set gifts limit per promotion rule * Apply code review suggestions * Clear gift line when not applicable (#15335) * Clear gift line in _clear_checkout_discount * Add tests for CheckoutAddPromoCode and CheckoutRemovePromoCode * Fix test * Adjust gift line removing * Apply code review suggestion * Adjust gifts input in `PromotionRuleUpdateInput` (#15358) * Update gifts input in PromotionRuleUpdateInput * Add tests to gift validators * Benchmark tests after implementing order promotions. (#15359) * Add tests for checkout mutations. * Add benchmarks for promotoin mutations. * Add checks if discount was created. * Appply review comments. * Fix tests * Update changelog * Apply code review suggestions --------- Co-authored-by: Piotr Zabieglik <55899043+zedzior@users.noreply.github.com>

zedzior added 2 commits February 5, 2024 19:13

Add tests for checkout mutations.

978bad7

Add benchmarks for promotoin mutations.

c752263

zedzior self-assigned this Feb 5, 2024

zedzior added the promotions label Feb 5, 2024

Add checks if discount was created.

4272613

zedzior linked an issue Feb 5, 2024 that may be closed by this pull request

Add benchmarks for order promotions #15350

Closed

zedzior requested review from IKarbowiak and korycins February 5, 2024 19:34

IKarbowiak reviewed Feb 6, 2024

View reviewed changes

Appply review comments.

fdfbe98

zedzior requested a review from IKarbowiak February 6, 2024 11:15

IKarbowiak approved these changes Feb 6, 2024

View reviewed changes

korycins reviewed Feb 7, 2024

View reviewed changes

saleor/graphql/discount/tests/benchmark/test_promotion_create.py Show resolved Hide resolved

korycins approved these changes Feb 7, 2024

View reviewed changes

zedzior merged commit a4e2ffa into feature/gift-promotions Feb 7, 2024
6 checks passed

zedzior deleted the 15350-benchmark-tests branch February 7, 2024 11:52

IKarbowiak pushed a commit that referenced this pull request Feb 7, 2024

Benchmark tests after implementing order promotions. (#15359)

ae7172b

* Add tests for checkout mutations. * Add benchmarks for promotoin mutations. * Add checks if discount was created. * Appply review comments.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmark tests after implementing order promotions. #15359

Benchmark tests after implementing order promotions. #15359

zedzior commented Feb 5, 2024

db-queries bot commented Feb 5, 2024 •

edited

Loading

codecov bot commented Feb 5, 2024 •

edited

Loading

Ashir40 commented Feb 7, 2024 via email

Benchmark tests after implementing order promotions. #15359

Benchmark tests after implementing order promotions. #15359

Conversation

zedzior commented Feb 5, 2024

Impact

Docs

Pull Request Checklist

db-queries bot commented Feb 5, 2024 • edited Loading

codecov bot commented Feb 5, 2024 • edited Loading

Codecov Report

Ashir40 commented Feb 7, 2024 via email

db-queries bot commented Feb 5, 2024 •

edited

Loading

codecov bot commented Feb 5, 2024 •

edited

Loading