Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add additional Rewards metrics #38567

Closed
DJAndries opened this issue May 24, 2024 · 3 comments · Fixed by brave/brave-core#23833
Closed

Add additional Rewards metrics #38567

DJAndries opened this issue May 24, 2024 · 3 comments · Fixed by brave/brave-core#23833
Assignees
@DJAndries
Labels
feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) feature/rewards features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Milestone
1.68.x - Release

Comments

@DJAndries
Copy link
Collaborator

DJAndries commented May 24, 2024 •
edited
Loading

Add the following Rewards metrics:

If you have rewards enabled, which ad types do you have enabled?
Brave.Rewards.AdTypesEnabled.2

  1. None
  2. NTP only
  3. Push only
  4. NTP + Push
  5. Search ads only
  6. Search + NTP
  7. Search + Push
  8. Search + NTP + Push

If you have Rewards enabled and a payout account connected, did you enable search ads this week? (desktop only)
Brave.Rewards.SearchResultAdsOptin

  1. Yes

If you are a Rewards user, did you view the 30-day Ads history page in the past week? (desktop only)
Brave.Rewards.AdsHistoryView

  1. Yes

cc @Miyayes
@DJAndries DJAndries added feature/rewards QA/Yes release-notes/exclude OS/Android Fixes related to Android browser functionality features/P3A OS/Desktop feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) labels May 24, 2024
@DJAndries DJAndries self-assigned this May 24, 2024
@DJAndries DJAndries mentioned this issue May 24, 2024
Merged
24 tasks
@brave-builds brave-builds added this to the 1.68.x - Nightly milestone May 31, 2024
@GeetaSarvadnya GeetaSarvadnya added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Jun 11, 2024
@GeetaSarvadnya
Copy link

@btlechowski: Could you add the test plan to trigger search ads? Also, is there any option to enable/disable the search ads?

cc: @LaurenWags

@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Jun 11, 2024 •
edited
Loading

Verification PASSED on

Brave | 1.68.85 Chromium: 126.0.6478.40 (Official Build) beta (64-bit)
-- | --
Revision | 8e76a669cf78bdfa933c8d569460092a01166588
OS | Windows 10 Version 22H2 (Build 19045.4412)

Brave.Rewards.AdTypesEnabled.2

Brave.Rewards.AdTypesEnabled.2 = 0_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Disable NTP ads, Search ads and Notification ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 0 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 1_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads enabled, disable Search ads and Notification ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 1 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 2_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Notification ads enabled, disable Search ads and New Tab page ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 2 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 3_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads and New Tab page ads enabled, disable Search ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 3 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 4_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Search ads enabled, disable Notification ads and New tab page ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 4 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 5_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads and Search ads enabled, disable Notification ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 5 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 6_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Notification ads and Search ads enabled, disable New tab page ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 6 as expected
Example Example
image image
Brave.Rewards.AdTypesEnabled.2 = 7_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://local-state file
  4. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  5. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 7 Search + NTP + Push as expected
    image

Brave.Rewards.SearchResultAdsOptin

Brave.Rewards.SearchResultAdsOptin = Yes_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards and connect a custodian
  3. Enable Search ads via brave://rewards
  4. Open brave://local-state file
  5. Look for the metrics Brave.Rewards.SearchResultAdsOptin
  6. Confirmed that the metrics response value is 1 as expected
Example Example
image image

Brave.Rewards.AdsHistoryView

Brave.Rewards.AdsHistoryView = Yes_PASSED

Case 1: No ads view

  1. Clean profile 1.68.x
  2. Open brave://local-state file
  3. Look for the metrics Brave.Rewards.AdsHistoryView
  4. Confirmed that the metrics Brave.Rewards.AdsHistoryView isn't shown on the local state file as expected

Case 2: View Push/NTP Ads

  1. Clean profile 1.68.x
  2. Enable Rewards and view NTP and Push notifications
  3. Ensured that ads viewed count is shown on the brave://rewards page
  4. Open brave://local-state file
  5. Look for the metrics Brave.Rewards.AdsHistoryView
  6. Confirmed that the metrics Brave.Rewards.AdsHistoryView isn't shown on the local state file as expected
  7. Click on the 30-days Ads History link
  8. Reload brave://local-state file
  9. Look for the metrics Brave.Rewards.AdsHistoryView
  10. Confirmed that the metrics Brave.Rewards.AdsHistoryView response value is 1 as expected
Example Example Example
image image image

Case 3: View Inline content Ad via Brave News

  1. Clean profile 1.68.x
  2. Open an NTP and enable Brave News
  3. View an inline content ad
  4. Open brave://local-state file
  5. Looks for the metrics Brave.Rewards.AdsHistoryView
  6. Confirmed that the metrics Brave.Rewards.AdsHistoryView isn't shown as expected
  7. Enable Rewards
  8. Open an NTP and view an inline content ad
  9. Open brave://local-state file
  10. Looks for the metrics Brave.Rewards.AdsHistoryView
  11. Confirmed that the metrics Brave.Rewards.AdsHistoryView isn't shown as expected
  12. Click on the 30-days Ads History link
  13. Reload brave://local-state file
  14. Look for the metrics Brave.Rewards.AdsHistoryView
  15. Confirmed that the metrics Brave.Rewards.AdsHistoryView response value is 1 as expected
Example Example Example
image image image

@GeetaSarvadnya GeetaSarvadnya added QA Pass-Win64 and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Jun 20, 2024
@GeetaSarvadnya GeetaSarvadnya added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Jul 2, 2024
@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Jul 2, 2024 •
edited
Loading

Verification PASSED on Vivo X70 Pro version 12 running Bravemonoarm64.apk_1.68.101

Brave.Rewards.AdTypesEnabled.2

Brave.Rewards.AdTypesEnabled.2 = 0_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Disable NTP ads, Search ads and Notification ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 0 as expected
Example Example
Screenshot_20240702_151624 Screenshot_20240702_151617
Brave.Rewards.AdTypesEnabled.2 = 1_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads enabled, disable Search ads and Notification ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 1 as expected
    Screenshot_20240702_151751
Brave.Rewards.AdTypesEnabled.2 = 2_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Notification ads enabled, disable Search ads and New Tab page ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 2 as expected

Screenshot_20240702_151842

Brave.Rewards.AdTypesEnabled.2 = 3_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads and New Tab page ads enabled, disable Search ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 3 as expected
Example Example
Screenshot_20240702_151928 Screenshot_20240702_151915
Brave.Rewards.AdTypesEnabled.2 = 4_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Search ads enabled, disable Notification ads and New tab page ads switches
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 4 as expected
Example Example
Screenshot_20240702_151940 Screenshot_20240702_151959
Brave.Rewards.AdTypesEnabled.2 = 5_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep NTP ads and Search ads enabled, disable Notification ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 5 as expected
Example Example
Screenshot_20240702_152033 Screenshot_20240702_152047
Brave.Rewards.AdTypesEnabled.2 = 6_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://rewards page
  4. Keep Notification ads and Search ads enabled, disable New tab page ads switch
  5. Open brave://local-state file
  6. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  7. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 6 as expected
Example Example
Screenshot_20240702_152108 Screenshot_20240702_152127
Brave.Rewards.AdTypesEnabled.2 = 7_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards
  3. Open brave://local-state file
  4. Look for the metrics Brave.Rewards.AdTypesEnabled.2
  5. Confirmed that the metrics Brave.Rewards.AdTypesEnabled.2 response value is 7 Search + NTP + Push as expected
Example Example
Screenshot_20240702_152408 Screenshot_20240702_152348

Brave.Rewards.SearchResultAdsOptin

Brave.Rewards.SearchResultAdsOptin = Yes_PASSED
  1. Clean profile 1.68.x
  2. Enable rewards and connect a custodian
  3. Enable Search ads via brave://rewards
  4. Open brave://local-state file
  5. Look for the metrics Brave.Rewards.SearchResultAdsOptin
  6. Confirmed that the metrics response value is 1 as expected
Example Example Example
Screenshot_20240702_153758 Screenshot_20240702_153804 Screenshot_20240702_153907

Brave.Rewards.AdsHistoryView

Note: This metrics is applicable only for Desktop as Android doesn't have ads history via brave://rewards page

@GeetaSarvadnya GeetaSarvadnya added QA Pass - Android ARM and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Jul 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DJAndries DJAndries

Labels
feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) feature/rewards features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Projects
Brave Rewards
Status: Done
Milestone
1.68.x - Release
Development

Successfully merging a pull request may close this issue.

Add additional Rewards metrics brave/brave-core
3 participants
@DJAndries @GeetaSarvadnya @brave-builds