Send VA Profile data on Email Notification Delivery status

[MackHalliday]

User Story - Business Need

VA Profile is interested in tracking email bounces so that they might offer data to their partners regarding the quality of email data/the likelihood of delivery success. VA Notify needs to send VA Profile the delivery status of ALL email notifications so VA Profile can track this information.

• Ticket is understood, and QA has been contacted (if the ticket has a QA label).

User Story

As a developer on the VA Notify team
I want to send real-time email delivery status updates to VA Profile
So that the VA system can improve the management and reliability of email data, thereby enhancing communication with veterans and maintaining the organization's email reputation.

Additional Info and Resources

• Meeting notes with VA Profile for requirements discussion
• #1297 Spike - Provide Email Delivery Status Data to VA Profile

Engineering Checklist

• Create POST request method in the VAProfileClient class to send delivery status data to VA Profile. The request body should mirror our current callback structure

  "id":"6ba01111-f3ee-4a45-9d04-234asdfb6abbb9a",  -- this is the notification id
  "reference":null,  
   "to":"",  -- this is the recipient's contact info
   "status":"delivered",  -- this will specify the delivery status of the notification
   "created_at":"2023-01-10T00:04:25.273410Z",  
   "completed_at":"2023-01-10T00:05:33.255911Z",  
   "sent_at":"2023-01-10T00:04:25.775363Z",  
   "notification_type":"SMS",   -- this is the channel/type of notification
   "status_reason": "", -- populated if there's additional context on the delivery status
   "provider":"pinpoint"
  

• Add the bearer token value in the secrets section of the task definitions that extracts values from the parameter store using valueFrom in the secrets section
• Hard code the path to their endpoint (example here)
• Modify process_ses_results Celery Task to use new POST method. The status of ALL email notifications should be sent to VA Profile.
• Implement error handling for new feature
• Add unit testing
• Add a feature flag in the Celery Tasks Template for all environments, set to False
• Update documentation

Acceptance Criteria

• When ANY email delivery status is processed, it should trigger a POST request to VA Profile with accurate and complete delivery status data in the DEV/PERF environments
  • SMS delivery data should NOT trigger a POST request to VA Profile
  • Email status data prior to the delivery attempt should NOT trigger a POST request to VA Profile
• The system should handle errors gracefully and log them appropriately.
• The feature should work seamlessly with existing infrastructure, with minimal need for new development.
• All changes should be well-documented

QA Considerations

• Verify that a delivery status triggers the POST request to VA Profile. This probably requires checking the logs. Work with developer to understand the logging.
• Attempt negative test cases.
• Verify the feature flag and parameters
• Take a look at the documentation.

[cris-oddball]

Hey team! Please add your planning poker estimate with Zenhub @EvanParish @kalbfled @ldraney @MackHalliday @mchlwellman

[kalbfled]

Clarification about TTL: We don't want to lookup the access token every time we need to make a request to VA Profile. We discussed using the TTL cache package for this purpose.