Email templating

Introduction
Domain glossary
Resources
Tickets
First Implementation Considerations
Contexts and Scenarios to Come
- Aperta Email Templating Status

Introduction

The goal of the email templating system is to move existing hard-coded email text into the database. The end users (staff) will have the ability to change the templates to suit the current business needs and messaging. To ensure a robust system, there are concepts to codify and plans to make. Here we have the notes and decisions from planning sessions and implementation work.

Please remember that this is a Wiki and editable. If you find a mistake, do not hesitate to correct it!

Domain glossary

Template - a stream of text containing static content and {{merge.fields}}.

Technically this is the template source that is compiled to create a Liquid::Template.
Liquid compiles the template source and then renders it with data coming from a Scenario object. It replaces merge fields with values derived from that scenario.

Merge Field - an identifier that can be put into a template, to be replaced when the template is rendered.

The value of the merge field is retrieved by calling the merge field as a method on the TemplateScenario object that is passed in to the template render method.
A merge field can represent a chain of method calls, dot delimited. example: {{ manuscript.editor.last_name }}

Scenario - Identifies the list of available merge fields that can be used in a template.

Passed to Liquid::Template.render()
Contains multiple contexts that are relevant to the given use case.
Generic scenarios can handle most cases
- these include ManuscriptScenario, ReviewerReportScenario, and InvitationScenario
Specialized scenarios can be created to handle data that would not be useful in a general scenario
- such as tech check sendback reasons

Context - a class that provides data to replace merge fields when a template is rendered. Scenarios are composed of Contexts.

Inherits from Liquid::Drop to facilitate lazy loading of data.

*Use Case - *a conceptual location or action in the application where a liquid template will be rendered.

This is considered "downstream" usage and does not modify the template itself.
This may occur in a user-facing view, where for example the template might be rendered and then edited by staff before being sent.
Or it could be a background job that sends emails without review

Resources

Liquid markup documentation
- To see a lot of this on a single page: Liquid for Designers
Liquid rubydoc
Important Liquid classes that we use directly:
- Liquid::Template - Compiles template source and renders it in a scenario.
- Liquid::Drop - The base class for our TemplateContext. It is an alternative to using a simple Hash as a context. It's methods are lazy-loaded.
Email Template Administration for Users

First Implementation Considerations

The implementation is broken into two major parts – defining Contexts and Scenarios. The latter are the collections of top-level merge fields that will exist for a template writer. If a staff person wants to have a template say "{{ journal.name }}", then the `journal` object needs to reside at the top-level of the scenario (or environment) passed into the render function. These scenarios will be built on root objects available in the respective templating workflows.

In this first step, two scenarios covering two main use cases: `PaperReviewerScenario` and `RegisterDecisionScenario`. These provide the data for the reviewer invitation and decision letter templates respectively. The reviewer invitation is used when inviting reviewers to the paper. The rendered template is sent to a user. The decision letter is similar and has a rendered body sent to the user. These two are also different from others we will find in the system because the rendered template bodies are presented to a staff user in Aperta before sending. This means that for these cases, the rendered template may be interactively modified (without changing the template) before sending to the user.

For the templating system (Liquid) to traverse down the object relationships, contexts need to be added that wrap the underlying models. `Invitation`, for example, will have an `InvitationContext` that wraps the underlying `invitation.paper` in a `PaperContext`. These expose the available data to the renderer. In many cases, this is easy and straightforward. In some, there is a bit of processing to do so that the data is easy to grab and use for the template authors.

All of these contexts stem from a base, `TemplateContext`. This provides a bit of metaprogramming to whitelist attributes and make the data available to the template. As we move forward, this can be expanded to provide discoverability of which "merge fields" can be used on the templates. It is easy to imagine how this would contribute to the front end through either a palette of available merge fields or even auto-completion in the template editor.

This initial implementation was about installing Liquid and making the templates render correctly. We will have some follow-on work to put the templates into the database and make them viewable and editable.

Current Questions and Concerns

Keep existing mailer views?
1. Most templates are body sections?
2. Templatize 'to' and 'subject' fields? not 'to' only 'subject'
How do we allow HTML in templates?
1. When do they get sanitized/rendered? Taken care of by tinymce stuff
How do we provide dummy data for template writers?
1. Show a preview of the fields they used. Keep sample data (in table or code in class)
2. Show the whole email through the view
How/when do we validate templates? validate at template-edit-time that the user used available merge fields.
1. Documentation of Liquid's error modes: http://www.rubydoc.info/gems/liquid/file/README.md
How do we message on template errors?
1. Do we need to let it render (in a broken way) sometimes for debug?
How do we organize templates for selection and editing? Trifecta: each letter_templates row will belong to a scenario, optionally specify a "type" (ex decision verdict), and template name.
Keep items like accept/decline in the mailer view?
We need to collect and present errors that happen when an email is rendered in the background.

There may also be concerns around permissions. Do we need to change contexts/scenarios/templates based on permissions anywhere?

Contexts and Scenarios to Come

Building from the Emails sent from Aperta page, we have a table in the Appendix Section with some notes about the contexts and scenarios around the current mailers. But first, let's discuss how to integrate with the mailers. Most (all?) of these mailers have associated views.

The implemented scenarios have one major feature in common – the mailer views are just thin wrappers around a given body. The decision letters share a mailer, but swap out the body contents for the appropriate messaging. Several of the other mailing situations rely more on some of the elements in the view for the content. For uniformity (and to take full advantage of the templates), we may want to make most of the mailer views act like shells where we can insert a message body. For specialized cases like invitations, we will maybe want to keep the links (accept/decline) in the view and not push that to the template side.

A list of emails to be converted (scenarios created) follows – broken down by recipient and function.

To Admins
1. Services
  1. FTP Uploader
  2. Billing Salesforce
  3. Notify Author of Changes Service
2. Events
  1. Feedback
  2. Paper Withdrawl
  3. Invitation Accepted
  4. Invitation Declined
To Users
1. Invitations
  1. Invite Reviewer *
  2. Invite Editor *
2. Collaborations
  1. Create Collaboration
3. Discussions
  1. User Mention
  2. Add Participant
4. Paper Status Change
  1. Notify Creator (author)
  2. Notify Coauthor
  3. Initial Decision
  4. Register Decision *
  5. Changes for Author

Appendix

Aperta Email Templating Status

Mailers	Scenario	Migration Priority	Scenario Needed?	Scenario Notes	Context Elements	Recipient	Sender	Called by
reviewer_mailer.rb notify_invited	Invitation	1 2 3			Journal Manuscript (Paper) Invitation Invitee (User) Inviter (User)	invitation.email, apertachasing@plos.org hardcoded bcc Not editable	Aperta	paper_reviewer_task.rb#invitation_invited
reviewer_accepted	Invitation	1 2 3	Maybe	It doesn't look like the task is used in the template – only as a way to get to the paper. Also, we will need to expose the `token` field for InvitationContext This is to the staff (or inviter).	Invitation Inviter (as User) InviteReviewerTask Paper Journal Reviewer (as User)	inviter	Aperta	paper_reviewer_task.rb#invitation_accepted
reviewer_declined	Invitation	1 2 3	Maybe	This is mostly the same as the accepted case and probably the same context.	Invitation Inviter/Assigner (as User) Paper Journal Assignee/Reviewer (as User)	inviter	Aperta	paper_reviewer_task.rb#invitation_declined
welcome_reviewer	ReviewerReport	1 2 3				invitee	Aperta	reviewer_report_task_creator.rb#find_or_create_related_task calls reveiwer_mailer#welcome_reviewer
remind_before_due	Reviewer Report	1 2 3	no		Reviewer Report	reviewer	Aperta	ReivewerMailer#remind_before_due
first_late_notice	Reviewer Report	1 2 3	no		Reviewer Report	reviewer	Aperta	ReivewerMailer#first_late_notice
second_late_notice	Reviewer Report	1 2 3	no		Reviewer Report	reviewer	Aperta	ReivewerMailer#second_late_notice
thank_reviewer	Reviewer Report	1 2 3	no		Reviewer Report	reviewer	Aperta	ReviewerMailer#thank_reviewer
register_decision_mailer.rb notify_author_email	Register Decision	1 2 3	no	The body of this one is stored in the database as a decision letter. In the Register Decision task, an editor will choose a LetterTemplate associated with the RegisterDecisionScenario. The template is then rendered in the context of the paper and the rendered content is then edited in a TinyMCE editor. One of these has already been converted to the Liquid syntax and these are already in the database.	Decision Paper Author (paper.creator)	paper.creator	Aperta	register_decision_task.rb#send_email
generic_mailer.rb send_email		1 2 3	Yes	Exceptions may need to have a wrapper. The other two elements here are simple strings	filename Exception ftp message	Staff admins	Aperta (billing_ftp, apex_service)	ftp_uploader_service.rb#notify_admin
send_email		1 2 3	No	This is the basic mailer for tasks the :subject and :body params should have the template applied :recipients may also need to run through the templater	(none)	All users listed in the task	Aperta(ad hoc task)	tasks_controller.rb#send_message
feedback_mailer.rb contact		1 2 3	Maybe	This mails feedback to a the staff. This likely does not need a template.	feedback (string) subject?	ADMIN_EMAIL environment variable	user who filled feedback	feedback_controller.rb#create
user_mailer.rb add_collaborator		1 2 3	Yes		Paper as manuscript User (as invitee) Journal	collaborator	logged in user	collaborations_controller.rb#create
notify_staff_of_paper_withdrawal		1 2 3	Yes	Withdrawl needs a context with reason, withdrawn_by_user (as user or author) The lists of editors and reviewers are pulled from paper. It may be nice to have these available at the top-level of the template.	Paper as manuscript Authors (corresponding authors) Journal Withdrawl (paper.latest_withdrawl) HandlingEditors CoverEditors AcademicEditors Reviewers	Email populated in `staff_email` for that journal	Aperta	papers_controller.rb#withdraw
mention_collaborator		1 2 3	Yes	(mention_collaborator view)	Comment User (commenter and commentee) Task Paper Journal	comentee	commenter	comment.rb#notify_mentioned_people
add_participant		1 2 3	Yes	(add_participant view)	Task Paper Journal User (assignee and assigner)	assignee	assigner	task.rb#notify_new_participant, participation_factory.rb#send_notification
notify_added_to_topic		1 2 3	Yes	(notify_added_to_topic view)	Paper User (invitor, invitee) DiscussionTopic	participant	Aperta	subscribers/../email_new_participant.rb#call
notify_mention_in_discussion		1 2 3	Yes	(notify_mention_in_discussion_view)	Paper User DiscussionTopic DiscussionReply	mentionee	Aperta	subscribers/../email_people_mentioned.rb#call
notify_creator_of_revision_submission notify_creator_of_check_submission notify_creator_of_initial_submission notify_creator_of_paper_submission		1 2 3	Yes	There are four different notifications here that follow the same "shape" and have the same data. Only the submission state differs and is captured in the table above	Paper User (as author) Journal	author	Aperta	subscribers/../email_creator.rb#call
add_editor_to_editors_discussion		1 2 3	Yes	Oddly, there is no discussion topic here. It is invited editors to the discussion, but only at the task level.	Task User (as invitee) Paper	invitee (editor)	Aperta	editors_discussion_task.rb#notify_new_participant
notify_coauthor_of_paper_submission		1 2 3	Yes	These emails are sent to each coauthor, so that record is brought to the top. There is also a recommendations_url in the view for this one.	Paper Journal Coauthor (From author list) Authors (user list)	coauthor	Aperta	subscribers/../email_coauthors.rb#call
billing_salesforce_mailer.rb notify_site_admins_of_syncing_error		1 2 3	Maybe	This relies on a message passed in There are two cases, but these get filled in with code variables. I don't think this requires a template.	Paper Admin Emails message	journal_admin_emails (paper.journal.admins)<–may change after old roles ripped out	Aperta	salesforce_manuscript_update_worker.rb#email_admin_error
initial_decision_mailer.rb notify		1 2 3	Yes	This will be similar to the register decision method. Or maybe the same?	Decision Paper Creator (as Author)	paper creator	Aperta	initial_decision_task.rb#send_email
paper_editor_mailer.rb notify_invited	InvitationScenario	1 2 3	Yes	Mailer uses invitation.body	Invitation Invitee (User) Paper Journal Task	invitation.email, apertachasing@plos.org hardcoded bcc'ed	Aperta	paper_editor_task.rb#invitation_invited
changes_for_author_mailer.rb notify_changes_for_author		1 2 3	Yes	This could be an interesting one. There is the overall template text that could be converted. There is also a section that inserts a `task.letter` portion. That may require an independent template.	Author Task Paper Journal	paper.creator	Aperta	changes_for_author_task.rb#notify_changes_for_author
notify_changes_for_author		1 2 3	Yes	This could be an interesting one. There is the overall template text that could be converted. There is also a section that inserts a `task.letter` portion. That may require an independent template.	Author Task Paper Journal	paper.admins	Aperta	notify_author_of_changes_needed_service.rb#notify_tech_fixed

Current Scenarios and Merge Fields (as of 1/10/2018)

Manuscript, Preprint Decision	Reviewer Report	Invitation, Paper Reviewer	Decision	Tech Check
journal __logo_url __name __staff_email manuscript __abstract __academic_editors[] ____email ____first_name ____full_name ____last_name __authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __corresponding_authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __creator ____email ____first_name ____full_name ____last_name __editor ____email ____first_name ____full_name ____last_name __handling_editors[] ____email ____first_name ____full_name ____last_name __paper_type __title __url	journal __logo_url __name __staff_email manuscript __abstract __academic_editors[] ____email ____first_name ____full_name ____last_name __authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __corresponding_authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __creator ____email ____first_name ____full_name ____last_name __editor ____email ____first_name ____full_name ____last_name __handling_editors[] ____email ____first_name ____full_name ____last_name __paper_type __title __url review __answers[] ____ident ____question ____string_value ____value ____value_type __datetime __due_at __due_at_with_minutes __invitation_accepted? __rendered_answer_idents __rendered_answers __reviewer ____email ____first_name ____full_name ____last_name __reviewer_name __reviewer_number __revision __state __status reviewer __email __first_name __full_name __last_name	invitation __decline_reason __due_in_days __reviewer_suggestions __state invitee __email __first_name __full_name __last_name invitee_name_or_email inviter __email __first_name __full_name __last_name journal __logo_url __name __staff_email manuscript __abstract __academic_editors[] ____email ____first_name ____full_name ____last_name __authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __corresponding_authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __creator ____email ____first_name ____full_name ____last_name __editor ____email ____first_name ____full_name ____last_name __handling_editors[] ____email ____first_name ____full_name ____last_name __paper_type __title __url	journal __logo_url __name __staff_email manuscript __abstract __academic_editors[] ____email ____first_name ____full_name ____last_name __authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __corresponding_authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __creator ____email ____first_name ____full_name ____last_name __editor ____email ____first_name ____full_name ____last_name __handling_editors[] ____email ____first_name ____full_name ____last_name __paper_type __title __url reviews[] __answers[] ____ident ____question ____string_value ____value ____value_type __datetime __due_at __due_at_with_minutes __invitation_accepted? __rendered_answer_idents __rendered_answers __reviewer ____email ____first_name ____full_name ____last_name __reviewer_name __reviewer_number __revision __state __status	author __email __first_name __full_name __last_name footer intro journal __logo_url __name __staff_email manuscript __abstract __academic_editors[] ____email ____first_name ____full_name ____last_name __authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __corresponding_authors[] ____affiliation ____author_initial ____email ____first_name ____last_name __creator ____email ____first_name ____full_name ____last_name __editor ____email ____first_name ____full_name ____last_name __handling_editors[] ____email ____first_name ____full_name ____last_name __paper_type __title __url paperwide_sendback_reasons sendback_reasons[] __ident __question __string_value __value __value_type