Ability to mute people and places

[amandacilek-zz]

In response to medic/medic-projects#4329

Full details and additional screenshots can be found in the design spec: https://docs.google.com/document/d/1we_Qu1B4x8PuAi9pxiIu4MeXtdH1kY9W29kMidHThsc/edit?usp=sharing

Summary of muting in general

• Muting is a form accessible from the normal form menu on profiles (or Reports)
• Muting is not an individual per-user preference. All users of an instance (both CHWs and nurses) stop receiving new tasks and SMS reminders for any muted person/place.
• The muted people/places are not deleted and all historical data for all muted people/places remains in the app
• Any currently enrolled schedules are not deleted or paused - just muted (quieted)
• Any condition cards should remain on the person/place profile
• All of the forms that would normally appear are still accessible from a muted person/place profile (as usual, forms are configurable per project)
• If a user begins any new form for a muted person/place that is not the "Unmute" form, they will first get a warning modal saying “This person belongs to a family that is currently muted. Are you sure you want to proceed?”
• A user cannot unmute a person/place from within any form, only from within the official “Unmute” form. If they see the above warning message while trying to submit a different form, they can either proceed knowing the person/place is still muted, or they can close that form, open up the “Unmute” form, and unmute first before submitting the other form.
• A project may support both death reporting and muting, they are not mutually exclusive
• Manager approval should not be required for muting, but, like death reporting, it should be made a configurable option for partners that want it
• Muting should work for both web-app and SMS workflows. We aren't managing "families" over SMS yet, though this is likely to be introduced at some point.

Muting households

• Muting a household mutes schedules attached to household-level workflows as well as all schedules for all people in the household. It serves as a shortcut to muting all individuals in a household.
• To mute a family, you navigate to the family profile, open up the action window, and select the “Mute Family” form
• If you access the “Mute Family” form from the Reports tab, you first need to select the family to mute from a dropdown of active families
• A question on the form asks “What is the reason for muting?” and provides options for the user to select from (exact questions and options to be configurable by partner)
• If the partner does not require a manager verification step, the family will instantly be muted as soon as the user clicks the “Submit” button on the form
• If the partner does require manager verification, a task will be sent to the manager. The family will remain active until the manager confirms the muting.
• All users stop receiving new tasks or SMS for any muted family or family members
• If there was an existing open task for a now muted person/place on the Tasks tab, that open task should disappear from both the Tasks tab and the muted person/place’s profile
• Muted household profiles are styled to reflect their muted status
• Muted households move to a new section at the bottom of the main People page list
• There is no such thing as an unmuted person in a muted household. Whenever muting is set at the household level, it applies to all individuals within that household.
• We technically can have a muted person in an unmuted household, but that is not the primary focus of this request (see individual muting ticket Update the on/off handling to mark an individual as muted #4768)
• A user may add a new person to a muted household. If they do, and choose to leave the family muted, the new person will also be automatically muted.

UI changes to main People page list

• A family that has been muted moves into a new “muted families” section at the bottom of the main people page list
• The muted families section only appears if there is at least one muted family
• The number in parenthesis reflects the number of families that are currently muted
• UI matches that of death reporting bar on family profiles: grey background (#E0E0E0), height 45px, black FontAwesome arrow () on the right side, text normal 16px
• Clicking on the “muted families” section will display a list of all the families that are currently muted
• Clicking again on a family name will take you to that family’s profile

UI changes to profiles

• Very similar to profile styling for death
• The person/place icon becomes grey (#A7A9AC)
• And we display the text “Muted” on the second line below the person/place name
• Any forms that were previously available on the profile remain available
• Instead of the “Mute person/place” form, the user now sees “Unmute person/place”

[abbyad]

Given that some of the above will be covered by configuration (eg the mute form, not generating Tasks for muted contacts), I'll try to summarize the changes/features needed, along with some questions before we implement. Once outstanding questions are answered we spawn new issues for individual parts of this issues, or just leave them all part of this one.

Configurable Field

An field in app_settings that can contain an array of form IDs that mute a contact, and another field for unmuting contacts. We already have a similar fields for muting SMS notifications:

"notifications": {
    "off_form": "OFF",
    "on_form": "ON",

It makes sense to combine this new muting with the existing muting of SMS notifications since the reasons for muting via SMS should also mean there shouldn't be Tasks in the app for that contact. We should first verify though that this is compatible with how projects use the existing SMS version. If that is the case, this field would need to change from accepting only a single form to also accepting an array of form fields.

Updating a doc

If a submitted report's form ID is on the mute list then the contact the report is about should get a muted flag (eg muted: true). Conversely when a report on the unmute list is submitted, the contact has the muted flag unset (eg muted: false, or remove the field). Note that we've often entertained a similar feature, that of a submitted report updating fields on the contact. Eg setting/updating a person's birthday via a SMS form. It would be a bonus if the way we implement this issue is generic enough or extendable to satisfy other doc updates.

Rejecting SMS if the person's place is muted

As mentioned in the issue, we cannot have an unmuted person/place in a muted place. We can avoid this situation in the webapp by setting form context accordingly. By SMS however, we'd need to be able to validate the unmute form by checking that no parent of the contact is muted. We likely either need to allow validation on notification reports, and ensure that fields for all levels of the lineage can be accessed, or perhaps just an event_type like patient_not_found eg muted_in_lineage.

Displaying muted contacts

Amanda has clearly laid out what a muted contact looks like in the list. Curious if this is necessary for the first version of this feature, or if we are better off with a simpler implementation (eg just graying out the contact in the contact list) until we get real project use and feedback of the feature.

Not showing Tasks for muted contacts

Done via configuration by checking for any muted parent in lineage.

Muting SMS notifications

Done via pre-exisiting feature to turn notifications off, if we can combine with that feature.

Modal warning prior to actions on muted contacts

When opening an action form for a muted contact, if the form is not an unmute form then show a modal before continuing eg “This person belongs to a family that is currently muted. Are you sure you want to proceed? Yes/No".

Some outstanding questions:

• Can we combine this muting with the existing muting of SMS notifications? Perhaps @ngamita or @derickl have some thoughts on this.
• Are muted contacts excluded from Targets? (This is dealt with in configuration, but good to know here)
• Do we need to group muted contacts together, or can they be shown as before but grayed out? Curious to hear more about the design perspective from @amandacilek, and also the dev effort for options.
• Ideally this seems like something we'd want working offline... but is that a requirement? If no, then this can be handled with a transition. If yes... then we need to figure something out!

[amandacilek-zz]

• In regards to Targets, I think we will want the muted contacts included in some but not included in others. Perhaps @michaelkohn could provide some suggestions here.
• The grouping of the muted families into a new section on the LHS is not absolutely necessary, although this is the preferred approach. If the LOE to achieve that is extraordinary we are fine with an alternative like just turning the icon and text of the family name grey while leaving them in their original location in the list.
• It is not a requirement to have this work offline. We are fine having this work "online" only at the moment, the same way death reporting currently works. Enock says, "The CHPs are trained to go online at the end of the day and sync data, so I believe once they go online the muting will take effect once it hits the server."

[michaelkohn]

Are muted contacts excluded from Targets? (This is dealt with in configuration, but good to know here)

In regards to Targets, I think we will want the muted contacts included in some but not included in others. Perhaps @michaelkohn could provide some suggestions here.

Agreed... I think this'll need to be decided on a target-by-target basis. For simple form counts, I think you will want to include them because the Mute feature should prevent submission of forms while the contact is muted anyway... and if they weren't muted at the time of form submission, they should be counted. However, I can definitely imagine some other scenarios where you wouldn't want to include them... perhaps in some denominators for expected home visits, for example.

I don't know exactly how Targets are configured, but I imagine some of the processing could also be pretty heavy if you have to look for muting. For example.... if you needed to calculate something like % visits within 24h and wanted to excluded certain muted contacts, you'll probably need to JOIN the relevant forms not only with the contact but with the contact's parent to see if the family has been muted. Just something to consider.

[abbyad]

Ok, thanks for the info. Since the muting will be on the place it will be easy to include or exclude it on a per widget basis. It means that we still need to process all contacts, and shouldn't exclude them from Nools just because they are muted. We'll be able to handle the muted contacts in config of targets pretty easily.

[michaelkohn]

Since the muting will be on the place

I assume you mean 'in addition to' the contact, not just the place

[michaelkohn]

Also... sorry I just noticed this comment

If a user begins any new form for a muted person/place that is not the "Unmute" form, they will first get a warning modal saying “This person belongs to a family that is currently muted. Are you sure you want to proceed?”

Which means my comment

...because the Mute feature should prevent submission of forms while the contact is muted anyway...

Is not necessarily true. Totally fine, just means we'll need to be more diligent in configuring targets and determining when to include/exclude things.

[abbyad]

Since the muting will be on the place

I assume you mean 'in addition to' the contact, not just the place

I wouldn't assume that, but gather that is what you want?

When muting a place (aka family) adding the muted flag to every person within place seems problematic as it would mean changes to more docs. I was thinking a person would be muted if they themselves, or any of their parents have the muted flag.
This allows us to:

• mute a place (aka family)
• mute a person
• mute a place and not allow a person within it to be unmuted

The muted flag would be available in the person' lineage docs.

[michaelkohn]

Since the muting will be on the place

I assume you mean 'in addition to' the contact, not just the place

I wouldn't assume that, but gather that is what you want?

Sorry... I'm OK with family muting only updating the clinic and not cascading those updates to the person docs. I just meant that you'd have to look at the clinic and the person to know whether or not the person was muted, and therefore whether or not to include a given person specific form in a count.

When you say any of their parents, I take that to mean this includes muting health_center, district_hospital, etc... I was thinking we were only addressing muting person and clinics. It would be awesome if we were allowing muting of health_centers.

[michaelkohn]

If a submitted report's form ID is on the mute list then the contact the report is about should get a muted flag (eg muted: true). Conversely when a report on the unmute list is submitted, the contact has the muted flag unset (eg muted: false, or remove the field).

1. I assume we will have the muted field on the "Place" (clinic) as well, since you can mute at that level.
2. My preference is to set muted: false rather than removing the field.
3. My preference is to have the muted field (defaulted to false) on all new places / contacts.
4. Ideally we would add the muted field to every existing place/contact when they upgrade to the version that includes this feature.

Some additional questions/comments:

1. For SMS projects, if you submit a P form (for example) for a muted contact, I assume we won't have the "are you sure" workflow. Will we simply accept the P and create the schedule?
2. Will we differentiate between ON/OFF and muting when it comes to message states?
3. If a schedule-creating form is submitted (and accepted) for a muted contact, I assume we will still create the schedule but automatically place it in the new mute status?
4. If possible/easy, I can imagine it might help some debugging if we stamp the contact/place's current mute status on the data_record.

[abbyad]

Collection of responses below:

you'd have to look at the clinic and the person to know whether or not the person was muted

Correct

would be awesome if we were allowing muting of health_centers.

I don't see a good reason why we wouldn't make it generic enough to mute any place level.

add the muted field to every existing place/contact when they upgrade to the version that includes this feature.

We are unlikely to add a migration for this due to the performance costs (eg rebuilding views, re-downloading all contacts). Given that the absence of a muted flag means that the place is not muted, what is the advantage of adding muted: false to places by default?

Answers to the additional listed questions:

1. For SMS projects, if you submit a P form (for example) for a muted contact, I assume we won't have the "are you sure" workflow. Will we simply accept the P and create the schedule?

Yes, I believe that is the best way to proceed.

2. Will we differentiate between ON/OFF and muting when it comes to message states?

The reports that turn notifications OFF set the state of scheduled messages to muted. I don't see a reason to have different states... did you have something in mind?

3. If a schedule-creating form is submitted (and accepted) for a muted contact, I assume we will still create the schedule but automatically place it in the new mute status?

That is a good point, and I would not have assumed that as it is not how it currently works -- but it is worth specifying as such. In config we should make sure that the response mentions that the patient is muted if that is the case.

4. If possible/easy, I can imagine it might help some debugging if we stamp the contact/place's current mute status on the data_record.

Would this be only at doc creation, eg contact_mute_state_at_creation_time, or updated whenever the contact's mute state changes, eg contact_mute_state?

[michaelkohn]

would be awesome if we were allowing muting of health_centers.

I don't see a good reason why we wouldn't make it generic enough to mute any place level.

Actually... I guess it's less about muting the place, but rather muting the contact of that place. What I'm specifically looking for is the ability to know which CHW's are still with the program and therefore should show up in dropdowns in analytics, for example. This was discussed in https://github.com/medic/medic-projects/issues/1894 . This is different than muting the place (health_center), which would have the net effect of muting all schedules beneath it. That isn't what I'm looking for, I just want the ability to mute a specific CHW.

We are unlikely to add a migration for this due to the performance costs (eg rebuilding views, re-downloading all contacts). Given that the absence of a muted flag means that the place is not muted, what is the advantage of adding muted: false to places by default?

The data model (and all our queries) will be much cleaner, not to mention easier to understand for the community, if we can simply check the value of a field that is known to exist rather than having to always check to see if fields exist first, and then check the value of them. As far as I am aware, we don't have any other situations where the existence of a field is conditional (other than fields being added later, but from that point on they always exist).

Based on the first part of your response, I recognize we'll have to first check for the existence of that field for projects that have upgraded from a previous version, but ideally all projects that start from this release onwards would have the field no matter what.

The reports that turn notifications OFF set the state of scheduled messages to muted. I don't see a reason to have different states... did you have something in mind?

Not necessarily... just wanted to understand the plan. This will make it pretty tricky to know why a particular schedule was muted (OFF message, individual muted, family muted, etc...), but you can figure it out by looking at the timeline of potential mute messages and comparing to the schedule date/time.

Would this be only at doc creation, eg contact_mute_state_at_creation_time, or updated whenever the contact's mute state changes, eg contact_mute_state?

I was specifically thinking of the first one (contact_mute_state_at_creation_time), but it's not a big deal because technically we can build the timeline of mute messages and I imagine it (creating schedules for muted people) will be pretty rare anyway.

[diannakane]

@abbyad @michaelkohn I'm trying to follow the convo you're having. These are some assumptions I have about our muting workflow. Can you confirm which are still true? cc @amandacilek

-- muting a household mutes the people in that household

-- unmuting a person in a household unmutes the household. this is to prevent us from having "active" people in hidden households. you can then individually mute members of the household.

-- I had expected that if you reported for a muted person, particularly if it is a schedule-creating form, this would prompt you to unmute the person. It sounds like that is not a given, and I'm curious about that. If a CHW is actively treating a person but choosing not to receive notifications about them (and hiding them from their list, and some targets), what is the reason for this? This seems to signal a different motivation for mute than what was requested (if a person moved away or doesn't want to receive services anymore) and worth deeper consideration before we allow it.

-- OFF and mute are functionally the same, but we haven't had a "OFF" form for households yet (I think in part because it was created for SMS projects and we don't have households on SMS yet). Continuing to use "OFF" for muting individuals (or even naming it that way for households) works for me, as this is how our users have been trained and are used to it.

-- In order to keep the scope of this issue in check, we hadn't designed this for other levels of the hierarchy yet or taken into consider unique rules that might affect muting a place like a branch and applying that to all the people in that place, i.e. CHPs. It makes sense to build it to work generically, but we'll want to think through that before we broadcast to teams that it can be used for that.

[amandacilek-zz]

First to note that family muting and individual muting have been written as separate issues. My understanding has been that family muting is the primary focus for right now (and for this ticket) and that enhancements to turn on/off into a true individual muting feature are nice to have but not required and not necessarily bundled together with this work.

1. Yes. Muting a household automatically mutes schedules attached to household-level workflows as well as all schedules for all people in the household.

2. Yes, if a user wants to unmute an individual in a muted household, the whole household will need to be unmuted. There is no such thing as an unmuted person in a muted household. BUT, I seem to recall discussing that the unmuting of the household needs to happen on the household profile, meaning I don't think you can do it directly from the person profile from the individual on/off form. So my understanding is the best we can do is warn them that the individual is muted, and recommend that the user exit, go back to the family profile, and unmute family before proceeding.

3. The plan I tried to describe is that: yes, whenever a user tries to start a form for someone who is muted, they will first get a warning dialogue message telling them that the person/family is currently muted. So the "prompt" is just a warning dialogue that requires the user to do additional actions. But we never consider Michael's point about SMS users...clearly they wouldn't get the pop-up warning, which does make me question our plan a bit...

[diannakane]

I keep trying to work this out - it seems that we're already supporting individual muting. What individual muting features are left in the backlog beyond what we can already do? Are you referring to changes such as showing muted people differently in the household list or selectively muting schedules on a profile?

[amandacilek-zz]

I believe this sentence is originally from Gareth: "we need to make an extension of that to also mark the patient muted and make the UI changes to their profile to reflect the “Muted” status". My interpretation of this was that it's not just a matter of the UI changes to the profile. I guess we currently don't "mark" the patient as muted in the way that we need to.

[diannakane]

Got it. That sounds like the field that Michael was suggesting. Muted: true/false.

[amandacilek-zz]

((My above 3 answers are based on my original understanding of the plan, may not reflect what Michael and Marc have been discussing. Curious to hear their thoughts. Also, I am clearly not the expert in how this should be handled on the back end - I wrote the original issue but I'm relying on Michael and Marc to fill in the gaps and give recommendations or suggestions if something needs to be changed))

[newtewt]

Yeah I think that would be good. So we could easily show when they were muted

[dianabarsan]

Thanks!

[Enock1990]

Hey @newtewt & co. I did a few tests and I think this works well for me. Just an observation/bug; after muting a Health Facility, then I click on a CHV Area I'm able to unmute the CHV Area, other CHV Areas plus the Health Facility.

[dianabarsan]

Merged into master, back ported to 3.3.x and 3.2.x.

[newtewt]

This looks good to me. I'm waiting for responses on feedback and will then close this out.

[michaelkohn]

From a data architecture perspective, this ended up being implemented as follows.

When a mute or unmute form (data_record) is submitted, this will update the appropriate place/person records in the medic DB and the corresponding -info record in the medic-sentinel database. The updates are done as follows:

medic DB

• Amuted field is added whenever a person or place record is muted
• The value of the muted field will be the timestamp of when the person/place was muted
• The timestamp used is when the sentinel transition was run on this document (ie when the muting/unmuting form was synced), NOT the timestamp of when the form was submitted.
• If an already muted person or place is muted again, nothing is updated. The muted field will retain the original timestamp
• When person or places are unmuted, the muted field is completely removed
• This applies to the following document types person, clinic, health_center, district_hospital
• To check (in SQL) whether or not a person/place is muted, you can simply use doc ? 'muted', this checks the existence of a key in JSON. If the key does not exist, doc ? 'muted' will evaluate to false

Example of a muted health_center

{
  "_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
  "_rev": "11-58ec8cb4a26de08bd12ea1ba80446a35",
  "parent": {
    "_id": "45cc55c5-302d-49ce-8edf-fe77a8e0f110"
  },
  "name": "Red Foreman Area",
  "geolocation": "",
  "contact": {
    "_id": "ef0d26b8-bf27-41f3-a9ac-f7613238b0c3",
    "parent": {
      "_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
      "parent": {
        "_id": "45cc55c5-302d-49ce-8edf-fe77a8e0f110"
      }
    }
  },
  "type": "health_center",
  "reported_date": 1542228245476,
  "muted": "2018-12-03T15:36:08.639Z"
}

medic-sentinel DB

The muting history for a person/place is accessible from the corresponding info document stored in the medic-sentinel DB.

• When a person/place is first muted, the muting_history key will be added
• The value of this key is an array of mute/unmute activities
• Each array element will include (1) whether it was muted or unmuted; (2) timestamp of when the activity occurred; and (3) the data_record that caused this record to be muted or unmuted.
• The timestamp used is when the sentinel transition was run on this document (ie when the muting/unmuting form was synced), NOT the timestamp of when the form was submitted.
• Like the docs in the medic DB, if the person/place is already muted, muting them again will not add or update anything in the muting_history. The same is true for unmuting.

Example muting history stored on the info doc

{
  "_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f-info",
  "_rev": "20-14c194aa7f039a72aee3367d4d778cbd",
  "type": "info",
  "doc_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
  "initial_replication_date": "2018-11-14T20:44:05.657Z",
  "latest_replication_date": "2018-12-03T15:36:10.617Z",
  "transitions": {},
  "muting_history": [
    {
      "muted": true,
      "date": "2018-11-29T14:31:57.250Z",
      "report_id": "7e09d2fb-0561-4806-84f7-0d362b959827"
    },
    {
      "muted": false,
      "date": "2018-11-29T21:47:07.020Z",
      "report_id": "749aaf7c-bee7-4fc1-bc49-7d0c7f445458"
    },
    {
      "muted": true,
      "date": "2018-12-03T15:27:08.287Z",
      "report_id": "b30945da-19cd-44e6-bd74-7b56e55a917a"
    },
    {
      "muted": false,
      "date": "2018-12-03T15:27:26.628Z", 
      "report_id": "94634b6b-2054-4422-85cb-34f7a75883e5"
    },
    {
      "muted": true,
      "date": "2018-12-03T15:36:08.639Z",
      "report_id": "526ee71b-4c90-499a-b8f7-2f4a9864aefb"
    }
  ]
}

Scheduled SMS messages

Nothing has changed. Review documentation about message states here

[abbyad]

This is a really helpful summary @michaelkohn! We should make sure it is persisted as documentation too. Can you add it to this doc, or a separate one in medic-docs/configuration?