Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Risk score recording #5708

Open
garethbowen opened this issue Jun 4, 2019 · 9 comments
Open

Risk score recording #5708

garethbowen opened this issue Jun 4, 2019 · 9 comments
Labels
Priority: 2 - Medium Normal priority Type: Feature Add something new

Comments

@garethbowen
Copy link
Member

The ability to store risk scores when they’re updated in a server-side doc.

Right now, there is no way to know which individuals the app has identified as high risk. This makes it incredibly challenging to monitor and evaluate our initiative. Currently, our analysis plans require us to look at task completion rates for high risk tasks. To calculate this, we have to reproduce who the app thinks is high risk, and then also reproduce the rules for task generation. This gets even more confusing because people's risk status can change over time. Many of the concepts we want to evaluate require knowing who is high risk.

There are also other requests that have come up for accessing this information including:

  • supervisors being able to see how many high risk patients each CHW is responsible for

A note/idea from Matt: Creating a doc similar to this which is stored only server side so that we can use it in analytics to know when tasks are created and don’t have to recreate the calculations in postgres/elsewhere.

Original issue

@garethbowen garethbowen added Type: Feature Add something new Priority: 2 - Medium Normal priority labels Jun 4, 2019
@garethbowen garethbowen added this to To do in 3.8.0 via automation Jun 4, 2019
@MaxDiz
Copy link
Contributor

MaxDiz commented Jun 5, 2019

clarification on request - ideal functionality would be to track risk scores over time. From @yembrick

When we go back to do analysis, we need to know who was high risk at different points in time, as a person's risk status can change.

@christinewere
Copy link

Hi @garethbowen @MaxDiz this was part of the iteration 5 for the PA initiative and we had committed to a timeline of end of Sept. @yembrick .The initiative is coming to a closeout in November can we try and get this ready sooner? What is the new timeline we can communicate to the LG about this? cc @PhilipNgari @SMurithi

@yembrick
Copy link

yembrick commented Oct 4, 2019

Hi @christinewere . @garethbowen and @ecsalomon are working on a 1 off solution that will allow better verification of risk scores for the end of the pilot. This will ensure we get the data we need, and will give the product team time to not rush this feature out. I don't know the timeline of that workaround, @ecsalomon or @garethbowen should be able to answer that better.

@ecsalomon
Copy link

This is the issue for the alternative we are implementing https://github.com/medic/medic-projects/issues/6951

@MaxDiz MaxDiz added this to Needs Triage in Data Scientists Backlog via automation Oct 8, 2019
@MaxDiz MaxDiz moved this from Needs Triage to In progress in Data Scientists Backlog Oct 8, 2019
@garethbowen
Copy link
Member Author

Now that we have a temporary solution for the pilot there is no urgent need for this, and it will potentially be easier to implement once the tasks and targets refactoring is complete so I'm moving this to 3.9.0.

@garethbowen garethbowen added this to To do in 3.9.0 via automation Nov 3, 2019
@garethbowen garethbowen removed this from To do in 3.8.0 Nov 3, 2019
@garethbowen garethbowen removed their assignment Nov 3, 2019
@garethbowen garethbowen removed this from To do in 3.9.0 Dec 5, 2019
@garethbowen garethbowen added this to To do in 3.11.0 via automation Dec 5, 2019
@MaxDiz MaxDiz added this to New: Proposed by Backlog Owners in CHT Product Roadmap: Initiatives 2021 via automation Dec 19, 2019
@MaxDiz MaxDiz moved this from New: Proposed by Backlog Owners to Scheduled: Assigned to a release in CHT Product Roadmap: Initiatives 2021 Dec 19, 2019
@ecsalomon
Copy link

ecsalomon commented Mar 13, 2020

Thinking about capturing scores over time and scores that do not necessarily trigger tasks, there are a few implementations that may work (all of which rely on a minimum score recalculation window to control computation resources):

  • Keep a log of risk score-timestamp tuples as documents with a high purge frequency and keep only the most recent tuple in the patient profile
  • Track telemetry data for scores, including n_times_above_k as one of the metrics, keeping only the most recent score-timestamp tuple in the patient profile
  • Keep a rolling history of score-timestamp tuples in the patient profile (up to X scores) and use a backend solution to capture the history when the CHW syncs
  • Other ideas?

@ecsalomon
Copy link

More on this issue here

@garethbowen
Copy link
Member Author

garethbowen commented May 26, 2020

Moved to 3.12.0 to free up engineers to work on 3.11.0.

@MaxDiz MaxDiz removed this from Scheduled: Assigned to a release in CHT Product Roadmap: Initiatives 2021 Aug 11, 2020
@michaelkohn michaelkohn moved this from Ready for dev to In design in Dev team workstream Sep 22, 2020
@MaxDiz
Copy link
Contributor

MaxDiz commented Oct 9, 2020

Project timing has been delayed so removing from release.

@MaxDiz MaxDiz removed this from In design in Dev team workstream Oct 9, 2020
@benkags benkags removed their assignment Oct 22, 2020
@MaxDiz MaxDiz moved this from Scheduled to To do in Data Scientists Backlog Nov 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: 2 - Medium Normal priority Type: Feature Add something new
Projects
No open projects
Development

No branches or pull requests

6 participants