Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Caseflow OIT Metrics #12748

Closed
3 tasks done
alisan16 opened this issue Nov 19, 2019 · 0 comments
Closed
3 tasks done

Caseflow OIT Metrics #12748

alisan16 opened this issue Nov 19, 2019 · 0 comments
Assignees
@yoomlam
Labels
Eng: Data Mark when data has been backfilled, or the issue has any data integrity concerns Stakeholder: BVA Functionality associated with the Board of Veterans' Appeals workflows/feature requests Team: Delta 🔺 Type: Metrics or Reporting

Comments

@alisan16
Copy link
Contributor

alisan16 commented Nov 19, 2019 •
edited

Background/context

We are required to deliver these 7 metrics to OIT on the scheduled reporting frequency. Metrics were approved in June 2019.

Metrics listed on this VA Sharepoint site

Metric ID 2. Metric Description 3. Unit of Measure 4. Performance Measurement Category Mapping 5. Agency Baseline Capability 6. 2018 Target 7. 2019 Target 8. Measurement Condition 9. Reporting Frequency
1909040936 Caseflow - Percentage of all cases certified with Caseflow, which assists Agency Original Jurisdiction in uploading all required documents of record to a shared electronic file repository and automatically prepares the Certification of Appeal to transfer the appeal to the Board of Veterans’ Appeals. (Number of appeals transferred to the Board using Caseflow/Total number of appeals transferred to the Board (manual certifications of paper appeals and Caseflow certifications)) Strategic and Business Results 85.7 90 90 Over target Monthly
1909040937 Caseflow Dispatch - Percent of non-denial decisions with an EP created within 7 days. Caseflow facilitates the transfer of cases to the Regional Office following issuance of a Board of Veterans’ Appeals decision, with an end product (EP) created to track work. Not creating an EP increases the possibility of miscommunication and slows average processing times, delaying ratings adjustment and working of remand orders. (Number EPs with date<=7 days after outcoding date/number non-denial decisions) Strategic and Business Results 79.6 80 80 Over target Quarterly
1909040938 Increase Veteran show rates at Board hearings using Caseflow Hearing Schedule, which enables a standardized scheduling process providing the ability to schedule hearings at both regional offices and alternate locations closer to Veterans’ homes. (Total scheduled meetings held – postposed hearings/total scheduled meetings from Caseflow Hearing Schedule) Strategic and Business Results   N/A 75 Over target Quarterly
1909040939 Percentage of AMA reviews established within 7 days Date in Intake appeal established minus date entered Strategic and Business Results   N/A 90% Over target Monthly
1909040940 Adoption of Caseflow Reader for appeal decisions (percentage) Date of decision minus date of appeal establishment Customer Satisfaction (Results)   N/A 98% Over target Annual
1909040942 Average Customer Effort Score (CES) 5 out of 7 positive responses or better Customer Satisfaction (Results)   N/A 5 out of 7 or better Over target Annual
1909040943 Mean (average) time to recovery (minutes) Outage recovery time minus outage report time Innovation   N/A 30 minutes or less Under target Quarterly

Goals

Deliver accurate metrics to OIT according to the recording frequency indicated.

Success criteria

  • Deliver monthly metrics (2) to OIT by Jan 2020
  • Deliver quarterly metrics (3) to OIT by Jan 2020
  • Deliver annual metrics (2) to OIT by Sep 2020

Stakeholders

OIT, Product Managers

Open questions

  • Some metrics require further investigation (Reader adoption, Caseflow Certification, Caseflow Reader)

References/Resources

Designs and workflows

Business value

Dependencies
This was referenced Nov 19, 2019
Closed
Closed
@D-L-Ware D-L-Ware added Eng: Data Mark when data has been backfilled, or the issue has any data integrity concerns metrics labels Dec 19, 2019
@jimruggiero jimruggiero added Stakeholder: BVA Functionality associated with the Board of Veterans' Appeals workflows/feature requests Priority: Medium Blocking issue w/workaround, or "second in" priority for new work. and removed Priority: Medium Blocking issue w/workaround, or "second in" priority for new work. labels Dec 28, 2019
This was referenced Feb 7, 2020
Closed
Merged
@alisan16 alisan16 mentioned this issue Apr 1, 2020
Closed
@alisan16 alisan16 self-assigned this Apr 1, 2020
@alisan16 alisan16 mentioned this issue Jan 11, 2021
Merged
2 tasks
va-bot pushed a commit that referenced this issue Jan 12, 2021
@alisan16
Add certification metric to OIT Monthly Metrics Report Job (#15768)
c9cef22 
Connects #12748

### Description
Add the final OIT monthly metric (certification usage) to the MonthlyMetricsReportJob so that the metric is reported to Slack. With this, someone on the Caseflow team can simply copy and paste the Slack message from #appeals-job-alerts to send to OIT, rather than needing to login to the production console to run the certification metric manually every month.

Also refactored `spec/services/metrics/certification_usage_spec.rb` so the same test cases can be reused

### Acceptance Criteria
- [ ] Code compiles correctly
- [ ] Rspec tests pass

### Testing Plan
Updated test `spec/jobs/monthly_metrics_report_job_spec.rb`
Run locally and printed out rather than sending to Slack:
```
[1] pry(main)> MonthlyMetricsReportJob.perform_now
Monthly report 2020-12-01 to 2020-12-31
Percentage of all cases certified with Caseflow: 100.0
Appeals established within 7 days: 0 (100%)
Supplemental Claims within 7 days: 0 (NaN%)
Higher Level Reviews within 7 days: 0 (NaN%)
type,total,in_progress,cancelled,processed,established_within_seven_days,established_within_seven_days_percent,median,avg,max,min
supplemental_claims,0,0,0,0,0,NaN,00:00:00,00:00:00,00:00:00,00:00:00
higher_level_reviews,0,0,0,0,0,NaN,00:00:00,00:00:00,00:00:00,00:00:00
request_issues_updates,0,0,0,0,0,NaN,00:00:00,00:00:00,00:00:00,00:00:00
Performed MonthlyMetricsReportJob (Job ID: fcfef14e-d829-463b-b2b8-ed43939eb4d9) from Shoryuken(caseflow_development_low_priority) in 536.47ms
```
@alisan16 alisan16 removed their assignment May 19, 2021
@yoomlam yoomlam self-assigned this Jun 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yoomlam yoomlam

Labels
Eng: Data Mark when data has been backfilled, or the issue has any data integrity concerns Stakeholder: BVA Functionality associated with the Board of Veterans' Appeals workflows/feature requests Team: Delta 🔺 Type: Metrics or Reporting
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@alisan16 @lomaxap @D-L-Ware @jimruggiero @yoomlam @ThorntonMatthew