Skip to content

Commit

Permalink
[Insights][US_CA] No treatment starts client events (Recidiviz/recidi…
Browse files Browse the repository at this point in the history
…viz-data#29333)

## Description of the change

Updates `supervision_client_events` to include "treatment_starts" events
as those clients who have not had a treatment start within the last year
(time period). This removes the events that show real treatment start
data, and replaces them with events that show no treatment starts for
US_CA which is relevant to the metric events table feature.

This has been run in the jovergaag_treatment sandbox, and comparisons
were run as well. There are lots of differences in events for metrics
that are not "treatment_starts", almost entirely in the `description`
column. There are 80 non treatment start events differences due to other
column data changes or new events, all of them look like they are likely
legitimate changes but I would appreciate a second set of eyes on those.
I'll run the comparisons again today after the pipelines run to see if
any of those disappear.

## Type of change

> All pull requests must have at least one of the following labels
applied (otherwise the PR will fail):

| Label | Description |
|-----------------------------
|-----------------------------------------------------------------------------------------------------------
|
| Type: Bug | non-breaking change that fixes an issue |
| Type: Feature | non-breaking change that adds functionality |
| Type: Breaking Change | fix or feature that would cause existing
functionality to not work as expected |
| Type: Non-breaking refactor | change addresses some tech debt item or
prepares for a later change, but does not change functionality |
| Type: Configuration Change | adjusts configuration to achieve some end
related to functionality, development, performance, or security |
| Type: Dependency Upgrade | upgrades a project dependency - these
changes are not included in release notes |

## Related issues

Closes Recidiviz/recidiviz-data#28251

## Checklists

### Development

**This box MUST be checked by the submitter prior to merging**:
- [x] **Double- and triple-checked that there is no Personally
Identifiable Information (PII) being mistakenly added in this pull
request**

These boxes should be checked by the submitter prior to merging:
- [ ] Tests have been written to cover the code changed/added as part of
this pull request

### Code review

These boxes should be checked by reviewers prior to merging:

- [ ] This pull request has a descriptive title and information useful
to a reviewer
- [ ] Potential security implications or infrastructural changes have
been considered, if relevant

GitOrigin-RevId: f54019f4ca8c6cd07ccca9c0e56a982567adc715
  • Loading branch information
jovergaag authored and Helper Bot committed May 14, 2024
1 parent 3dd2654 commit 29250a2
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,31 @@
) AS attributes
FROM `{{project_id}}.us_tn_raw_data_up_to_date_views.VantagePointRecommendations_latest` vpr
INNER JOIN `{{project_id}}.normalized_state.state_person_external_id` pid ON pid.external_id = vpr.OffenderID)
)
-- TREATMENT_STARTS events are treated differently than the other events, they represent when a client did NOT
-- have a treatment start within the time period (US_CA only)
, treatment_starts AS (
SELECT
"US_CA" AS state_code,
"treatment_starts" AS metric_id,
-- When there is not a treatment start, the event date is either the last day of the current time period
DATE_SUB(population_end_date, INTERVAL 1 DAY) AS event_date,
a.person_id,
TO_JSON_STRING(
STRUCT(
NULL as code,
"No treatment start within the past year" as description
)) AS attributes
FROM `{{project_id}}.aggregated_metrics.supervision_officer_metrics_person_assignment_sessions_materialized` a
CROSS JOIN
latest_year_time_period period
LEFT JOIN `{{project_id}}.normalized_state.state_program_assignment` spa
USING(person_id)
WHERE a.state_code = "US_CA"
-- Find clients who do not have any treatment starts, or those who do not have a treatment start within the time period
AND (spa.person_id IS NULL OR spa.start_date NOT BETWEEN population_start_date AND population_end_date)
-- Find officer periods that fall within the current time period
AND a.assignment_date <= population_end_date AND IFNULL(a.end_date, population_end_date) >= population_start_date
),
all_events AS (
SELECT * FROM events_with_metric_id
Expand All @@ -215,6 +240,8 @@
SELECT * FROM sanctions
UNION ALL
SELECT * FROM treatment_referrals
UNION ALL
SELECT * FROM treatment_starts
),
supervision_client_events AS (
SELECT
Expand Down
3 changes: 3 additions & 0 deletions recidiviz/calculator/query/state/views/outliers/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from recidiviz.calculator.query.state.views.outliers.outliers_enabled_states import (
get_outliers_enabled_states_for_bigquery,
)
from recidiviz.outliers.constants import TREATMENT_STARTS
from recidiviz.outliers.outliers_configs import get_outliers_backend_config
from recidiviz.outliers.types import MetricOutcome

Expand Down Expand Up @@ -64,6 +65,8 @@ def format_state_specific_person_events_filters(years_lookback: int = 2) -> str:
state_code = '{state_code}'
-- Limit the events lookback to minimize the size of the subqueries
AND event_date >= DATE_SUB(CURRENT_DATE('US/Eastern'), INTERVAL {str(years_lookback)} YEAR)
-- TREATMENT_STARTS has custom logic and is handled in a separate cte
AND '{metric.name}' != '{TREATMENT_STARTS.name}'
{f"AND ({metric.metric_event_conditions_string})" if metric.metric_event_conditions_string else ""}
"""
)
Expand Down

0 comments on commit 29250a2

Please sign in to comment.