Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of unique tests on all Core models #497

Merged
merged 7 commits into from
Jun 20, 2024

Conversation

thutuva
Copy link
Member

@thutuva thutuva commented Jun 5, 2024

Describe your changes

Please include a summary of any changes.

  • Added unique tests to all Core models after determining which column are the primary keys:
    • condition = condition_id
    • eligibility = eligibility_id (patient_id, enrollment start, enrollment end, payer, plan)
    • encounter = encounter_id
    • lab_result = lab_result_id
    • location = location_id
    • medical_claim = medical_claim_id (new column; concatenation of claim_id and claim_line_number)
    • medication = medication_id
    • observation = observation_id
    • patient = patient_id
    • pharmacy_claim = pharmacy_claim_id (new column; concatenation of claim_id and claim_line_number)
    • practitioner = practitioner_id
    • procedure = procedure_id

How has this been tested?

Please describe the tests you ran to verify your changes. Provide instructions or code to reproduce output.
ran dbt build locally

Reviewer focus

Please summarize the specific items you’d like the reviewer(s) to look into.

Checklist before requesting a review

  • I have added at least one Github label to this PR (bug, enhancement, breaking change,...)
  • My code follows style guidelines
  • (New models) YAML files are categorized by sub folder and models listed in alphabetical order
  • (New models) I have added a config to each new model to enable it for claims and/or clinical data
  • (New models) I have added the variable tuva_last_run to the final output
  • (Optional) I have recorded a Loom to explain this PR

Package release checklist

  • I have updated dbt docs
  • I have updated the version number in the dbt_project.yml

(Optional) Gif of how this PR makes you feel


Loom link

@thutuva thutuva marked this pull request as draft June 5, 2024 18:34
@thutuva thutuva marked this pull request as ready for review June 5, 2024 19:12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI this won't work on athena
cast(unpivot_cte.claim_id||''||unpivot_cte.claim_line_number||''||unpivot_cte.diagnosis_rank||'_'||unpivot_cte.source_code as {{ dbt.type_string() }} ) as condition_id

athena makes you cast claim line number explicitly as varchar

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'll have to cast as varchar instead of date for athena

   cast(patient_id as {{ dbt.type_string() }} ) || '-' || cast(enrollment_start_date as date ) || '-' || cast(enrollment_end_date as date )
        || '-' ||  cast(payer as {{ dbt.type_string() }} ) || '-' || cast(plan as {{ dbt.type_string() }} ) as eligibility_id
   , cast(patient_id as {{ dbt.type_string() }} ) as patient_id

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this won't work on athena, need to cast as same type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@bradmontierth bradmontierth merged commit 6cb91a0 into main Jun 20, 2024
5 checks passed
@bradmontierth bradmontierth deleted the primary-key-adds branch June 20, 2024 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants