Creating Queries That Return Cohort Information

[Jay-sanjay]

This PR fixes #58

PR Checklist

• GetCohortSubjects function and it's test suites
• GetCohortSubjectEndDate function and it's test suites
• GetDatabaseCohorts function and it's test suites
• GetCohortSubjectStartDate function and it's test suites

[codecov]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (5a0a09a) 73.81% compared to head (8864f30) 74.51%.

Files	Patch %	Lines
src/getters.jl	80.00%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev      #61      +/-   ##
==========================================
+ Coverage   73.81%   74.51%   +0.69%     
==========================================
  Files           6        6              
  Lines         275      310      +35     
==========================================
+ Hits          203      231      +28     
- Misses         72       79       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[TheCedarPrince]

Yea, I just tried testing this and here is what I see:

GetCohortSubjects Tests: Error During Test at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:405
  Got exception outside of a @test
  BoundsError: attempt to access 0-element Vector{Union{Missing, Float64}} at index [1:8]
  Stacktrace:
    [1] throw_boundserror(A::Vector{Union{Missing, Float64}}, I::Tuple{UnitRange{Int64}})
      @ Base ./abstractarray.jl:744
    [2] checkbounds
      @ ./abstractarray.jl:709 [inlined]
    [3] getindex(A::Vector{Union{Missing, Float64}}, I::UnitRange{Int64})
      @ Base ./array.jl:921
    [4] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:412 [inlined]
    [5] macro expansion
      @ ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
    [6] top-level scope
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/sqlite/getters.jl:407
    [7] include(fname::String)
      @ Base.MainInclude ./client.jl:478
    [8] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:51 [inlined]
    [9] macro expansion
      @ ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
   [10] macro expansion
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:51 [inlined]
   [11] macro expansion
      @ ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:1498 [inlined]
   [12] top-level scope
      @ ~/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:47
   [13] include(fname::String)
      @ Base.MainInclude ./client.jl:478
   [14] top-level scope
      @ none:6
   [15] eval
      @ ./boot.jl:370 [inlined]
   [16] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:280
   [17] _start()
      @ Base ./client.jl:522
Test Summary:                                       | Pass  Error  Total   Time
OMOPCDMCohortCreator                                |   43      1     44  10.3s
  SQLite Helper Functions                           |    2             2   0.6s
  SQLite Getter Functions                           |   33      1     34   9.3s
    GetDatabasePersonIDs Tests                      |    1             1   0.6s
    GetDatabaseYearRange Tests                      |    1             1   0.1s
    GetPatientGender Tests                          |    1             1   0.5s
    GetPatientRace Tests                            |    1             1   0.1s
    GetPatientAgeGroup Tests                        |    2             2   0.9s
    GetPatientVisits Tests                          |    2             2   0.2s
    GetMostRecentConditions Tests                   |    2             2   0.3s
    GetMostRecentVisit Tests                        |    2             2   0.6s
    GetVisitConcept Tests                           |    1             1   0.2s
    GetVisitCondition Tests                         |    2             2   0.1s
    GetPatientEthnicity Tests                       |    1             1   0.1s
    GetVisitDate Tests                              |    2             2   0.2s
    GetDrugExposures Tests                          |    1             1   0.9s
    GetDrugConceptIDs Tests                         |    1             1   0.2s
    GetPatientGender multiple dispatch Tests        |    1             1   0.3s
    GetPatientRace multiple dispatch Tests          |    1             1   0.1s
    GetPatientAgeGroup multiple dispatch Tests      |    1             1   0.4s
    GetPatientVisits multiple dispatch Tests        |    1             1   0.4s
    GetMostRecentConditions multiple dispatch Tests |    1             1   0.2s
    GetMostRecentVisit multiple dispatch Tests      |    1             1   0.4s
    GetVisitConcept multiple dispatch Tests         |    1             1   0.2s
    GetVisitCondition multiple dispatch Tests       |    1             1   0.2s
    GetPatientEthnicity multiple dispatch Tests     |    1             1   0.1s
    GetVisitDate multiple dispatch Tests            |    2             2   0.1s
    GetDrugExposures multiple dispatch Tests        |    1             1   0.2s
    GetDrugConceptIDs multiple dispatch Tests       |    1             1   0.5s
    GetCohortSubjects Tests                         |           1      1   1.1s
  SQLite Filter Functions                           |    7             7   0.2s
  SQLite Executors Functions                        |    1             1   0.2s
ERROR: LoadError: Some tests did not pass: 43 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/thecedarprince/FOSS/OMOPCDMCohortCreator.jl/test/runtests.jl:46
ERROR: Package OMOPCDMCohortCreator errored during testing

The most important pieces it seems are:

  Got exception outside of a @test
  BoundsError: attempt to access 0-element Vector{Union{Missing, Float64}} at index [1:8]

and that it occurs around lines 412 and 407. Don't have time to take a look at the moment but this seems like where a problem is arising from.

[TheCedarPrince]

Hm. Curious. I think it has something to do with the cohort table in the dataset. I know you were saying it passed locally on your computer but can you try deleting everything from your Cohort table within your dataset copy and re-run the test suite @Jay-sanjay ?

[TheCedarPrince]

Hey @Jay-sanjay , I checked out your code locally to test it and also get a feel for the API. On that side, everything is great. I haven't checked your tests yet and I left some comments within your code as I was making a few docstring fixes.

Let me know if you have any questions! Great work on this!

[TheCedarPrince]

Almost there! just a few changes left to make and then I think this is completely ready to merge! Let me know if you have any further questions.

Thanks for the great work!

[TheCedarPrince]

Hey Jay, almost there I think. I am seeing this error when I try out the new functionality:

import SQLite:
    DB

conn = DB(eunomia);

import DBInterface:
    execute 

import OMOPCDMCohortCreator as occ

# Defines what kind of database occ is connecting to
occ.GenerateDatabaseDetails(
    :sqlite,
    "main"
)

#= 

Generates internal representation of what tables are 
available for occ to operate upon

=#
occ.GenerateTables(conn)

occ.GetDatabaseCohorts(conn)

Which gives

julia> GetDatabaseCohorts(conn)
ERROR: MethodError: no method matching iterate(::FunSQL.SQLNode)

Closest candidates are:
  iterate(::Union{LinRange, StepRangeLen})
   @ Base range.jl:880
  iterate(::Union{LinRange, StepRangeLen}, ::Integer)
   @ Base range.jl:880
  iterate(::T) where T<:Union{Base.KeySet{<:Any, <:Dict}, Base.Val
ueIterator{<:Dict}}
   @ Base dict.jl:698
  ...

Stacktrace:
 [1] unique(itr::FunSQL.SQLNode)
   @ Base ./set.jl:175
 [2] GetDatabaseCohorts(; tab::FunSQL.SQLTable)
   @ OMOPCDMCohortCreator ~/FOSS/OMOPCDMCohortCreator.jl/src/gette
rs.jl:1793
 [3] GetDatabaseCohorts(conn::DB; tab::FunSQL.SQLTable)
   @ OMOPCDMCohortCreator ~/FOSS/OMOPCDMCohortCreator.jl/src/gette
rs.jl:1769
 [4] GetDatabaseCohorts(conn::DB)
   @ OMOPCDMCohortCreator ~/FOSS/OMOPCDMCohortCreator.jl/src/gette
rs.jl:1765
 [5] top-level scope
   @ REPL[21]:1

Looking at the stack trace, it seems to be coming from the unique call you added into the function. Also, could you turn the above code into a simple test for this function? I think that would be a good test for this function.

Thanks!

[TheCedarPrince]

LGTM -- Looks Good To Me @Jay-sanjay !

[TheCedarPrince]

GOOD JOB!!! That was a lot of work and I am thrilled with how this turned out! 😄 Congratulations @Jay-sanjay -- very proud of the work here that you did!

[Jay-sanjay]

Thanks for all the reviews and guidance till now @TheCedarPrince they mean a lot to me :)