Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maternal/Newborn health cohort model #1320

Draft
wants to merge 60 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
6a29cee
update contraception.py logging to capture df for new pregnancies
joehcollins Apr 19, 2024
50c2238
Merge branch 'master' into jcollins/mnh_cohort
joehcollins Apr 19, 2024
ad7ec65
Merge branch 'master' into jcollins/mnh_cohort
joehcollins Apr 24, 2024
19cb935
Merge branch 'master' into jcollins/mnh_cohort
joehcollins May 30, 2024
e23faaf
create cohort module and cohort resource file. updated demography log…
joehcollins May 31, 2024
460f870
updates to mnh module
joehcollins May 31, 2024
878695d
updates to mnh module
joehcollins Jun 10, 2024
12a56a7
update to cohort and util.py
joehcollins Jun 26, 2024
973cbff
update to cohort and util.py
joehcollins Jul 24, 2024
da7080b
Merge branch 'master' into jcollins/mnh_cohort
joehcollins Oct 1, 2024
06246c4
update to cohort
joehcollins Oct 2, 2024
86d130a
update to cohort
joehcollins Oct 2, 2024
3a90ab7
small updates to prevent scheduling of events in the past when sim st…
joehcollins Oct 4, 2024
930973e
new calibration file. prevent new pregnancies during cohort run.
joehcollins Oct 4, 2024
f1354c5
update scenario for azure test
joehcollins Oct 7, 2024
3c0d485
update scenario for azure test
joehcollins Oct 7, 2024
d51ff89
make cohort df come from excel
joehcollins Oct 7, 2024
0c7aa41
created mnh_outcome_logger to eventually replace logging for complica…
joehcollins Oct 7, 2024
87d6f1f
created mnh_outcome_logger to eventually replace logging for complica…
joehcollins Oct 7, 2024
cdd9842
added first logging to pregnancy supervisor
joehcollins Oct 7, 2024
b9a193d
finalising logging event
joehcollins Oct 8, 2024
ada05e0
add health system logging
joehcollins Oct 8, 2024
bfac245
old logging removed
joehcollins Oct 8, 2024
905091b
fix
joehcollins Oct 8, 2024
5a04b44
fix
joehcollins Oct 8, 2024
c62d5df
fix
joehcollins Oct 8, 2024
d8423e1
comments for clarity
joehcollins Oct 8, 2024
bfd1e1a
Merge branch 'master' into jcollins/mnh_cohort
joehcollins Oct 8, 2024
6cf3710
Merge branch 'jcollins/updated_mnh_logging' into jcollins/mnh_cohort
joehcollins Oct 8, 2024
cd3bd34
improved commenting
joehcollins Oct 8, 2024
a6b12fe
removed counting which wasnt needed
joehcollins Oct 8, 2024
5ca2314
fixes to pregnancy_supervisor test
joehcollins Oct 9, 2024
64b53c8
Merge branch 'jcollins/updated_mnh_logging' into jcollins/mnh_cohort
joehcollins Oct 9, 2024
8c6309c
Merge branch 'master' into jcollins/mnh_cohort
joehcollins Oct 9, 2024
ba22a99
fix logic in HIV causing crashes
joehcollins Oct 9, 2024
c271e72
Merge branch 'master' into jcollins/updated_mnh_logging
joehcollins Oct 10, 2024
011a001
updates to tests and new dummy analysis script
joehcollins Oct 10, 2024
09a216f
fix denom error
joehcollins Oct 10, 2024
03161e0
fix indentation error
joehcollins Oct 10, 2024
05473ff
Merge branch 'jcollins/updated_mnh_logging' into jcollins/mnh_cohort
joehcollins Oct 10, 2024
fd239bc
fix indentation error
joehcollins Oct 10, 2024
ae57c40
revert comit
joehcollins Oct 10, 2024
de3ee93
added missing logging
joehcollins Oct 10, 2024
bb7de0f
Merge branch 'jcollins/updated_mnh_logging' into jcollins/mnh_cohort
joehcollins Oct 10, 2024
7c3bf25
work on cohort
joehcollins Oct 14, 2024
53ed3bb
work on cohort
joehcollins Oct 15, 2024
0c3af64
work on cohort
joehcollins Oct 16, 2024
53fa8fa
work on cohort
joehcollins Oct 16, 2024
699b608
work on cohort
joehcollins Oct 16, 2024
dae7e93
work on cohort
joehcollins Oct 17, 2024
12178c2
updated tests
joehcollins Oct 17, 2024
24a14d0
changes to scripts
joehcollins Oct 18, 2024
9a39510
fix error in analysis script
joehcollins Oct 18, 2024
c8905bb
edits
joehcollins Oct 31, 2024
6c9ae17
edits
joehcollins Nov 6, 2024
1d3d3b4
edits
joehcollins Nov 6, 2024
85a77df
edits
joehcollins Nov 6, 2024
0f0c0f2
edits
joehcollins Nov 7, 2024
775776e
edits
joehcollins Nov 7, 2024
75291ea
edits
joehcollins Nov 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions resources/ResourceFile_LabourSkilledBirthAttendance.xlsx
Git LFS file not shown
3 changes: 3 additions & 0 deletions resources/ResourceFile_PregnancyCohort.xlsx
Git LFS file not shown
4 changes: 2 additions & 2 deletions resources/ResourceFile_PregnancySupervisor.xlsx
Git LFS file not shown
Git LFS file not shown
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import numpy as np
import pandas as pd

from pathlib import Path

from tlo import Date, logging
from tlo.methods import mnh_cohort_module
from tlo.methods.fullmodel import fullmodel
from tlo.scenario import BaseScenario


class BaselineScenario(BaseScenario):
def __init__(self):
super().__init__()
self.seed = 562661
self.start_date = Date(2024, 1, 1)
self.end_date = Date(2025, 1, 2)
self.pop_size = 12_000
self.number_of_draws = 11
self.runs_per_draw = 20

def log_configuration(self):
return {
'filename': 'block_intervention_group_test', 'directory': './outputs',
"custom_levels": {
"*": logging.WARNING,
"tlo.methods.demography": logging.INFO,
"tlo.methods.demography.detail": logging.INFO,
"tlo.methods.contraception": logging.INFO,
"tlo.methods.healthsystem.summary": logging.INFO,
"tlo.methods.healthburden": logging.INFO,
"tlo.methods.labour": logging.INFO,
"tlo.methods.labour.detail": logging.INFO,
"tlo.methods.newborn_outcomes": logging.INFO,
"tlo.methods.care_of_women_during_pregnancy": logging.INFO,
"tlo.methods.pregnancy_supervisor": logging.INFO,
"tlo.methods.postnatal_supervisor": logging.INFO,
}
}

def modules(self):
return [*fullmodel(resourcefilepath=self.resources,
module_kwargs={'Schisto': {'mda_execute': False}}),
mnh_cohort_module.MaternalNewbornHealthCohort(resourcefilepath=self.resources)]

def draw_parameters(self, draw_number, rng):
if draw_number == 0:
return {'PregnancySupervisor': {
'analysis_year': 2024}}
else:
intervention_groups = [['oral_antihypertensives', 'iv_antihypertensives', 'mgso4'],
['oral_antihypertensives', 'iv_antihypertensives', 'mgso4'],

['amtsl', 'pph_treatment_uterotonics', 'pph_treatment_mrrp'],
['amtsl', 'pph_treatment_uterotonics', 'pph_treatment_mrrp'],

['post_abortion_care_core', 'ectopic_pregnancy_treatment'],
['post_abortion_care_core', 'ectopic_pregnancy_treatment'],

['caesarean_section', 'blood_transfusion', 'pph_treatment_surgery'],
['caesarean_section', 'blood_transfusion', 'pph_treatment_surgery'],

['abx_for_prom', 'sepsis_treatment', 'birth_kit'],
['abx_for_prom', 'sepsis_treatment', 'birth_kit']]

avail_for_draw = [0.0, 1.0,
0.0, 1.0,
0.0, 1.0,
0.0, 1.0,
0.0, 1.0]

return {'PregnancySupervisor': {
'analysis_year': 2024,
'interventions_analysis': True,
'interventions_under_analysis':intervention_groups[draw_number-1],
'intervention_analysis_availability': avail_for_draw[draw_number-1]}}


if __name__ == '__main__':
from tlo.cli import scenario_run
scenario_run([__file__])
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
import numpy as np
import pandas as pd

from pathlib import Path

from tlo import Date, logging
from tlo.methods import mnh_cohort_module
from tlo.methods.fullmodel import fullmodel
from tlo.scenario import BaseScenario


class BaselineScenario(BaseScenario):
def __init__(self):
super().__init__()
self.seed = 562661
self.start_date = Date(2024, 1, 1)
self.end_date = Date(2025, 1, 2)
self.pop_size = 12_000
self.number_of_draws = 9
self.runs_per_draw = 30

def log_configuration(self):
return {
'filename': 'block_intervention_test', 'directory': './outputs',
"custom_levels": {
"*": logging.WARNING,
"tlo.methods.demography": logging.INFO,
"tlo.methods.demography.detail": logging.INFO,
"tlo.methods.contraception": logging.INFO,
"tlo.methods.healthsystem.summary": logging.INFO,
"tlo.methods.healthburden": logging.INFO,
"tlo.methods.labour": logging.INFO,
"tlo.methods.labour.detail": logging.INFO,
"tlo.methods.newborn_outcomes": logging.INFO,
"tlo.methods.care_of_women_during_pregnancy": logging.INFO,
"tlo.methods.pregnancy_supervisor": logging.INFO,
"tlo.methods.postnatal_supervisor": logging.INFO,
}
}

def modules(self):
return [*fullmodel(resourcefilepath=self.resources,
module_kwargs={'Schisto': {'mda_execute': False}}),
mnh_cohort_module.MaternalNewbornHealthCohort(resourcefilepath=self.resources)]

def draw_parameters(self, draw_number, rng):
if draw_number == 0:
return {'PregnancySupervisor': {
'analysis_year': 2024}}
else:
interventions_for_analysis = ['oral_antihypertensives', 'oral_antihypertensives',
'iv_antihypertensives', 'iv_antihypertensives',
'mgso4', 'mgso4',
'post_abortion_care_core', 'post_abortion_care_core']

avail_for_draw = [0.0, 1.0,
0.0, 1.0,
0.0, 1.0,
0.0, 1.0]

return {'PregnancySupervisor': {
'analysis_year': 2024,
'interventions_analysis': True,
'interventions_under_analysis':[interventions_for_analysis[draw_number-1]],
'intervention_analysis_availability': avail_for_draw[draw_number-1]}}


if __name__ == '__main__':
from tlo.cli import scenario_run
scenario_run([__file__])
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
import os

import pandas as pd

import matplotlib.pyplot as plt
import numpy as np

from tlo.analysis.utils import extract_results, get_scenario_outputs, summarize

outputspath = './outputs/sejjj49@ucl.ac.uk/'

scenario = 'block_intervention_test-2024-11-06T145016Z'

results_folder= get_scenario_outputs(scenario, outputspath)[-1]

def get_data_frames(key, results_folder):
def sort_df(_df):
_x = _df.drop(columns=['date'], inplace=False)
return _x.iloc[0]

results_df = extract_results(
results_folder,
module="tlo.methods.pregnancy_supervisor",
key=key,
custom_generate_series=sort_df,
do_scaling=False
)

results_df_summ = summarize(extract_results(
results_folder,
module="tlo.methods.pregnancy_supervisor",
key=key,
custom_generate_series=sort_df,
do_scaling=False
))

return [results_df, results_df_summ]

results = {k:get_data_frames(k, results_folder)[0] for k in
['mat_comp_incidence', 'nb_comp_incidence', 'deaths_and_stillbirths','service_coverage',
'yearly_mnh_counter_dict']}

results_sum = {k:get_data_frames(k, results_folder)[1] for k in
['mat_comp_incidence', 'nb_comp_incidence', 'deaths_and_stillbirths','service_coverage',
'yearly_mnh_counter_dict']}

def get_data(df, draw):
joehcollins marked this conversation as resolved.
Show resolved Hide resolved
return (df.loc['direct_mmr', (draw, 'lower')],
df.loc['direct_mmr', (draw, 'mean')],
df.loc['direct_mmr', (draw, 'upper')])


mmrs = {'baseline':get_data(results_sum['deaths_and_stillbirths'], 0),
# 'oral_antihypertensives_min':get_data(results_sum['deaths_and_stillbirths'], 1),
# 'oral_antihypertensives_max': get_data(results_sum['deaths_and_stillbirths'], 2),
# 'iv_antihypertensives_min':get_data(results_sum['deaths_and_stillbirths'], 3),
# 'iv_antihypertensives_max': get_data(results_sum['deaths_and_stillbirths'], 4),
# 'amtsl_min':get_data(results_sum['deaths_and_stillbirths'], 5),
# 'amtsl_max': get_data(results_sum['deaths_and_stillbirths'], 6),
'mgso4_min':get_data(results_sum['deaths_and_stillbirths'], 7),
'mgso4_max': get_data(results_sum['deaths_and_stillbirths'], 8),
# 'post_abortion_care_core_min':get_data(results_sum['deaths_and_stillbirths'], 9),
# 'post_abortion_care_core_max': get_data(results_sum['deaths_and_stillbirths'], 10),
# 'caesarean_section_min':get_data(results_sum['deaths_and_stillbirths'], 11),
# 'caesarean_section_max': get_data(results_sum['deaths_and_stillbirths'], 12),
# 'ectopic_pregnancy_treatment_min':get_data(results_sum['deaths_and_stillbirths'], 13),
# 'ectopic_pregnancy_treatment_max': get_data(results_sum['deaths_and_stillbirths'], 14),
}

def get_mmr_diffs(df, draws):
diff_results = {}
baseline = results['deaths_and_stillbirths'][0]

for draw in draws:
# diff_df = ((results['deaths_and_stillbirths'][draw] - baseline)/baseline) * 100
diff_df = results['deaths_and_stillbirths'][draw] - baseline
diff_df.columns = pd.MultiIndex.from_tuples([(draw, v) for v in range(len(diff_df.columns))],
names=['draw', 'run'])
results_diff = summarize(diff_df)
results_diff.fillna(0)
diff_results.update({draw: results_diff})

return diff_results

diff_results = get_mmr_diffs(results, [7,8])


results_diff = {#'oral_antihypertensives_min':get_data(diff_results[1], 1),
# 'oral_antihypertensives_max':get_data(diff_results[2], 2),
# 'iv_antihypertensives_min':get_data(diff_results[3], 3),
# 'iv_antihypertensives_max': get_data(diff_results[4], 4),
# 'amtsl_min':get_data(diff_results[5], 5),
# 'amtsl_max': get_data(diff_results[6], 6),
'mgso4_min':get_data(diff_results[7], 7),
'mgso4_max':get_data(diff_results[8], 8),
# 'post_abortion_care_core_min':get_data(diff_results[9], 9),
# 'post_abortion_care_core_max': get_data(diff_results[10], 10),
# 'caesarean_section_min':get_data(diff_results[11], 11),
# 'caesarean_section_max': get_data(diff_results[12], 12),
# 'ectopic_pregnancy_treatment_min':get_data(diff_results[13], 13),
# 'ectopic_pregnancy_treatment_max': get_data(diff_results[14], 14)
}

# todo: compare deaths with demography logging...

data = mmrs

# Extract means and errors
labels = data.keys()
means = [vals[1] for vals in data.values()]
lower_errors = [vals[1] - vals[0] for vals in data.values()]
upper_errors = [vals[2] - vals[1] for vals in data.values()]
errors = [lower_errors, upper_errors]

# Create bar chart with error bars
fig, ax = plt.subplots()
ax.bar(labels, means, yerr=errors, capsize=5, alpha=0.7, ecolor='black')
ax.set_ylabel('MMR')
ax.set_title('Average MMR under each scenario')

# Adjust label size
plt.xticks(fontsize=8, rotation=90)
plt.tight_layout()
plt.show()
joehcollins marked this conversation as resolved.
Show resolved Hide resolved

#
# # Example data with uncertainties
# parameters = ['Blood Transfusion', 'Uterotonics', 'Sepsis treatment']
# base_value = results_sum['deaths_and_stillbirths'].at['direct_mmr', (0, 'mean')] # base case value for the output variable
# high_values = [results_sum['deaths_and_stillbirths'].at['direct_mmr', (1, 'mean')],
# results_sum['deaths_and_stillbirths'].at['direct_mmr', (3, 'mean')],
# results_sum['deaths_and_stillbirths'].at['direct_mmr', (5, 'mean')]] # lower-bound values for each parameter
# low_values = [results_sum['deaths_and_stillbirths'].at['direct_mmr', (2, 'mean')],
# results_sum['deaths_and_stillbirths'].at['direct_mmr', (4, 'mean')],
# results_sum['deaths_and_stillbirths'].at['direct_mmr', (6, 'mean')]] # upper-bound values for each parameter
#
# # Calculate deltas from base value
# low_deltas = [base_value - lv for lv in low_values]
# high_deltas = [hv - base_value for hv in high_values]
#
# # Sort parameters by absolute impact
# abs_impacts = [abs(low) + abs(high) for low, high in zip(low_deltas, high_deltas)]
# sorted_indices = np.argsort(abs_impacts)[::-1]
# parameters = [parameters[i] for i in sorted_indices]
# low_deltas = [low_deltas[i] for i in sorted_indices]
# high_deltas = [high_deltas[i] for i in sorted_indices]
#
# # Calculate changes from the base case
# low_deltas = [base_value - lv for lv in low_values]
# high_deltas = [hv - base_value for hv in high_values]
#
# # Sort parameters by absolute impact (for a tornado effect)
# abs_impacts = [abs(low) + abs(high) for low, high in zip(low_deltas, high_deltas)]
# sorted_indices = np.argsort(abs_impacts)[::-1]
# parameters = [parameters[i] for i in sorted_indices]
# low_deltas = [low_deltas[i] for i in sorted_indices]
# high_deltas = [high_deltas[i] for i in sorted_indices]
#
# # Plotting
# fig, ax = plt.subplots(figsize=(8, 6))
#
# # Plot each bar for the low and high values
# for i, (param, low, high) in enumerate(zip(parameters, low_deltas, high_deltas)):
# ax.barh(param, high, left=base_value, color='skyblue')
# ax.barh(param, low, left=base_value + low, color='salmon')
#
# # Reference line for base value
# ax.axvline(base_value, color='black', linestyle='--', label="Base Value")
#
# # Labels and title
# ax.set_xlabel('Output Variable')
# ax.set_title('Tornado Plot')
# plt.legend(['Base Value'])
# plt.show()

# import matplotlib.pyplot as plt
# import numpy as np
# # Sample data
# mmr_data = {
# 'int_1': [(235, 250, 265), (335, 350, 365)],
# 'int_2': [(170, 195, 200), (290, 305, 320)],
# 'int_3': [(280, 295, 310), (295 ,310, 325)],
# 'int_4': [(165, 180, 195), (385, 400, 415)]
# }
# # Plotting
# fig, ax = plt.subplots()
# for key, intervals in mmr_data.items():
# for idx, (lower, mean, upper) in enumerate(intervals):
# x = np.arange(len(mmr_data)) * len(intervals) + idx
# ax.plot(x, mean, 'o', label=f'{key}' if idx == 0 else "")
# ax.fill_between([x, x], [lower, lower], [upper, upper], alpha=0.2)
# ax.set_xticks(np.arange(len(mmr_data)) * len(intervals) + 0.5)
# ax.set_xticklabels(mmr_data.keys())
# plt.legend()
# plt.show()
Loading
Loading