Add propensity weighting schemes and covariate balance plot functionality

[NathanielF]

In relation to this issue: #303

I'm opening the PR which includes functionality for fitting a propensity score model and analysing the experimental outcomes under different re-weighting schemes.

I've added the relevant classes to and models. I've also demonstrated their use an example notebook with a parameter recovery exercise and an application to real data.

I've added two plotting functions to experiment class to both analyse covariate balance and plot the overlap of the propensity scores and the uncertainty in the estimation of causal effects.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov]

Codecov Report

Attention: Patch coverage is 95.65217% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 80.09%. Comparing base (3dc2ffe) to head (0a54532).
Report is 14 commits behind head on main.

Files	Patch %	Lines
causalpy/pymc_experiments.py	95.31%	9 Missing ⚠️
causalpy/data_validation.py	81.81%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #311      +/-   ##
==========================================
+ Coverage   77.27%   80.09%   +2.82%     
==========================================
  Files          21       21              
  Lines        1395     1648     +253     
==========================================
+ Hits         1078     1320     +242     
- Misses        317      328      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[drbenvincent]

Sory, not had time to look at this yet I'm afraid - a combination of work + illness. I might also not get time to look next week because of a deadline on a client project. Looking forward to when I can dive into this 👍

Because this will be a new feature addition, and probably trigger a minor version bump (semantic versioning), I'll ask for at least one other review.

[NathanielF]

Added some explanation.

---

View entire conversation on ReviewNB

[NathanielF]

Again, have generally a more explanatory approach this time.

---

View entire conversation on ReviewNB

[NathanielF]

Added a note that these are the propensities we will seek to use.

---

View entire conversation on ReviewNB

[NathanielF]

Added more explict flagging.

---

View entire conversation on ReviewNB

[NathanielF]

Added full stop.

---

View entire conversation on ReviewNB

[NathanielF]

Added markdown separator.

---

View entire conversation on ReviewNB

[NathanielF]

Added bibtex

---

View entire conversation on ReviewNB

[drbenvincent]

nice!

---

View entire conversation on ReviewNB

[drbenvincent]

There are a number of paragraphs missing full stops at the very end

[drbenvincent]

Can you run make uml to update the UML diagrams?

Can we add a docstring to plot_balance_ecdf

Maybe add an assert for perc is 0-1 in the InversePropensityWeighting.weighted_percentile method

Excellent updates. The notebook is really nice at this point.

[drbenvincent]

missing "}" causing the docs to fail to build

[NathanielF]

fixed this

[drbenvincent]

So PropensityScore will inherit a fit method from ModelBuilder. So I think we can delete this whole method. If not, can you add a note into the docstring to explain why ModelBuilder.fit() is being overridden?

[NathanielF]

Added a note here. I overwrite the method just because the base method assumes a y input, and i wanted to keep the t to emphasise we're modelling the treatment not the outcome variable...

[drbenvincent]

Can you also please update from main? Might help fix some of the failing tests.

[NathanielF]

Can you run make uml to update the UML diagrams?
Can we add a docstring to plot_balance_ecdf
Maybe add an assert for perc is 0-1 in the InversePropensityWeighting.weighted_percentile method
Can you also please update from main? Might help fix some of the failing tests.

Addressed these. Not sure why the doctests are systematically failing. Every one seems to pass for me locally except for one print coefficents:

But oddly i haven't touched that method

[drbenvincent]

Addressed these. Not sure why the doctests are systematically failing. Every one seems to pass for me locally except for one print coefficents

So I was getting all tests passing locally, but 12 failing remotely. But after rebuilding the environment I replicated the 12 failing tests. We've got an issue for it #323

[drbenvincent]

So I think we are very close to being able to merge this. Just a few things:

• Once Fix failing doctests #330 is merged, can you update from main and just double check all tests pass.
• And I just realised, because this introduces nice new functionality, can you append to README.md (for the GitHub repo) and index.rst (for readthedocs).

[NathanielF]

Will merge from main this evening

[drbenvincent]

2 failing tests. These are the new ones that you've just introduced @NathanielF. This is fixable by adding <BLANKLINE> into the docstrings in the same way that I did in #330. That should hopefully fix everything.