Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Off-by-1 error in reporting stopping / iteration counting? #32

Closed
nickeubank opened this issue Feb 22, 2021 · 1 comment · Fixed by #33
Closed

BUG: Off-by-1 error in reporting stopping / iteration counting? #32

nickeubank opened this issue Feb 22, 2021 · 1 comment · Fixed by #33
Assignees
@haoningjiang

Comments

@nickeubank
Copy link
Contributor

I think the iteration counter may be using human indices (starting at 1) while the arguments use 0-counting, leading to confusing printouts (compare top and last lines?):

image
@nickeubank
Copy link
Contributor Author

After playing around, I think the text error should just say "after doing". Will PR.

nickeubank added a commit to nickeubank/DAME-FLAME-Python-Package that referenced this issue Feb 23, 2021
@nickeubank
Correct error message on when iteration stops
30053a2 
Closes almost-matching-exactly#32
@nickeubank nickeubank mentioned this issue Feb 23, 2021
Merged
@nickeubank nickeubank mentioned this issue Feb 21, 2022
Closed
@nehargupta nehargupta mentioned this issue Feb 21, 2023
Merged
nehargupta added a commit that referenced this issue Feb 21, 2023
@nehargupta
Off by one (#60)
bc3ead2 
* remove db code

* fixed off by one errors

Iterations start at 0. An iteration is completed once we have successfully attempted the match for that iteration. Also fixed pe_stopping and adjusted defaults for pre_dame and early_stop_iterations to be float('inf')

* eliminated manual call to unittest

* removed flame_db tests

* update documentation

early_stop_pe_frac is now eliminated and documentation is updated as it pertains to iterations.

---------

Co-authored-by: Vittorio Orlandi <52432620+vittorioorlandi@users.noreply.github.com>

Per @vittorioorlandi

1. Fixed the definition of early stopping with respect to PE frac. Previously it was comparing to the previous iteration PE when it should be comparing to the PE at baseline.
Accordingly, the PE at the 0'th iteration is always the PE using all covariates — even if we make no exact matches.

2. Have everything starting at 0, with a completed iteration corresponding to successfully attempting matching.
Updated defaults for pre_dame and early_stop_iterations; they are now float(‘inf’). This makes sense if they are supposed to be the number of iterations before something happens; if they are infinite they never happen (ie never early stop because of iteration and never switch to dame).

3. Updated some notebooks to reflect the above.

4. Eliminated the database folder and corresponding test folder for it -- this is temporary to get it up on PyPi with all the recent updates but will be put back shortly.

5. Updates documentation to reflect the above

This takes care of #47 and #48 without breaking #32 or #33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@haoningjiang haoningjiang

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Correct error message on when iteration stops nickeubank/DAME-FLAME-Python-Package
2 participants
@nickeubank @haoningjiang