ENH: Dcm logit rebased2 #8120

josef-pkt · 2022-02-14T17:56:44Z

rebased version of #1605
no additional changes

no merge conflicts, but likely outdated in inherited parts, no changes in PR since 2014

(I'm not planning to work on this soon, but want to have it closer to current code and wanted to see if there are serious merge conflicts)

* first draft of CLogit based on TryCLogit(sandbox-statsmodels:runmnl.py)

* add df_model and df_ resid. * add summary. * deal with names and number of exogs. * add 2 examples: one alternative specific varible with and without alternative-specific constants. * some clean.

* first draft of results class. * added analytical score/gradient, hessian and Jacobian. * examples moved to another file.

* fixed default method to Newton: solves the problem of high iterations. * new value of z: allows estimate alternative specific variables + individual specific variables (new example 4).

* start params estimated from the standard logit, * data entry by dictionary, data handle inside class, * work with alternative specific and/or individual/case specific variables.

* new in summary: method, iterations, elapsed time, num. cases, num. and frequencies of alternatives. Note than from previous commit, when a specific alternative variable isn't present on all utilities, no longer need to do previous work like: df ['vrble_alternative'] = df ['variable'] * (choice_index == 0)

ordered when choice set are strings.

* new on summary: LL-Null, Pseudo R-squ., LLR p-value, Likelihood ratio test, AIC. * added fitted_values.

…lete prints inside classes. * New tests: bse, llf, llnull, aic.

for params, results cross check with biogeme, new tests for Likelihood ratio test, score and predic values.

…rent from the optimum. Load forgotten files in last commit: clogit_predict.csv and Biogeme results.

…fied, prediction table. * added notes for marginal effect.

* specification random Coefficients. * simulated maximum likelihood. * fixed and/or random coefficients.

* replicate R results (with mean and sd fixed)

(second try)

…mation procedure. * added examples with generic and alternative specific coef. * tested against R: more slow but good results.

move example to module, add smoke test

pep8speaks · 2022-02-14T17:56:51Z

Hello @josef-pkt! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file statsmodels/discrete/dcm_base.py:

Line 76:32: E711 comparison to None should be 'if cond is None:'

In the file statsmodels/discrete/dcm_clogit.py:

Line 20:1: F401 'pandas as pd' imported but unused
Line 122:15: E271 multiple spaces after keyword
Line 185:15: E271 multiple spaces after keyword
Line 421:19: E711 comparison to None should be 'if cond is None:'
Line 469:23: E124 closing bracket does not match visual indentation

In the file statsmodels/discrete/dcm_mxlogit.py:

Line 31:68: W605 invalid escape sequence '\s'
Line 33:40: W605 invalid escape sequence '\s'
Line 126:41: W605 invalid escape sequence '\g'
Line 130:15: W605 invalid escape sequence '\q'
Line 130:36: W605 invalid escape sequence '\s'
Line 147:11: E271 multiple spaces after keyword
Line 234:28: W605 invalid escape sequence '\i'
Line 234:66: W605 invalid escape sequence '\i'
Line 234:96: W605 invalid escape sequence '\s'
Line 236:20: W605 invalid escape sequence '\p'
Line 239:28: W605 invalid escape sequence '\i'
Line 239:58: W605 invalid escape sequence '\s'
Line 239:83: W605 invalid escape sequence '\p'
Line 246:15: E271 multiple spaces after keyword
Line 258:17: E117 over-indented
Line 298:46: W605 invalid escape sequence '\s'
Line 302:19: W605 invalid escape sequence '\L'
Line 302:25: W605 invalid escape sequence '\s'
Line 302:40: W605 invalid escape sequence '\s'
Line 302:62: W605 invalid escape sequence '\l'
Line 439:23: E124 closing bracket does not match visual indentation

In the file statsmodels/discrete/example_mxlogit.py:

Line 3:1: F401 'numpy as np' imported but unused
Line 8:1: F401 'statsmodels.discrete.tests.results.results_dcm_clogit.Travelmodechoice' imported but unused

In the file statsmodels/discrete/tests/test_mxlogit_smoke.py:

Line 3:1: F401 'numpy as np' imported but unused
Line 8:1: F401 'statsmodels.discrete.tests.results.results_dcm_clogit.Travelmodechoice' imported but unused

josef-pkt · 2022-02-14T17:58:26Z

notebook has output in it.

This might need a lot of squashing.

lgtm-com · 2022-02-14T18:41:43Z

This pull request introduces 8 alerts when merging 06890fe into 152e27d - view on LGTM.com

new alerts:

3 for Unused import
2 for Testing equality to None
2 for Variable defined multiple times
1 for `__init__` method calls overridden method

josef-pkt · 2022-02-14T18:59:31Z

unit tests error on import

statsmodels/discrete/dcm_clogit.py:17: in <module>
    from statsmodels.compat.collections import OrderedDict
E   ModuleNotFoundError: No module named 'statsmodels.compat.collections'

tatsmodels/discrete/tests/test_mxlogit_smoke.py:2: in <module>
    from statsmodels.compat.collections import OrderedDict
E   ModuleNotFoundError: No module named 'statsmodels.compat.collections'

large number of style, pep-8 violations

bashtage · 2022-02-14T19:02:29Z

These are long gone.

josef-pkt · 2022-02-14T19:09:13Z

no code update since 2014 is a long time (this was originally written for python 2)

AnaMP and others added 30 commits February 14, 2022 12:49

statsmodels: conditional logit model (statsmodels#941)

dca35ed

* first draft of CLogit based on TryCLogit(sandbox-statsmodels:runmnl.py)

BUG: clogit example fix index, exog columns

76db1a3

REF/ENH: streamline exog creation, add summary to example

57fd0ca

fixed some errors. Still a draft.

d23c886

clogit draft: work in some stuff and fixed some bugs.

6a93d9c

Work on draft:

6c493aa

* add df_model and df_ resid. * add summary. * deal with names and number of exogs. * add 2 examples: one alternative specific varible with and without alternative-specific constants. * some clean.

ENH:add unit test. Initial draft

c19a2c4

ENH, BUG: add check hessian result.

093b672

ENH,STY: work on conditional logit.

6651800

* first draft of results class. * added analytical score/gradient, hessian and Jacobian. * examples moved to another file.

BUG:

2299d0b

* fixed default method to Newton: solves the problem of high iterations. * new value of z: allows estimate alternative specific variables + individual specific variables (new example 4).

ENH, STY: start params, data entry, all type of variables.

6eece7b

* start params estimated from the standard logit, * data entry by dictionary, data handle inside class, * work with alternative specific and/or individual/case specific variables.

BUG: replaced dictionary to ordered dictionary to keep the data

b4cb3cf

ordered when choice set are strings.

ENH:summary improved

7b3dc1d

* new on summary: LL-Null, Pseudo R-squ., LLR p-value, Likelihood ratio test, AIC. * added fitted_values.

TST,STY: new tests, decorators in results class, clean example and de…

3f855c7

…lete prints inside classes. * New tests: bse, llf, llnull, aic.

ENH,TST: predicted values (linear and not linear), explanatory names

9f01ca3

for params, results cross check with biogeme, new tests for Likelihood ratio test, score and predic values.

BUG, TEST: fixed indentation level and test score at parameters diffe…

e6c7c6e

…rent from the optimum. Load forgotten files in last commit: clogit_predict.csv and Biogeme results.

typo fixed

fc66de6

ENH: residuals, residuals indicating which observations are misclassi…

3a8cb20

…fied, prediction table. * added notes for marginal effect.

Added Conditional Logit Notebook example

5c5fe1e

Draft of Mixed Logit

cf04840

* specification random Coefficients. * simulated maximum likelihood. * fixed and/or random coefficients.

ENH: Halton sequence

7a8454d

* replicate R results (with mean and sd fixed)

minor changes: rename file and added .txt with R results

c9889cd

(second try)

ENH: normal distributions. Estimate loc and scale as part of the esti…

6b80e3e

…mation procedure. * added examples with generic and alternative specific coef. * tested against R: more slow but good results.

STY: Common superclass for the dcm models and results classes.

69c3c91

rdataset replaced by modechoice dataset

df5984e

BUG: add import statsmodels.api for fix travis failure.

11c90d7

BUG: set nobs to integer to solve Travis CI error on python 3.

3933c0c

BUG/REF: python compatibility, clean up imports

0489e8e

REF compat 2to3 example

e0fcc33

josef-pkt added 3 commits February 14, 2022 12:50

REF/BUG: more python compat

3dc879a

REF/BUG: more py3 compat, fix missing variables (globals temporarily)

0ee664b

BUG/CLN: mxlogit: fix location of random options

06890fe

move example to module, add smoke test

josef-pkt added type-enh comp-discrete labels Feb 14, 2022

josef-pkt mentioned this pull request Feb 14, 2022

Dcm logit rebased #1605

Open

josef-pkt mentioned this pull request Feb 14, 2022

WIP: Conditional Logit and Mixed Logit #1120

Open

josef-pkt mentioned this pull request Mar 23, 2023

ENH: multinomial/categorical with different exog in choices, example 0-1 inflated beta #7918

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Dcm logit rebased2 #8120

ENH: Dcm logit rebased2 #8120

josef-pkt commented Feb 14, 2022

pep8speaks commented Feb 14, 2022

josef-pkt commented Feb 14, 2022

lgtm-com bot commented Feb 14, 2022

josef-pkt commented Feb 14, 2022 •

edited

bashtage commented Feb 14, 2022

josef-pkt commented Feb 14, 2022

ENH: Dcm logit rebased2 #8120

Are you sure you want to change the base?

ENH: Dcm logit rebased2 #8120

Conversation

josef-pkt commented Feb 14, 2022

pep8speaks commented Feb 14, 2022

josef-pkt commented Feb 14, 2022

lgtm-com bot commented Feb 14, 2022

josef-pkt commented Feb 14, 2022 • edited

bashtage commented Feb 14, 2022

josef-pkt commented Feb 14, 2022

josef-pkt commented Feb 14, 2022 •

edited