Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TALK: What about backward, stepwise and other approaches? #7

Closed
tobwen opened this issue Dec 15, 2019 · 2 comments
Closed

TALK: What about backward, stepwise and other approaches? #7

tobwen opened this issue Dec 15, 2019 · 2 comments

Comments

@tobwen
Copy link

tobwen commented Dec 15, 2019

I know that might me not the best place to start a talk, but this was, others might be able to take place.

As a scientist, I'm torn. An automatic selection of variables are driven by statistical and not theoretical considerations. Of course, the whole thing depends on the research question and the research approach. Fortunately, that's a new field to be explored :-)

Are you planning to integrate backward, stepwise and other approaches of selection?

@tobwen tobwen changed the title TALK: What about _backward_, _stepwise_ and other approaches? TALK: What about backward, stepwise and other approaches? Dec 15, 2019
@HannaMeyer
Copy link
Owner

Backward selection is implemented in the caret package (https://topepo.github.io/caret/recursive-feature-elimination.html). But it cannot be used to avoid overfitting caused by to spatial dependencies in the data because it is based on the internal variable importance ranking (see https://www.sciencedirect.com/science/article/pii/S1364815217310976).

I fully understand your preference of a selection based on theoretical considerations. But an automatic selection does not mean there shouldn't be theoretical considerations in advance. But testing how certain variables behave in the statistical model is important to avoid overfitting and that can be challenging to see without an automatic strategy.

@tobwen
Copy link
Author

tobwen commented Dec 16, 2019

Thanks for your reply.

Yet, I don't have access to the paper via our university library. However, I have already asked to buy the article. I will test your approach intensively next week with my own data - I am very excited about the results.

@tobwen tobwen closed this as completed Dec 16, 2019
HannaMeyer pushed a commit that referenced this issue Feb 7, 2024
* implement quantitative density layer

* document parameter k

* comments

* just spaces to be in sync with qmd scripts

* change QD to LPD

* update validation and maxLPD derivation

* update plot and print (#2)

* add exploreAOA function (#3)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* Dev (#4)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* Dev (#6)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* Dev (#12)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* Change `maxLPD` parameter handling & enable `trainLPD` calculation with only training data provided (no model) (#7)

* add to maxLPD percantages

* update maxLPD parameter validation

* enable trainLPD calc with only trainDat (no model)

* minor change in maxLPD setting

* Add `LPDtoErrormetric` functionality (#8)

* add LPDtoErrormetric and adjust plot functions

* small change

* minor changes on docs and comments

* major docs update

* update DItoErrormetric and LPDtoErrormetric

* New calculation methods for LPD of training data and LPD of prediction data, add DI_LPDtoErrormetric  (#9)

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* Feat train di new (#11)

* Dev (#6)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* fix error in LPDtoErrormetric

* Dev (#13)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* Change `maxLPD` parameter handling & enable `trainLPD` calculation with only training data provided (no model) (#7)

* add to maxLPD percantages

* update maxLPD parameter validation

* enable trainLPD calc with only trainDat (no model)

* minor change in maxLPD setting

* Add `LPDtoErrormetric` functionality (#8)

* add LPDtoErrormetric and adjust plot functions

* small change

* minor changes on docs and comments

* major docs update

* update DItoErrormetric and LPDtoErrormetric

* New calculation methods for LPD of training data and LPD of prediction data, add DI_LPDtoErrormetric  (#9)

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* Feat train di new (#11)

* Dev (#6)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* fix error in LPDtoErrormetric

* masking order

* adapt default degrees of freedom

* add exp and scam exp errormodels

* update docs

* update docs

* Dev (#14)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* Change `maxLPD` parameter handling & enable `trainLPD` calculation with only training data provided (no model) (#7)

* add to maxLPD percantages

* update maxLPD parameter validation

* enable trainLPD calc with only trainDat (no model)

* minor change in maxLPD setting

* Add `LPDtoErrormetric` functionality (#8)

* add LPDtoErrormetric and adjust plot functions

* small change

* minor changes on docs and comments

* major docs update

* update DItoErrormetric and LPDtoErrormetric

* New calculation methods for LPD of training data and LPD of prediction data, add DI_LPDtoErrormetric  (#9)

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* Feat train di new (#11)

* Dev (#6)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* fix error in LPDtoErrormetric

* masking order

* adapt default degrees of freedom

* add exp and scam exp errormodels

* update docs

* update docs

* Dev (#15)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* Change `maxLPD` parameter handling & enable `trainLPD` calculation with only training data provided (no model) (#7)

* add to maxLPD percantages

* update maxLPD parameter validation

* enable trainLPD calc with only trainDat (no model)

* minor change in maxLPD setting

* Add `LPDtoErrormetric` functionality (#8)

* add LPDtoErrormetric and adjust plot functions

* small change

* minor changes on docs and comments

* major docs update

* update DItoErrormetric and LPDtoErrormetric

* New calculation methods for LPD of training data and LPD of prediction data, add DI_LPDtoErrormetric  (#9)

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* Feat train di new (#11)

* Dev (#6)

* update plot and print

* add exploreAOA to package

* add message for trainLPD calculation

* update explore AOA docs

* shift package to description

* update exploreAOA docs

* update exploreAOA docs and add xmark

* update aoa, trainDI and exploreAOA docs

* update aoa docs

* change order of print.aoa

* update aoa.Rd

* update DESCRIPTION

* update aoa & trainDI

* add trainDat upload options

* raster to terra and new layout (#5)

* update plot function

* update exploreAOA

* update exploreAOA

* fix error

* fix error

* fix crs bounding box issue

* rm commented code

* update plot.aoa & exploreAOA

* fix error in trainLPD calculation

* fix MD calculation

* fix order in print.aoa & message in aoa()

* add predictors tif to CAST package

* add countryboundaries

* add elevation

* update trainLPD implementation in trainDI

* add DI_LPDtoErrormetric

* update docs

* update `toErrormetric` functions

* update `...toErrormetric` functions

* update CAST docs

* change legend position in exploreAOA

* update aoa and .knndist - enhance performance of LPD calculation

* update trainDI - add progress bars and messages

* update DI_LPDtoErrormetric & LPDtoErrormetric to enable multiCV

* fix error in LPDtoErrormetric

* masking order

* adapt default degrees of freedom

* add exp and scam exp errormodels

* update docs

* update docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants