-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How can I validate my modeling for future periods? #348
Comments
Validating future projections, or projections in the present for areas without occurrence data, is difficult because you lack the occurrence data you need to calculate performance metrics that require them, like AUC. Other kinds of validation might include inspecting the prediction for environmental values outside those used for model training with MESS, for example, to determine areas of uncertainty. I guess a model that results in a prediction with big areas that have very negative MESS values might be considered "less valid" than another model that does not, but we might be stretching the term "validation" here. |
+1 to @jamiemkass. MESS (or MOP in KUENM) will show you in which areas your model predictions are most likely to fail. If you have enough occurrence data to do so, you could also try hindcasting the model and seeing how well it predicts occurrences at a past time point. Then you at least have a bit of an idea of how robust your model is through time. |
A lot of thanks. You can tell how can i do this on Wallace?
Em seg, 26 de abr de 2021 06:30, Hannah Owens ***@***.***>
escreveu:
… +1 to @jamiemkass <https://github.com/jamiemkass>. MESS (or MOP in KUENM
<https://github.com/marlonecobos/kuenm/>) will show you in which areas
your model predictions are most likely to fail. If you have enough
occurrence data to do so, you could also try hindcasting the model and
seeing how well it predicts occurrences at a past time point. Then you at
least have a bit of an idea of how robust your model is through time.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#348 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATY6KZU3X5YH2TOXW6VSIQ3TKUXELANCNFSM43KV3ISQ>
.
|
To run MESS, you first need to project your model (in area or time). Then, you just need to use the last module "Calculate Environmental Similarity" in the Project Component (or Tab). |
I ask because the AUC values made available in the modeling process are only for the modeling of the present time.
The text was updated successfully, but these errors were encountered: