Complete rewrite of the Ecology chapter #771

jannes-m · 2022-04-14T13:58:57Z

Hello everyone,

in this PR I have:

converted everything from mlr to mlr3
replaced raster by terra
replaced RQGIS by qgisprocess
adjusted the text accordingly
updated the exercises accordingly

To keep in mind:

this branch only works with the github version of spDataLarge
DESCRIPTION file of geocompkg needs to be udpated (mlr3packages and alike are still missing) -> should I put in a corresponding geocompkg PR (would be probably the best solution)?
before merging this branch, we should merge rewrite_cv since rewrite_eco is a child branch of rewrite_cv. The only thing I haven’t done in rewrite_cv apart from a few other minor things is to run mlr3::benchmark() because it will probably take ages to finish...

…rediction and plotting predictions

Merge branch 'main' into rewrite_eco # Conflicts: # 01-introduction.Rmd # 03-attribute-operations.Rmd # 05-geometry-operations.Rmd # 16-synthesis.Rmd # refs.bib

Robinlovelace · 2022-04-15T19:13:16Z

Many thanks @jannes-m this looks like an impressive piece of work. Will take a look now.

Robinlovelace · 2022-04-15T19:26:37Z

First step: fix merge conflicts with the main branch. I'll give that a go now but may need your assistance @jannes-m if there are any ambiguities..

Robinlovelace · 2022-04-15T19:28:28Z

Seems this will merge it:

git checkout main
git merge --no-ff rewrite_eco
git push origin main

Robinlovelace · 2022-04-15T19:28:52Z

01-introduction.Rmd

@@ -116,7 +116,7 @@ With a wide range of packages, R also supports advanced geospatial statistics\in
 \index{R!language}
 New integrated development environments (IDEs\index{IDE}) such as RStudio\index{RStudio} have made R more user-friendly for many, easing map making with a panel dedicated to interactive visualization.

-At its core, R is an object-oriented, [functional programming language](http://adv-r.had.co.nz/Functional-programming.html) [@wickham_advanced_2019], and was specifically designed as an interactive interface to other software [@chambers_extending_2016]. 
+At its core, R is an object-oriented, [functional programming language](https://adv-r.hadley.nz/fp.html) [@wickham_advanced_2019], and was specifically designed as an interactive interface to other software [@chambers_extending_2016]. 


Robinlovelace · 2022-04-15T19:29:09Z

12-spatial-cv.Rmd

@@ -6,21 +6,25 @@ This chapter assumes proficiency with geographic data analysis\index{geographic
 A familiarity with generalized linear models (GLM)\index{GLM} and machine learning\index{machine learning} is highly recommended [for example from @zuur_mixed_2009;@james_introduction_2013].

 The chapter uses the following packages:^[
-Package **kernlab**, **pROC**, **RSAGA**\index{RSAGA (package)} and **spDataLarge** must also be installed although these do not need to be attached.
+Packages **GGally**, **lgr**, **kernlab**, **ml3measures**, **paradox**, **pROC**, **progressr** and **spDataLarge** must also be installed although these do not need to be attached.


Robinlovelace · 2022-04-15T19:29:14Z

12-spatial-cv.Rmd

 library(sf)
-library(raster)
+library(terra)


Robinlovelace · 2022-04-15T19:29:59Z

12-spatial-cv.Rmd

-library(mlr)
-library(parallelMap)
+```{r 12-spatial-cv-1-2, eval=FALSE}
+library(future)


New dependency I think. We should update geocompkg DESCRIPTION if so.

See PR here: geocompx/geocompkg#32

Robinlovelace · 2022-04-15T19:40:20Z

12-spatial-cv.Rmd

@@ -69,83 +73,64 @@ Spatial CV\index{cross-validation!spatial CV} alleviates this problem and is the
 ## Case study: Landslide susceptibility {#case-landslide}

 This case study is based on a dataset of landslide locations in Southern Ecuador, illustrated in Figure \@ref(fig:lsl-map) and described in detail in @muenchow_geomorphic_2012.
-A subset of the dataset used in that paper is provided in the **RSAGA**\index{RSAGA (package)} package, which can be loaded as follows:
+A subset of the dataset used in that paper is provided in the **spDataLarge**\index{spDataLarge (package)} package, which can be loaded as follows:


Robinlovelace · 2022-04-15T19:50:16Z

15-eco.Rmd

 library(dplyr)
+library(mlr3)


Same comment: I suggest not loading all of these, some can be used with pkg::fun().

This probably won't work. For example, you need to attach mlr3spatiotempcv to make available the spatial partitioning stuff used in mlr3::rsmp(). Equally, you need to attach mlr3learners and mlr3extralearners to make available additional learners. And the autoplot() function will only work with benchmark results if you have previoulsy attached mlr3viz to make the autoplot(). And there are more examples.

Robinlovelace · 2022-04-15T19:51:40Z

15-eco.Rmd

@@ -40,7 +47,7 @@ In this chapter we will demonstrate ecological applications of some of the techn
 This case study will involve analyzing the composition and the spatial distribution of the vascular plants on the southern slope of Mt. Mongón, a *lomas* mountain near Casma on the central northern coast of Peru (Figure \@ref(fig:study-area-mongon)).

 ```{r study-area-mongon, echo=FALSE, fig.cap="The Mt. Mongón study area, from Muenchow, Schratz, and Brenning (2017).", out.width="60%", fig.scap="The Mt. Mongón study area."}
-knitr::include_graphics("figures/study-area-mongon.png")
+knitr::include_graphics("figures/15_study_area_mongon.png")


Robinlovelace · 2022-04-15T19:52:00Z

15-eco.Rmd

  dplyr::pull(dem)
 # rotating NMDS in accordance with altitude (proxy for humidity)
-rotnmds = MDSrotate(nmds, elev)
+rotnmds = vegan::MDSrotate(nmds, elev)


Robinlovelace · 2022-04-15T19:52:33Z

15-eco.Rmd

+library(ranger)
+library(sf)
+library(terra)
+library(tree)
 library(vegan)


Do we need to load it if we're using it below with vegan::?

probably not

Robinlovelace · 2022-04-15T19:53:37Z

_15-ex.Rmd

+library(progressr)
+library(qgisprocess)
+library(terra)
+library(tictoc)


New dep. See geocompx/geocompkg#32

Robinlovelace · 2022-04-15T19:55:10Z

These are major changes but they are clearly good ones. I would err on the side of merging soon and tidying up early? Thoughts welcome @jannes-m and @Nowosad, there are a few suggestions in my comments above that may be worth conversation but overall pretty non-controversial stuff. Really happy to see this moving forward.

jannes-m · 2022-04-16T22:11:31Z

First step: fix merge conflicts with the main branch. I'll give that a go now but may need your assistance @jannes-m if there are any ambiguities..

Acutally, there shouldn't be any merge conflicts with the main branch since I harmonized with the main branch regularly but well, you never know....

jannes-m · 2022-04-16T22:59:12Z

Robin, thanks for your review! I have polished further chapters 12 and 15. And I think we can merge, and fix soon if something goes amiss as you have proposed!

jannes-m · 2022-04-17T14:27:06Z

FYI: will compile the book locally and try to resolve all issues occurring in chapters 12 and 14. Will try to do so this night.

…ile size

jannes-m · 2022-04-18T00:21:42Z

mmh, there are still problems when trying to read mlr3 benchmark output results via readRDS() when knitting. Not yet sure how to solve this since it works when running readRDS() on the same files in the console. Probably I need to ask the mlr3 team and in the meantime we hard-code the results (though this is neither elegant nor reproducible). But now I need to get some sleep...

jannes-m · 2022-04-18T00:40:58Z

ok, I think I have found a better solution. I will implement it in the coming days.

Robinlovelace · 2022-04-18T14:42:21Z

Thanks for this mega PR Jannes, great to have it merged to keep the momentum going. We can do post merge fixes!

@Nowosad plans to do a post merge review, should be easier with reference to the updated version online.

jannes.muenchow and others added 30 commits August 3, 2021 14:58

start to write mlr3 code

aa1a728

add 2 remarks

d313e4b

add svm autotuner and mlr3 advantages

6e7907d

rewrite case study section

5aa0871

update pat-s citation

2dbdc49

add mlr3 book reference

8f66590

evaluate spDataLarge data code chunk

20f9d57

update adv-r ref

9f82d04

create a mlr3 taskclassifst

5d223b6

delete superfluous following

a5533b7

add to-dos

29e4b93

add missing comma

e30f2f0

create log_reg learner and run resampling

a7edbfa

start writing the tuning section

c74ef1f

use lsl study area

019fd19

replace all instances of study_area by study_mask

dcd952b

add future parallelization

665565b

replace mlr by mlr3 output

a83cf3b

use benchmark(_grid) for comparing sp with conventional cv, rewrite p…

36bf6c2

…rediction and plotting predictions

replace old boxplot, rewrite parallel stuff

162cb19

update contents

ee9a95b

rewrite svm stuff using mlr3

fb5cc34

delete old readRDS commands

aab55e3

run glm and svm sp and nsp cv using benchmark

d7a633d

use only one script for glm and svm cvs

f2ce68d

update svm resampling and exercises

c5f2c7e

mlr3 code for predicting floristic composition

845bd3a

update 14-rf_mlr3.R

8a68e8d

harmonize with main

d245799

Merge branch 'main' into rewrite_eco # Conflicts: # 01-introduction.Rmd # 03-attribute-operations.Rmd # 05-geometry-operations.Rmd # 16-synthesis.Rmd # refs.bib

rename code and output files

3b57594

Robinlovelace marked this pull request as ready for review April 15, 2022 19:12

Robinlovelace approved these changes Apr 15, 2022

View reviewed changes

jannes-m added 3 commits April 16, 2022 23:33

use system.file to attach ta

ce69f05

replace raster by terra in _12-ex.Rmd

564f4c3

Merge branch 'rewrite_cv' into rewrite_eco

60c0596

jannes-m added 6 commits April 17, 2022 00:28

replace raster by terra and use new dplyr syntax

062c439

use :: style to make clear which packages are used when

20e32dd

fix typo

127273a

delete old, no longer necessary sentence from comments

13b7b65

delete merge conflict HEAD

dd58c32

Merge branch 'rewrite_cv' into rewrite_eco

60d14f8

jannes-m added 3 commits April 18, 2022 01:15

further polish 12-spatial-cv.Rmd

2e6c789

delete no longer required mlr3 file

b305dc2

check if not storing the models decreases substantially runtime and f…

57358be

…ile size

jannes-m added 2 commits April 18, 2022 02:51

use only score output

5f691a8

Merge branch 'rewrite_cv' into rewrite_eco

92c3492

Robinlovelace approved these changes Apr 18, 2022

View reviewed changes

Robinlovelace merged commit 92c3492 into main Apr 18, 2022

Robinlovelace deleted the rewrite_eco branch April 18, 2022 14:41

jannes-m mentioned this pull request Apr 19, 2022

Chapter 15 failing #773

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Complete rewrite of the Ecology chapter #771

Complete rewrite of the Ecology chapter #771

jannes-m commented Apr 14, 2022 •

edited

Loading

Robinlovelace commented Apr 15, 2022

Robinlovelace commented Apr 15, 2022

Robinlovelace commented Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

jannes-m Apr 16, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

Robinlovelace Apr 15, 2022

jannes-m Apr 16, 2022

Robinlovelace Apr 15, 2022

Robinlovelace commented Apr 15, 2022

jannes-m commented Apr 16, 2022

jannes-m commented Apr 16, 2022

jannes-m commented Apr 17, 2022

jannes-m commented Apr 18, 2022

jannes-m commented Apr 18, 2022

Robinlovelace commented Apr 18, 2022

		library(dplyr)
		library(mlr3)

Complete rewrite of the Ecology chapter #771

Complete rewrite of the Ecology chapter #771

Conversation

jannes-m commented Apr 14, 2022 • edited Loading

Robinlovelace commented Apr 15, 2022

Robinlovelace commented Apr 15, 2022

Robinlovelace commented Apr 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Robinlovelace commented Apr 15, 2022

jannes-m commented Apr 16, 2022

jannes-m commented Apr 16, 2022

jannes-m commented Apr 17, 2022

jannes-m commented Apr 18, 2022

jannes-m commented Apr 18, 2022

Robinlovelace commented Apr 18, 2022

jannes-m commented Apr 14, 2022 •

edited

Loading