5 write muskrat data prep scripts

[SanderDevisscher]

fixes #5

Deze workflow werkt de logica uit om de muskusratten data van gbif binnen te halen en te koppelen met de gemeentes en de provincies.

Na de goedkeuring wordt deze flow iedere maandag getriggered.

[EmmaCartuyvels1]

These will need to updated once all the data is published, at the moment we are still missing non-RATO East-Flemish data and West-Flemish data. Best to check with @LienReyserhove or @DimEvil

[EmmaCartuyvels1]

Also, these datasets contain both catches and obervations, it would be good to distinguish between those

[SanderDevisscher]

any suggestions how ?

[EmmaCartuyvels1]

There is a field in the GBIF export called "Sampling Protocol"
For example in the "VMM - Rat control occurrences in Flanders, Belgium" this is coded as either "RAT TRAP" or "casual observation". Although I think a lot of catches are also marked as casual observation, so again this would need to be checked within the data or with Dimi or Lien

[SanderDevisscher]

So drop casual observation when dataset == "VMM - Rat control occurrences in Flanders, Belgium" ?

[EmmaCartuyvels1]

If it is correct in the dataset, but I'm really not sure about this.

[EmmaCartuyvels1]

Looks good!

[SanderDevisscher]

Options in sampling protocol column

                                              bait trap 
                                          97309                                                1911 
                                      Bait trap                Bait trap, Conibear trap, Fish traps 
                                              3                                                   1 
                                 bait trap;fike                          bait trap;fike;ground trap 
                                             83                                                   2 
                       bait trap;fike;raft trap                bait trap;fike;raft trap;ground trap 
                                             48                                                   1 
                          bait trap;ground trap                      bait trap;material lost/broken 
                                              7                                                   1 
                            bait trap;raft trap                                  Bak/buis geplaatst 
                                             71                                                   4 
                             casual observation                                       conibear trap 
                                           8892                                                3257 
                                  Conibear trap                            Conibear trap, Bait trap 
                                             16                                                   1 
                      Conibear trap, Fish traps                             conibear trap;bait trap 
                                              8                                                 136 
                   conibear trap;bait trap;fike            conibear trap;bait trap;fike;ground trap 
                                             68                                                  53 

conibear trap;bait trap;fike;material lost/broken conibear trap;bait trap;fike;raft trap
1 18
conibear trap;bait trap;fike;raft trap;ground trap conibear trap;bait trap;ground trap
18 36
conibear trap;bait trap;material lost/broken conibear trap;bait trap;raft trap
6 37
conibear trap;bait trap;raft trap;ground trap conibear trap;fike
9 1062
conibear trap;fike;ground trap conibear trap;fike;ground trap;material lost/broken
266 1
conibear trap;fike;material lost/broken conibear trap;fike;raft trap
17 11
conibear trap;fike;raft trap;ground trap conibear trap;ground trap
4 333
conibear trap;material lost/broken conibear trap;raft trap
38 30
conibear trap;raft trap;ground trap connibear clamp
9 3444
fike fike;ground trap
2383 47
fike;material lost/broken fike;raft trap
4 66
fike;raft trap;ground trap Fish traps
3 49
Fish traps, Bait trap Fish traps, Conibear trap
1 3
Fish traps, Conibear trap, Bait trap fyke
1 1377
ground trap Ground trap
372 2
Ground trap, Conibear trap ground trap;material lost/broken
1 1
material lost/broken Other
8 10
raft trap raft trap;ground trap
521 4
rat trap round fikes
9971 537
valve trap
593

[EmmaCartuyvels1]

8892

This looks like way too many casual observations

[SanderDevisscher]

all casual observations are from the VMM dataset. This dataset also includes 6078 records with sampling protocol == "rat trap"

[SanderDevisscher]

I suggest dropping "casual observations" as well as all "lost/broken" material

Muskrat captures also contains a large body of sampling protocol "" (n = 95940), MICA east flanders has 528 of these and RATO 841. What should we do with these ?

stuff has changed

[SanderDevisscher]

There are also 18983 records with NA as individual count. What to do with these ?
9977 after dropping casual observations & lost/broken material.