make_receiver_push_summary.qmd

---
params:
  project_name: NULL
  project_number: NULL
  qualified: NULL
  unqualified: NULL
  push_log: system.file("push_log.csv", package = "otndo")
  deployment: NULL
  since: NULL
format:
  html:
    self-contained: true
execute:
  echo: false
title: "`r paste('Receiver data push summary:', params$project_name)`"
author: "Created with the [`otndo` R package](https://otndo.obrien.page) (v. `r packageVersion('otndo')`)"
---
```{r, setup, include=FALSE}
knitr::opts_chunk$set(
  echo = FALSE, warning = FALSE, message = FALSE
)
```


```{r}
#| label: packages
#| message: false

library(otndo)
library(data.table)
library(mapview)
library(reactable)
```

```{r}
#| label: extraction-files-read

qualified <- data.table::fread(params$qualified)
unqualified <- data.table::fread(params$unqualified)

push_log <- data.table::fread(params$push_log)

if (!is.null(params$deployment)) {
  deployment <- data.table::fread(params$deployment)
}
```

```{r}
#| label: n-pis
#| warning: false

pis <- project_contacts(qualified, type = "receiver")
```

```{r}
#| label: otn-query

projects <- unique(qualified$trackercode)

otn_tables <- otn_query(projects)
```


```{r}
#| label: proper-urls

# if(act_eval){
#   act <- paste0('https://matos.asascience.com/project/detail/',
#                 params$project_number)
#   otn <- paste0('https://members.oceantrack.org/OTN/project?ccode=ACT.PROJ',
#                 params$project_number)
# }else{
act <- "https://matos.asascience.com/project"
otn <- "https://members.oceantrack.org/OTN/projects"
# }
```

These are data related to receiver data from the `r format(push_log$date[nrow(push_log)], '%B %Y')` OTN data push. New matched detection files (["qualified" files](https://members.oceantrack.org/data/otn-detection-extract-documentation-matched-to-animals#autotoc-item-autotoc-2)) were uploaded on `r format(push_log$date[nrow(push_log)], '%B %d')`. The previous data push occurred on `r format(push_log$date[nrow(push_log) - 1], '%B %d, %Y')`.


## Overall

A total of `r format(nrow(qualified), big.mark = ',')` detections of `r uniqueN(qualified, by  = 'fieldnumber')` individuals have been matched via [ACT](`r act`), [FACT](https://secoora.org/fact/projects/), and/or [OTN](`r otn`). The detections span across `r uniqueN(qualified, by = 'trackercode')` different projects consisting of `r length(unique(unlist(strsplit(pis$PI, ',\\s?'))))` principal investigators.


```{r}
#| label: otn-match-table
#| column: page
#| fig-cap: Table 1. Total detections and number of individual fish heard from other OTN node projects.
#| fig-cap-location: top

match_table(qualified, type = "receiver")
```

```{r}
#| label: otn-match-map
#| fig-cap: Figure 1. Bounding box of detected external projects. This does not necessarily reflect the distribution of fish heard by this project's receivers.

match_map(otn_tables)
```


```{r}
#| label: temporal-distribution
#| fig-cap: Figure 2. Temporal distribution of detections by external project, i.e., the dates when this project heard other project's fish.

temporal_distribution(qualified, "receiver")
```


```{r}
#| label: deployment-gantt
#| fig-cap: Figure 3. Deployment record of receivers across time. Black bars indicate periods of deployments, while red bars denote download dates.

deployment_gantt(deployment)
```


```{r}
#| label: station-summary-table
#| fig-cap: Table 2. Total number of detections and number of individual fish heard per project receiver.
#| fig-cap-location: top

station_summary <- station_table(qualified, type = "receiver")

reactable::reactable(station_summary)
```


```{r}
#| label: detection-map-leaflet
#| fig-cap: Figure 4. Number of detections per project receiver. Points represent receiver locations, while size and color reflect the number of detections at that location.

station_spatial <- prep_station_spatial(qualified, type = "receiver")

mapview::mapview(
  station_spatial,
  zcol = "Detections",
  cex = "Individuals",
  layer.name = "Matched detections"
)
```



```{r results='asis', echo=FALSE}
if (is.null(params$since)) {
  cat("<!---")
}
```
## What's new?

```{r}
#| label: new-matches
if (!is.null(params$since)) {
  since_date <- data.table::as.IDate(params$since)
} else {
  since_date <- data.table::as.IDate("1970-01-01")
}
new_matches <- qualified[datelastmodified > since_date]
```

Since `r format(since_date, '%B %Y')`, `r format(nrow(new_matches), big.mark = ',')` detections of `r uniqueN(new_matches, by  = 'fieldnumber')` individuals have been updated or newly-matched.

```{r}
#| label: new-otn-match-table
#| column: page
#| fig-cap: Table 3. Total number of new detections and individual fish heard per project receiver.
#| fig-cap-location: top
#| eval: !expr "nrow(new_matches) > 0"

match_table(new_matches, type = "receiver")
```

```{r}
#| label: new-otn-match-map
#| fig-cap: Figure 5. Bounding box of external projects detected since the last data push. This does not necessarily reflect the distribution of fish heard by this project's receivers.
#| eval: !expr "nrow(new_matches) > 0"

new_otn_tables <- lapply(
  otn_tables,
  function(.) {
    .[grepl(
      paste(gsub(".*\\.", "", unique(new_matches$trackercode)), collapse = "|"),
      collectioncode
    )]
  }
)

match_map(new_otn_tables)
```


```{r}
#| label: new-station-summary-table
#| fig-cap: Table 4. Total number of detections and number of individual fish heard per project receiver since the last data push.
#| fig-cap-location: top
#| eval: !expr "nrow(new_matches) > 0"

new_station_summary <- station_table(new_matches, "receiver")

reactable::reactable(new_station_summary)
```
</figure>

```{r results='asis', echo=FALSE}
if (is.null(params$since)) {
  cat("-->")
}
```

## Unmatched detections

```{r}
#| label: unqualified

unqual_f <- unqualified[, .N, by = "fieldnumber"][N <= 1]
data.table::setnames(unqual_f, c("Transmitter", "N"))

unqual_real <- unqualified[, .N, by = "fieldnumber"][N > 1]
data.table::setnames(unqual_real, c("Transmitter", "N"))
```

Some detections were not matched to other investigators via OTN nodes. There are `r format(nrow(unqualified), big.mark = ',')` of these ["unqualified" detections](https://members.oceantrack.org/data/otn-detection-extract-documentation-matched-to-animals#autotoc-item-autotoc-2) across `r data.table::uniqueN(unqualified, by  = 'fieldnumber')` transmitters.

`r nrow(unqual_f)` of these are very likely false, as they only have one detection; this leaves detections of `r nrow(unqual_real)` transmitters that could find a home elsewhere.

### Likely false detections (N detections = 1)

```{r}
#| label: unqualified-false


## NOTE TO SELF START HERE
reactable(unqual_f)
```

### Possibly real detections (N detections > 1)

```{r}
#| label: unqualified-real

reactable(unqual_real)
```


```{r}
#| label: temporal-distribution-unqual
#| fig-cap: Figure 6. Temporal distribution of unmatched detections by receiver, i.e., the dates when a receiver in this project heard a transmitter that was not matched to an external project.

temporal_distribution(unqualified, "receiver")
```