-
Notifications
You must be signed in to change notification settings - Fork 5
/
akfin-api-r.qmd
105 lines (81 loc) · 3.46 KB
/
akfin-api-r.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
---
title: Access API data using R
number-sections: true
number-depth: 3
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(eval = TRUE, echo = TRUE)
```
AKFIN has developed web services (apis) to distribute GAP data. Like the GAP_PRODUCTS schema, these are under active development. These do not require VPN or an oracle connection but they are protected by Oracle authentication, please contact matt.callahan@noaa.gov for information on how to get an api token to use this option.
The url structure is "https://apex.psmfc.org/akfin/data_marts/gap_products/gap_[base table name]" . For example "https://apex.psmfc.org/akfin/data_marts/gap_products/gap_biomass" is the base url to get data from the akfin_biomass table. Web services linked to large tables have mandatory parameters to reduce data download size. For example to get agecomp data for Bering Sea pollock in area_id 10 in 2022 you would use "https://apex.psmfc.org/akfin/data_marts/gap_products/gap_biomass?survey_definition_id=98&area_id=10&species_code=21740&start_year=2022&end_year=2022”.
If you’re using R to pull data through web services you might find the [akfingapdata](https://github.com/MattCallahan-NOAA/akfingapdata/tree/main) (pronounced akfin-gap-data not ak-eff-ing-app-data) R package helpful.
```{r akfin-api-fn}
# load libraries
library(dplyr)
library(magrittr)
library(httr)
library(flextable)
# tell R to not use scientific notation
options(scipen=999)
# function for pulling data from the api using the httr package
get_gap_biomass<-function(area_id, species_code) {
# paste(... collapse=",") puts commas between vector elements
area_id <- paste(area_id, collapse = ",")
species_code <- paste(species_code, collapse = ",")
# httr code, parameters are after the '?'
httr::content(
httr::GET(paste0("https://apex.psmfc.org/akfin/data_marts/akmp/gap_biomass?area_id=",
area_id,
"&species_code=",
species_code)),
type = "application/json") %>%
# convert to data frame
bind_rows()
}
```
## Ex. Direct database query in R using the (akfingapdata readme)[https://github.com/MattCallahan-NOAA/akfingapdata/blob/main/README.Rmd] R package:
```{r}
#| label: test-2a
#| eval: false
# load packages
library(odbc)
library(getPass)
library(tidyverse)
# connect to AKFIN Oracle database
con <- dbConnect(odbc::odbc(), "akfin", UID=getPass(msg="USER NAME"), PWD=getPass())
```
```{r}
#| label: test-2b
#| eval: false
#| echo: false
con <- channel_akfin
```
```{r}
#| label: test-2c
#| tbl-cap: "Ex. 2: Load data with {akfingapdata}. "
#| eval: false
# define species code for pollock
my_species <- 21740
#query database
data<- dbFetch(dbSendQuery(con,
paste0("select * from gap_products.akfin_biomass
where species_name = ", my_species,
" and survey_definition_id = 98,
and area_id = 10"))) %>%
rename_with(tolower) # everyone likes lower case letters better
head(data)
```
## Ex. Direct database query in R using the (akfingapdata readme)[https://github.com/MattCallahan-NOAA/akfingapdata/blob/main/README.Rmd] R package:
```{r}
#| label: test-3
#| tbl-cap: "Ex. 2: Load catch data with {akfingapdata}. "
#| eval: true
#| echo: true
library(akfingapdata)
# Sign into akfin with token (need to request token from AKFIN)
token <- akfingapdata::create_token(file = paste0(dirname(here::here()), "/akfin_token.txt"))
akfingapdata::get_gap_catch()[,1:6] %>%
head() %>%
flextable::flextable() %>%
flextable::theme_zebra()
```