r311.Rmd

---
title: "r311: An R interface to the open311 standard"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{r311: An R interface to the open311 standard}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
httptest2::start_vignette("../man/figures/r311")
library(r311)
```


# Introduction

open311 is an international open-access standard for civic issue management
and public service communication. 
The standard allows administrations to better manage citizen requests,
citizens to more easily participate in administrative work, and researchers
and data scientists to access data regarding public service communication.
As an open standard, open311 is not a centralized API, but a framework
implemented by various cities (e.g. San Francisco, CA, Chicago, IL, Cologne,
DE, Turku, FI, Zurich, CH) and services (e.g. SeeClickFix, FixMyStreet).

It is way past the golden age of open311 APIs and much of development in
civic issue tracking has shifted to less open-access and less standardized
alternatives. Many former prime examples have abandoned or severely limited
their open311 endpoints. Nonetheless, open311 still constitutes a valuable
source for open government data. Many cities and services still maintain an
open311 service.

`r311` allows the seamless management and selection of endpoints and
retrieval of service and request data. It supports (but does not depend on)
many popular R frameworks such as the tidyverse, `sf` and `xml2` for
response formatting. `r311` is designed to be slim, both in content
and dependencies. It imports only two import-less packages used for HTTP
response handling. The functionality is limited to two main features:

- Endpoint management
- Sending requests

This vignette will briefly cover these two features.


# Endpoints

Since open311 is an open and decentralized standard, there is not one but
many endpoints that one can access. An endpoint is commonly implemented by
a city administration, but can also be managed by a service provider such
as FixMyStreet. Each endpoint can define multiple jurisdiction IDs, although,
in practice, most endpoints only define a single jurisdiction.

It can thus be difficult to manage the multitude of endpoints and jurisdictions.
Efforts have been made to list open311 servers, but most of them are
incomplete or outdated. `r311` offers an updated and modifiable endpoint list
that defines a number of open311 implementations that are defined for use in
the package. The list can be read using `o311_endpoints`.

```{r}
o311_endpoints()
```

The list does neither claim comprehensiveness nor up-to-dateness. It arguably
provides some of the most important and easily accessible endpoints as of 
`r format(Sys.Date(), "%Y")`. However, `r311` also offers the ability to add
new endpoints to `o311_endpoints` using `o311_add_endpoint`. You need to
provide a name (for lookup) and a root URL (the URL used to send requests).
The following code adds the open311 test server of Rostock, Germany.

```{r}
o311_add_endpoint(
  name = "Rostock Test",
  root = "https://demo.klarschiff-hro.de/backoffice/citysdk/",
  jurisdiction = "rostock.de"
)
```

Retrieving the endpoints list again confirms that you successfully added a 
new row to the endpoints dataframe.

```{r}
nrow(o311_endpoints())
```

You can now select the Rostock test API to the session using `o311_api`.
This function matches an API using endpoint name and jurisdiction ID and
attaches it to the active session. Query functions automatically detect the
attached API.

```{r}
o311_api("Rostock Test")
```

After attaching an API, `o311_ok` confirms that the selected endpoint is able
to handle request queries.

```{r}
o311_ok()
```

As the result is \code{TRUE}, you can safely start receiving real request data.



# Making requests

After selecting an API and attaching it to the session, all \code{o311_*}
functions can access it. You can now make requests.

## Services

To get an overview of the available services in a jurisdiction, you can
use `o311_services`, which returns a list of Rostock's administrative
services.

```{r}
o311_services()
```


## Requests

To get data about civic issues in the city area, run `o311_requests`.

```{r}
o311_requests()
```

Using the output of `o311_services`, you can further narrow down the output
of requests. Open311 defines a set of standard parameters which are implemented
by all endpoints. Using the `service_code` parameter with one of the previously
returned service codes, only complaints about businesses illegally using shared
spaces are returned.

```{r}
o311_requests(service_code = "16")
```

Similarly, using a `service_request_id` from the output, you can retrieve a
single service request from the API.

```{r}
o311_request("4")
```


## Bulk queries

Many endpoints define a page limit meaning that responses are divided into
pages. A query without parameters returns the first page. Pagination can be
controlled with the `page` argument. To control pagination, the
`o311_request_all` function can come in handy. It automatically iterates
through pages and heuristically decides when to stop. The following example
retrieves data from the first two pages, resulting in a tibble with 100
service requests.

```{r}
o311_api("Rostock, DE")
o311_request_all(max_pages = 2)
```


## Non-standard parameters

`r311` implicitly supports API extensions introducing custom paths and
parameters. One such API is Klarschiff Rostock which is based on CitySDK.
Klarschiff CitySDK defines a number of non-default paths and parameters which
extend the filtering abilities of open311 requests. Available parameters can
usually be found in the respective documentation (e.g. on GitHub for 
[Klarschiff CitySDK](https://github.com/bfpi/klarschiff-citysdk)).
The following query returns the last 50 requests tagged with the "idea" keyword.

```{r}
tickets <- o311_requests(keyword = "idea", max_requests = 50)
```

Some custom parameters can also alter the shape of the output. In the following
example, we query just the count of total requests using the `just_count`
parameter. The result is a 1×1 tibble containing a count value.

```{r}
o311_requests(just_count = TRUE)
```

The CitySDK extensions also offers additional URL paths which can be queried
using the generic `o311_query` function.

```{r}
poly <- o311_query("areas")
plot(poly$grenze)
plot(tickets$geometry, add = TRUE, pch = 16)
```


# Cleanup

Endpoint data is stored persistently between sessions so that you can create
your own database of open311 endpoints. This database is stored in the system's
user directory as returned by `tools::R_user_dir("r311")`. To reset the
database, run

```{r}
o311_reset_endpoints()
```

This will default back to the endpoints defined by `r311` and remove all
endpoints manually added by `o311_add_endpoints`.