merge.Rmd

---
title: "Integrating Conflict Data"
output:
  workflowr::wflow_html:
    toc: false
editor_options:
  chunk_output_type: console
---

This page shows how to merge Geo-PKO data with conflict data and visualise the results. The examples used here are Uppsala University's ViEWS project, which forecasts conflict risk, and the Uppsala Conflict Data Programme (UCDP), one of the world's leading sources of data on armed conflict. Merging these datasets can provide insights into the links between conflict risk and peacekeeping deployments, and help policymakers make effective peacekeeping decisions where the risk of conflict is high.

## Setting up

Load packages.
```{r, warning=FALSE, message=FALSE}
library(dplyr)
library(sp)
library(tidyr)
library(geojsonio)
library(broom)
library(rgdal)
library(ggplot2)
library(leaflet)
library(sf)
library(spdep)
library(maptools)
library(plyr)
library(rjson)
library(RJSONIO)
library(rmapshaper)
library(htmltools)
library(htmlwidgets)
```

### Geo-PKO x ViEWS
First, we import the datasets. We're using the published Geo-PKO dataset, and conflict forecast data from ViEWS for state-based conflict, non-state conflict, and one-sided violence over the next 36 months in Africa. Both datasets offer insights into the sub-national level, using a unique "PRIO-grid" identifier. The PRIO-grid is an innovative geospatial unit from the Peace Research Institute Oslo that divides the world into roughly 100km x 100km squares, allowing geographic analysis beyond the country level to be streamlined.
```{r}
library(readr)
geopko <- readr::read_csv("data/Geo_PKO_v2.0.csv") 
#unzip("data/ViEWS.zip", exdir="data/ViEWS")
predictors <- read.csv("data/ViEWS/ensemble_pgm.csv")
```
The Geo-PKO dataset includes detail on troop deployment numbers, types of troops, non-troop deployments, and contributing countries.
 
```{r}
library(kableExtra)
kable(geopko[9546:9550,]) %>% kable_styling() %>%
  scroll_box(width = "100%", height = "200px")
```
## Filtering, joining and subsetting the datasets at the PRIO-grid level
The predictor database begins with July 2020 and forecasts the risk of conflict over the next 36 months ahead. Here's a preview of the data within it, showing state-based (`sb`), non-state (`ns`) and one-sided violence (`os`) forecasts. The variable `month_id` codes months differently to the Geo-PKO dataset, with every month assigned a different numeric value.

```{r}
kable(predictors[90545:90550,]) %>% kable_styling() %>%
  scroll_box(width = "100%", height = "200px")
```

The Geo-PKO dataset we're working with includes data from previous years, but for this visualisation, we will only use the latest year (July 2019 - June 2020). First, we need to filter the data to include only that period. This way we'll be looking at deployment status against projected conflict risk from the end of that period until three years from that period. You'll see both the Geo-PKO and ViEWS datasets include the PRIO-grid identification variable (`pg_id or PRIOID`), which corresponds to a specific grid square on the map. This is what we'll use to merge the datasets. In addition to filtering for the time period we want, we will also calculate the average number of troops deployed over that time period.

```{r}
# filtering for troop deployments over Jul 2019 - Jun 2020

geopko2 <- geopko %>%
  select(mission, year, month, prioid, no.troops, country, location, latitude, longitude) %>% 
  mutate_at(vars(longitude, latitude, year, month, no.troops), as.numeric) %>% 
  filter(year==2019 & month>6 | year==2020 & month<7)

# calculating an average number of troops

geopko3 <- geopko2 %>% 
  group_by(prioid) %>%
  dplyr::mutate(no.troops = mean(no.troops, na.rm=TRUE)) %>%
  ungroup() %>%
  filter (! duplicated(no.troops)) %>% 
  mutate(no.troops=round(no.troops))

```

Here, we also calculate the average conflict risk (state-based, non-state, and one-sided) for each location.

```{r}
predictors2 <- predictors %>% 
  group_by(pg_id) %>%
  dplyr::mutate(average_allwthematic_sb = mean(average_allwthematic_sb, na.rm=TRUE)) %>%
  dplyr::mutate(average_allwthematic_ns = mean(average_allwthematic_ns, na.rm=TRUE)) %>%
  dplyr::mutate(average_allwthematic_os = mean(average_allwthematic_os, na.rm=TRUE)) %>%
  filter (! duplicated(average_allwthematic_sb))
```


Finally, we merge the two datasets.

```{r}
# merging geopko with conflict forecast data

geopko3$pg_id <- geopko3$prioid

priogriddf <- full_join(
  geopko3, predictors2,
  by = c("pg_id"), 
  na.rm = TRUE)
  
```

## Merging the data and preparing the shapefile

Like we mentioned before, the PRIO-grid unit involves dividing the entire world into roughly 100km x 100km squares. That means that if we want to map it, we'll be working with large files, so keep that in mind when you're reading in the shapefile:

```{r, message=FALSE, warning=FALSE}
shapefile <- rgdal::readOGR("data/ViEWS/priogrid.geojson")
```

The shapefile contains both geospatial polygon data and numerical data that corresponds to the ViEWS dataset; specifically, a PRIO-grid ID and a country ID. Here's what the non-spatial data looks like, showing five rows in the dataset.
 
```{r}
kable(shapefile@data[101:106,]) %>% kable_styling() %>%
  scroll_box(width = "100%", height = "200px")
```
To work with the data within this shapefile, we need to fortify the shapefile. We also convert the IDs to rownames to make it easier to work with. And, finally, we merge it with `pgnewdf2`, which we created earlier.
```{r}
# fortify
shapefile@data$id <- rownames(shapefile@data)
shapefile.df <- fortify(shapefile, region = "id") 
# merge data of interest
shapefile.df <- merge(shapefile, priogriddf, by.x = "pg_id", by.y = "pg_id", all.x=F, all.y=T, duplicateGeoms=TRUE)
```
So now `shapefile.df` has the new attributes, including variables from both Geo-PKO and ViEWS. `id` is a variable that ties the 'polygon', or a single square on the grid, to its location on the map
 
```{r}
kable(shapefile.df@data[356:360,]) %>% kable_styling() %>%
  scroll_box(width = "100%", height = "200px")
```
## Mapping ViEWS data
To map the data, we're going to use the `leaflet` package (and a bunch of others to support it). The first thing we do is set up our colour palette and bins. We're using the 'viridis' colour palette, designed for accessibility and continuous-scale representation. The other thing we include is a small segment of code that fixes spacing between any NA value in the legend, and the remainder of the legend, making it easier to see.
```{r}
bins <- c(0, 10, 20, 50, 100, 200, 500, 1000, Inf)
pal <- colorNumeric("viridis", NULL)
#to fix spacing of NA in legend
css_fix <- "div.info.legend.leaflet-control br {clear: both;}" # CSS to correct spacing
html_fix <- htmltools::tags$style(type = "text/css", css_fix)  # Convert CSS to HTML
```
Next, let's map. We include three colour layers to shade squares according to their conflict forecast value. These layers cover state-based conflict, non-state conflict, and one-sided violence. Simple markers show where troops are deployed. Troop deployment numbers are included as labels, which you can see for each square on hover.

```{r}
map <- leaflet(shapefile.df) %>%
  addTiles() %>%
  addPolygons(color = "#444444", weight = 0.25, smoothFactor = 0.5,
              opacity = 0.05, fillOpacity = 0.4,
              fillColor = ~pal(shapefile.df$average_allwthematic_sb),
              group = "State-Based Conflict",
              highlightOptions = highlightOptions(color = "white", weight = 2,
                                                  bringToFront = FALSE)) %>%
  addPolygons(color = "#444444", weight = 0.25, smoothFactor = 0.5,
              opacity = 0.05, fillOpacity = 0.4,
              fillColor = ~pal(shapefile.df$average_allwthematic_ns),
              group = "Non-State Conflict",
              highlightOptions = highlightOptions(color = "white", weight = 2,
                                                  bringToFront = FALSE)) %>%
  addPolygons(color = "#444444", weight = 0.25, smoothFactor = 0.5,
              opacity = 0.05, fillOpacity = 0.4,
              fillColor = ~pal(shapefile.df$average_allwthematic_os),
              group = "One-Sided Violence",
              highlightOptions = highlightOptions(color = "white", weight = 2,
                                                  bringToFront = FALSE)) %>%
    addCircleMarkers((data = shapefile.df@data$no.troops>0), lat = ~latitude, lng = ~longitude, 
              weight = 1, radius = 2, fillOpacity = 0.6, color = "darkblue") %>%
  addPolygons(color = "#444444", weight = 0.1, smoothFactor = 0.5,
              opacity = 0.0, fillOpacity = 0.0,
              fillColor = ~pal(shapefile.df$no.troops),
              label=paste("Troops Deployed: ", shapefile.df$No.troops),
              labelOptions = labelOptions(
                style = list("font-weight" = "normal", padding = "3px 8px", color="blue"),
                textsize = "15px", direction = "auto"),
              highlightOptions = highlightOptions(color = "white", weight = 2,
                                                  bringToFront = FALSE)) %>%
  addLegend("bottomright",
            pal = pal,
            values = shapefile.df$average_allwthematic_sb,
            title = "Conflict Forecast",
            opacity = 1) %>%
  
  addLayersControl(
    baseGroups = c("State-Based Conflict", "Non-State Conflict", "One-Sided Violence"),
    options = layersControlOptions(collapsed = FALSE)
  )

map <- map %>% htmlwidgets::prependContent(html_fix) # legend NA fix

# to save as HTML, you can use the following code:
# saveWidget(map, file="pkoviews - priogrid.html")

map

```

And here we have it: an interactive map to view recent peacekeeping deployments (2019-2020) and projected conflict risk over the next 36 months. Extensions of this visualisation can be even more useful, particularly with a time-slider that can help us identify how the risk of conflict changes given peacekeeping deployments (and vice versa). That might be a project for the future.


### Geo-PKO x UCDP

Another useful dataset is from the Uppsala Conflict Data Programme, which offers insights into deaths from armed conflict. Looking at this data in conjunction with peacekeeping data can be useful to draw conclusions into peacekeeping given the severity of armed conflict, or lack thereof. How to merge with this dataset, and a few examples, will be added soon. We start with importing "UCDP Georeferenced Event Dataset (GED) Global version 20.1", of which a small excerpt of 2011 is shown in the table below. 

```{r}
UCDP <- read_csv("data/UCDP/ged201.csv")
kable(UCDP[9546:9550,]) %>% kable_styling() %>%
  scroll_box(width = "100%", height = "200px")
```

## Preparing the data
For the GeoPKO data we only used certain variables, and calculate the average troop number by year. 
The UCDP data has different variables you could use, in this example we took "best", which indicates "the best (most likely) estimate of total fatalities resulting from integer an event"(see UCDP codebook), and "deaths_civilians", which tells us how many of this "best" variable were civilian deaths.

```{r}
# Preparing the GeoPKO dataset
GeoPKO_dataUCDP <- geopko %>% 
  select(mission, year, location, latitude, longitude, no.troops) %>% #Select only the variables you need
  mutate_at(vars(latitude, longitude, no.troops), as.numeric) %>% 
  group_by(mission, year, location) %>% 
  mutate(ave.no.troops = as.integer(mean(no.troops, na.rm=TRUE))) %>% #Sum the troop numbers by year through using a combination of group_by & mutate
  select(-no.troops) %>% #Deselect the previous troop number variable
  distinct() %>% #Delete any duplicate rows
  drop_na(ave.no.troops)%>% #Remove NAs from the average troop count
  filter(ave.no.troops > 0) # Exclude any troop numbers under the value of 0

# Preparing the UCDP dataset
UCDP_dataframe <- UCDP %>% 
  select(year, longitude, latitude, where_coordinates, best,deaths_civilians)%>% 
  drop_na(latitude,longitude)  %>% 
  group_by(year, where_coordinates) %>% 
  mutate(best = as.integer(mean(best, na.rm=TRUE))) %>% #Take the mean of the "best" variable
  mutate(deaths_civilians = as.integer(mean(deaths_civilians, na.rm=TRUE)))%>% #Take the mean of the "deaths_civilians" variable
  filter(best > 0 & year>=1995)%>% #Filter that data so that it only keeps rows for the years after 1995. 
  distinct()
```

##Setting the aesthetics 
You can either set all the colours by hand, as seen in pal3, or use the "viridis" package to create a colour scale for you, as shown in pal2. For the UCDP data we used shades of red. The GeoPKO is mapped with viridis, which exists out of blue, green, and yellow. 

```{r}
pal2 <- colorBin((viridis::viridis(10)), GeoPKO_dataUCDP$ave.no.troops, bins = c(1,50,100,500,1000,2000,4000,8000))
pal3 <- colorBin(c("#700524","#8d072e","#981f42","#ed4d3a","#af516c","#d19bab","#dcb4c0"), 
                 UCDP_dataframe$best, bins = c(1,50,100,500,1000,2000,4000,10000,Inf))
```

## Mapping Geo-PKO
Just as we used leaflet for the VIEWS map, we do the same here. Note that in this case we did not merge the two datasets into one dataframe. When hovering over the circles more information on either the UN peacekeeping deployment will be provided or on the UCDP conflict-related deaths. It is possible to use the PRIO-grid ID instead, both datasets include these. 

```{r}
UCDP_Overview_Map <- leaflet() %>%
  addTiles(options = providerTileOptions(noWrap = TRUE)) %>%
  addMeasure(position = "bottomleft", primaryLengthUnit = "kilometers")%>% #Adds a widget that can measure distances between two places to the map
  clearMarkers()%>%
  clearShapes()%>% 
  addLegend("topright",
            pal = pal3,
            values = UCDP$best,
            title = "Fatalities",
             opacity = 1) %>%
  addLegend("topright",
            pal = pal2,
            values = GeoPKO_dataUCDP$ave.no.troops,
            title = "Peacekeepers",
            opacity = 1) %>% 
  addLayersControl(baseGroups = c("2019", "2018","2015","2010", "2005","2000","1995"),
                   options = layersControlOptions(collapsed = FALSE), position = "topleft")%>% 
  addCircleMarkers(data=(GeoPKO2019<-GeoPKO_dataUCDP %>% filter(year==2019)), 
                   color = ~pal2(ave.no.troops), radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2019",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2019$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2019$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2019$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2019<-UCDP_dataframe %>% filter(year==2019)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2019",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2019$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2019$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2019$deaths_civilians)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO2018<-GeoPKO_dataUCDP %>%filter(year==2018)), 
                   color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2018",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2018$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2018$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2018$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2018<-UCDP_dataframe %>%filter(year==2018)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2018",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2018$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2018$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2018$deaths_civilians)%>%  lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO2015<-GeoPKO_dataUCDP %>%filter(year==2015)), 
                   color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2015",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2015$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2015$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2015$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2015<-UCDP_dataframe %>%filter(year==2015)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2015",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2015$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2015$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2015$deaths_civilians)%>%  lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO2010<-GeoPKO_dataUCDP %>%filter(year==2010)), 
                   color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2010",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2010$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2010$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2010$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2010<-UCDP_dataframe %>%filter(year==2010)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2010",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2010$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2010$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2010$deaths_civilians)%>%  lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO2005<-GeoPKO_dataUCDP %>%filter(year==2005)), 
                   color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2005",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2005$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2005$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2005$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2005<-UCDP_dataframe %>%filter(year==2005)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2005",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2005$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2005$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2005$deaths_civilians)%>%  lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO2000<-GeoPKO_dataUCDP %>%filter(year==2000)), color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                   opacity = 0.15, fillOpacity = 0.5,
                   lng = ~longitude, lat = ~latitude, group = "2000",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO2000$mission,
                               "<br/><strong>Location:</strong>",GeoPKO2000$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO2000$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>%
  addCircleMarkers(data = (UCDP2000<-UCDP_dataframe %>%filter(year==2000)),
                   lng = ~longitude, lat = ~latitude, 
                   color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "2000",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP2000$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP2000$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP2000$deaths_civilians)%>%  lapply(htmltools::HTML)) %>%
  addCircleMarkers(data=(GeoPKO1995<-GeoPKO_dataUCDP %>%filter(year==1995)), color = ~pal2(ave.no.troops),radius = ~(ave.no.troops)^(1/3),
                    lng = ~longitude, lat = ~latitude,
                   opacity = 0.15, fillOpacity = 0.5,
                   group = "1995",
                   label=paste("<strong>UN Peacekeeping Site<br/>Mission:</strong>",GeoPKO1995$mission,
                               "<br/><strong>Location:</strong>",GeoPKO1995$location,
                               "<br/><strong>Troops Deployed:</strong>", GeoPKO1995$ave.no.troops)%>% 
                     lapply(htmltools::HTML)) %>% 
  addCircleMarkers(data = (UCDP1995<-UCDP_dataframe %>%filter(year==1995)),
                   lng = ~longitude, lat = ~latitude, color = ~pal3(best),radius = ~(best)^(1/3),
                   opacity = 0.05, fillOpacity = 0.4,
                   group = "1995",
                   label=paste("<strong>UCDP Reported Fatalities<br/>Location:</strong>", UCDP1995$where_coordinates,
                               "<br/><strong>Total deaths:</strong>", UCDP1995$best,
                               "<br/><strong>Civilian deaths:</strong>", UCDP1995$deaths_civilians)%>%  lapply(htmltools::HTML)) 

# to save as HTML, you can use the following code:
# saveWidget(UCDP_Overview_Map, file="geopko&ucdp - geopko.html")

#Shows the map
UCDP_Overview_Map
```
And here we have it again: an interactive map to view peacekeeping deployments, in a few selected years, and conflict-related deaths within that same year. We opted to allow the user to select different years to see how both the numbers and locations of both datasets change over time. 


## Merging the UCDP and GeoPKO data

If you do want to combine the two datasets into one dataframe there are different ways to do so. As shown with the VIEWS data, you can merge the datasets based on certain variables using "full_join", causing the two datasets to be combined in one file behind each other (it creates a dataframe in which the variables of one of the dataset occur next to the other), this can be utilized to the UCDP data as well. However, you can also merge datasets "under" each other, for the UCDP we wanted to show this second way of merging. Variables that are the same between the two datasets might go by different names, that is why some of the variable of the UCDP are renamed so that they are in line with the GeoPKO ones. The newly combined dataset will first show all the UCDP rows, and under these the GeoPKO rows will be "pasted". 
Merging can be usefull for all kinds of reasons, such as statistical analysis. 

```{r}
# Rename variables you want to merge
#UCDP_dataframeMerge = UCDP_dataframe %>% rename (location=where_coordinates)

# Merge the datasets
#Combined_UCDP_GeoPKO<- bind_rows(UCDP_dataframeMerge, GeoPKO_dataUCDP)
```