index.Rmarkdown

---
title: "Basement Confidential: Vancouver's Informal Housing Stock" 
authors: 
  - Jens von Bergmann
  - Nathan Lauster
date: '2021-06-08'
slug: basement-confidential-vancouver-s-informal-housing-stock
categories:
  - affordability
  - cancensus
  - Empty Homes
  - land use
  - rental
  - Vancouver
  - zoning
tags: []
description: "Basement suites are the Schrödinger's cat of dwelling units, they span the space betweem formal and informal housing, viewed by some as the problem of, and by others as the solution to Vancouver's housing woes."
featured: ''
images: ["https://doodles.mountainmath.ca/blog/2021/06/08/basement-confidential-vancouver-s-informal-housing-stock/index_files/figure-html/suite-licenses-1.png"] 
featuredalt: "" 
featuredpath: ""
linktitle: ''
type: "post"
---


```{r setup, include=FALSE}
knitr::opts_chunk$set(
	echo = FALSE,
	message = FALSE,
	warning = FALSE,
	dpi = 150,
	fig.width = 8,
	fig.height = 6,
	cache = TRUE
)
library(tidyverse)
library(VancouvR)
library(cancensus)
library(statcanXtabs)
```

<p style="text-align:center;"><i>(Joint with Nathan Lauster and cross-posted at <a href="https://homefreesociology.com/2021/06/09/basement-confidential-vancouvers-informal-housing-stock/" target="_blank">HomeFreeSociology</a>)</i></p> 


# Informal housing
While housing is highly regulated via zoning bylaws, building code, and fire code, in situations of housing scarcity we often get informal housing that exists outside of - or only partially covered by - the existing regulatory framework. We often associate slums or shantytowns with the term *informal housing*, but it also applies to more organized settlements like Kowloon Walled City, or, in the context of subterranean Vancouver, [a good portion of our secondary suite stock](https://onlinelibrary.wiley.com/doi/10.1111/cag.12410).

Secondary suites in Vancouver span the spectrum between formal and informal housing. While secondary suites have long been a part of Vancouver's housing market, they only acquired a route to formal legalization across most of the City in 2004. However, to become fully formalized secondary suites still have to clear a number of regulatory hurdles, ranging from minimum dwelling size to minimum ceiling height and other building code and fire standards. Then, after paying a [small fee](https://vancouver.ca/doing-business/long-term-rental-business-licence.aspx), they can receive a city licence as a secondary suite rental unit. The vast majority of secondary suites still remain part of the un-licensed (and largely un-permitted) informal housing sector.

Even permitted and licensed secondary suites lack some of the regulatory tools that applies to other forms of housing. For example, neither the City of Vancouver Empty Homes Tax, nor the provincial Speculation and Vacancy Tax apply to suites. The flexibility of suites, which in most cases maintain an internal doorway to the main unit, mean they can easily be re-absorbed into the main dwelling unit. This weakens how the Residential Tenancy Act can be applied to tenancies in secondary suites, for instance by leaving landlords ready access to the "own use" rationale for eviction.

In general the informal market of secondary suites is mutually beneficial to owners, who can draw on additional rental income, and to tenants, who can access housing that is generally more affordable than what is available in the formal housing sector. The benefits don't accrue evenly though, and the interests of homeowning artisanal landlords and secondary suite tenants do not always align, [as evidenced by landlords looking for ways to circumvent the residential tenancy act by e.g. using suites as short-term rentals](https://www.theglobeandmail.com/real-estate/in-high-cost-vancouver-the-trick-is-getting-strangers-to-pay-the-rent/article15676047/).

Of course, informal basement suites are not a uniquely Vancouver form of informal housing, but they play an important role in Vancouver's housing market overall and, for better or worse, Vancouver housing discussions.

<video width="100%" autoplay="autoplay" loop="loop">
<source src="https://mountainmath.s3.ca-central-1.amazonaws.com/data/basements.mp4" type="video/mp4">
</video>

## The size of Vancouver's informal secondary suite market
Estimating the size of the informal housing market tends to be difficult precisely because the housing evades regulation, and Vancouver is no different in this regard. There have been a variety of attempts to quantify the number of secondary suites in the city, ranging from using census data, to roll data from BC Assessment, to MLS listings data (see [Metro Vancouver's last round-up estimates here](http://www.metrovancouver.org/services/regional-planning/PlanningPublications/MV_Housing_Data_Book.pdf#page=88)). Other potential sources of information include rental listings data.




```{r}
years <- seq(2001,2016,5)

vectors <- years %>% 
  lapply(function(y)setNames(
    paste0("v_CA",substr(y,3,4),c("xSD_5","xSD_32","xSD_14","xSD_23")),
    paste0(c("duplex_total_","duplex_Unoccupied_","duplex_Occupied by usual residents_",
             "duplex_Temporarily occupied_"),y))
    )%>%
  unlist

suites_data<-get_census("CA16",regions=list(CSD="5915022"),vectors=vectors) %>%
  pivot_longer(cols=matches("_\\d{4}$"),
               names_pattern = "^duplex_(.+)_(\\d{4})$",
               names_to=c("Document type","Year"),
               values_to="Count") %>%
  left_join(filter(.,`Document type`=="total") %>% select(Year,Total=Count),by="Year") %>%
  filter(`Document type`!="total") %>%
  mutate(name=factor(`Document type`,levels=c("Unoccupied","Temporarily occupied","Occupied by usual residents")))

tenure_dwellings<-get_sqlite_xtab("98-400-X2016227","https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/dt-td/CompDataDownload.cfm?LANG=E&PID=110570&OFT=CSV")

duplex_info <- tenure_dwellings %>%
  filter(`GEO_CODE (POR)`=="5915022",
         `Structural type of dwelling`=="Apartment or flat in a duplex") %>%
  select(`Household type including census family structure`,`Age of primary household maintainer`,
         Total=`Dim: Tenure (4): Member ID: [1]: Total - Tenure`,
         Owner=`Dim: Tenure (4): Member ID: [2]: Owner`,
         Renter=`Dim: Tenure (4): Member ID: [3]: Renter`) %>%
  collect()

detached_info <- tenure_dwellings %>%
  filter(`GEO_CODE (POR)`=="5915022",
         `Structural type of dwelling`=="Single-detached house") %>%
  select(`Household type including census family structure`,`Age of primary household maintainer`,
         Total=`Dim: Tenure (4): Member ID: [1]: Total - Tenure`,
         Owner=`Dim: Tenure (4): Member ID: [2]: Owner`,
         Renter=`Dim: Tenure (4): Member ID: [3]: Renter`) %>%
  collect()
  
close_sqlite_xtab(tenure_dwellings)
```

The Canadian Census has attempted to classify single family homes with secondary suites as two "duplex" units, the main unit and the secondary suite. In cases where the census determined there might be more than one suite the homes have been classified as "apartment, fewer than 5 storeys." But it's not always easy for Census workers to find suites or determine if or when they've been re-absorbed into the main dwelling. Nor is there a clear mechanism for residents filling out Census forms to make corrections. We can see how the Census has struggled to identify informal dwelling units by [observing the shift from "single-detached" dwellings to "duplex" units over time](https://doodles.mountainmath.ca/blog/2020/01/27/mythical-oversupply/).

```{r duplex-timeline}
ggplot(suites_data,aes(x=Year,y=Count,fill=name)) +
  geom_bar(stat="identity") +
  scale_y_continuous(labels = scales::comma) +
  labs(title="City of Vancouver duplex units",x=NULL,y="Number of units",fill=NULL,
       caption="StatCan Census 2001 through 2016")
```

The majority of the observed growth in duplex units is due to re-classification of existing units as the census got better at identifying secondary suites, initiated by the change in census methods between the 2001 and 2006 censuses and again between 2011 and 2016. Half these units were generally understood to be the main units in suited houses, the other half are the secondary suites. The census does not give us a straight-forward way to distinguish between the two. We note that a high share of duplex units register as "unoccupied" in the census, and we strongly suspect that many of these are secondary suites that have been re-absorbed into the main unit.

Understanding homes with more than one secondary suite through the census is hard, one way to get at this is to mix assessment and census data and look for "apartment, fewer than five storey" units in areas which are exclusively covered by "single family" homes (SFH). We have [done this in the past](https://doodles.mountainmath.ca/blog/2016/07/06/mixing-data/), and while this necessarily paints an incomplete picture, it can still reveal a [general sense of the prevalence of heavily suited homes and their geographic distribution](https://mountainmath.ca/census_mix/map?mapBase=2&layer=5&zoom=12&lat=49.2791&lng=-123.125). We have [previously estimated that there are around 7,000 SFH with more than one suite](https://doodles.mountainmath.ca/blog/2016/10/04/secondary-suites-and-taxes/), which come on top of the roughly 30,000 SFH with a single suite. We should probably revisit this estimate at some point with newer data and a more rigorous methodology.


To better understand who lives in these units we can again turn to census data, focusing in on duplexes (SFH with a single suite). This is only looking at the occupied units, and within these the units "occupied by usual residents". Splitting these units by tenure and age of the primary household maintainer, we notice that the maintainers of owner-occupied units skew older while those of rental units, likely mostly our secondary suites, skew younger.

```{r duplex-age}
duplex_info %>% 
  filter(`Household type including census family structure`=="Total - Household type including census family structure",
         `Age of primary household maintainer`!="Total - Age of primary household maintainer") %>%
  pivot_longer(c("Owner","Renter")) %>%
ggplot(aes(x=name,y=value,fill=`Age of primary household maintainer`)) +
  geom_bar(stat="identity") +
  scale_fill_viridis_d() +
  scale_y_continuous(labels=scales::comma) +
  labs(title="City of Vancouver households in secondary suite or main unit of suited SFH",
       caption="StatCan Census 2016",y="Number of dwelling units",x=NULL)
```

We also notice a curious pattern in that there are more owner-occupied units than total number of SFH of this type. Given that suites in the City of Vancouver cannot be stratified (and hence easily divided into separately owned properties), we might expect a more even division of owner-occupied to rental suites. We might even expect more rentals, to the extent that some houses are split into suites where both are rented out by landlords who live elsewhere. To find the opposite - that owner-occupied units dominate - can be interpreted multiple ways, speaking in part to the ambiguity of secondary suites. 

1. The census may be counting suites re-absorbed by owner-occupiers as still separate, but empty. This would boost owner-occupation rates. 
2. Rented secondary suites could simply turn over more often, leaving them temporarily empty around Census time, as when rented out to students. 
3. Or there may be other reasons those renting secondary suites may be less likely to respond to the long from Census. 
4. On the other side of the ledger, owner-occupying families and households may divide themselves up into suited homes and list themselves in the census in complicated ways. Children (or parents or other relatives) living in a secondary suites may not be paying rent or think of themselves as renters, but instead understand themselves to be part of an owner-occupied compound, even when reporting as a separate household from their parents within the Census.
5. Alternatively, those spread across multiple suites may still report themselves as part of the same Census household, in some cases leading to the reporting of complex household types.
6. A small number of "duplex" units may be stratified and allow for more than one owner.

Points 1, 2, and 5 only impact the renter/owner share, but can't explain why there are more owner households than overall buildings. Point 3 may lead to overreporting of owner households, but probably can't explain the discrepancy on it's own. Points 4 and 6 likely contribute to this. All of these points highlight the complexity of capturing information on informal housing.

We can examine the household types that occupy these units directly within the Census.

```{r duplex-hh_type}
hh_types <- c("Without children"="Couple without children",
              "With children"="Couple with children",
              "One lone-parent census family without other persons in the household"="Lone-parent census family",
              "Other census family households"="Complex census family households",
              "Non-census-family households"="Non-census-family households")

duplex_info %>% 
  filter(`Household type including census family structure` %in% names(hh_types),
         `Age of primary household maintainer`=="Total - Age of primary household maintainer") %>%
  mutate(hh_type=factor(hh_types[`Household type including census family structure`],levels=as.character(hh_types))) %>%
  pivot_longer(c("Owner","Renter")) %>%
ggplot(aes(x=name,y=value,fill=hh_type)) +
  geom_bar(stat="identity") +
  scale_fill_brewer(palette = "Dark2") +
  scale_y_continuous(labels=scales::comma) +
  labs(title="City of Vancouver households in secondary suite or main unit of suited SFH",
       fill="Houshold type",
       caption="StatCan Census 2016",y="Number of dwelling units",x=NULL)
```

We notice the elevated portion of non-census family households in secondary suites, which are either one-person households or roommate households. The ["Complex census family households"](https://doodles.mountainmath.ca/blog/2017/12/01/what-s-a-household/) are comprised of multi-generational households as well as households of census families with additional people living in the household. The prevalence of these on the ownership side probably speaks to suited homes being used by multi-generational households.


For comparison purposes, here is the same breakdown for single-detached homes, so those SFH where the census did not identify a secondary suite. This form still has a sizable portion of "complex" households, but at a lower share than what we see in suites SFH.

```{r sfh-hh_type}
detached_info %>% 
  filter(`Household type including census family structure` %in% names(hh_types),
         `Age of primary household maintainer`=="Total - Age of primary household maintainer") %>%
  mutate(hh_type=factor(hh_types[`Household type including census family structure`],levels=as.character(hh_types))) %>%
  pivot_longer(c("Owner","Renter")) %>%
ggplot(aes(x=name,y=value,fill=hh_type)) +
  geom_bar(stat="identity") +
  scale_fill_brewer(palette = "Dark2") +
  scale_y_continuous(labels=scales::comma) +
  labs(title="City of Vancouver households in single-detached (unsuited) homes",
       fill="Houshold type",
       caption="StatCan Census 2016",y="Number of dwelling units",x=NULL)

```


## The formal portion of secondary suite housing
To understand the formal portion of the secondary suite stock we can turn to City of Vancouver secondary suite rental licensing. There is a broad spectrum here between licensed units and those that have no hope of complying to building code because of e.g. ceilings that are too low. Some units may be easily license-able, but have not obtained a license for other unspecified reasons, including simple ignorance. While the City of Vancouver has a [relatively straight-forward webpage](https://vancouver.ca/doing-business/long-term-rental-business-licence.aspx) ddescribing the licensing process, the necessity of obtaining a license isn’t widely advertised, and some of the categories remain ambiguous. Have a look at the  [license application form](https://vancouver.ca/files/cov/long-term-rental-property-application.pdf).

```{r suite-licenses}
#search_cov_datasets("business")

licences <- get_cov_data("business-licences", where="folderyear='20'")

rental <- licences %>% 
  filter(status=="Issued") %>%
  filter(businesstype %in% c("Apartment House Strata","One-Family Dwelling","Short-Term Rental",
                                "Apartment House","Secondary Suite - Permanent","Duplex",
                             "Non-profit Housing")) %>% 
  count(businesstype) %>% 
  arrange(-n)


business_types <- c("Apartment House Strata","One-Family Dwelling","Short-Term Rental",
                                "Apartment House","Secondary Suite - Permanent","Duplex",
                             "Non-profit Housing")

where_clause <- business_types %>% 
  lapply(function(bt) paste0("businesstype = '",bt,"'")) %>% 
                                            unlist %>% paste0(collapse = " OR ")

suite_licences <- aggregate_cov_data("business-licences", 
                                     where=where_clause,#"businesstype='Secondary Suite - Permanent'",
                                     group_by = "folderyear, businesstype")
suite_licences2 <- aggregate_cov_data("business-licences-1997-to-2012", 
                                     where=where_clause, #"businesstype='Secondary Suite - Permanent'",
                                     group_by = "folderyear, businesstype")

bind_rows(suite_licences,suite_licences2 %>% mutate(folderyear=str_pad(folderyear,2,pad="0","left"))) %>%
  mutate(Year=paste0(ifelse(as.integer(folderyear)<22,"20","19"),folderyear)) %>%
  mutate(Date=as.Date(paste0(Year,"-01-01")))%>%
  filter(businesstype=='Secondary Suite - Permanent') %>%
  ggplot(aes(x=Date,y=count)) +
  #facet_wrap("businesstype") +
  scale_y_continuous(labels=scales::comma) +
  geom_bar(stat="identity",fill="deeppink3") +
  labs(title="Licensed secondary suites in the City of Vancouver",
       x=NULL,y="Number of licensed suites",
       caption="City of Vancouver Open Data")
```

Here we'll stick with secondary suite licenses, which is where landlords are supposed to go to formalize their rental suites. Looking at the number of licenses over time we note that the city has been successful in gradually bringing more secondary suites into the formal housing sector. But comparing to the Census it appears the overall share of formalized licensed suites is still quite low at around 10% of all rented suites.

## New suites
Suites in newly constructed housing will be up to building code, so there won't be any barriers to integrating them into the formal building stock. But how many new houses have suites? And how many suites get lost to teardowns. The latter question is hard to answer with publicly available data (and still difficult with non-open data sources like BC Assessment roll data and MLS data). But the answer to the former question, how many new homes have suites, can be obtained from building permit data.  We have [looked into this question a couple of years ago](https://doodles.mountainmath.ca/blog/2018/02/22/building-permits/), this is a good opportunity for an update with newer data.

```{r building-permits}
permits <- get_cov_data("issued-building-permits") %>%
  filter(propertyuse=="Dwelling Uses",
         specificusecategory %in% c("One-Family Dwelling","One-family Dwelling w/Sec Suite",
                                    "Secondary Suite","Dwelling Unit, One-Family Dwelling"),
         typeofwork %in% c("Addition / Alteration","Demolition / Deconstruction","New Building","Salvage and Abatement"))

permits %>% 
  filter(specificusecategory %in% c("One-Family Dwelling","One-family Dwelling w/Sec Suite")) %>%
  count(typeofwork,specificusecategory,year) %>%
  ggplot(aes(x=year,y=n,fill=specificusecategory)) +
  geom_bar(stat="identity",position="dodge") +
  facet_wrap("typeofwork") +
  scale_fill_manual(values=sanzo::duos$c033) +
  scale_y_continuous(labels=scales::comma) +
  labs(title="City of Vancouver building permits",
       x=NULL,y="Number of permits",fill=NULL,
       caption="City of Vancouver Open Data")
```

The addition/alterations data reflects the makeup of the SFH stock being formally altered and pulling a building permit for the work. Here we see that the majority of these homes do not have a suite, but the overall mixture of suited to non-suited homes roughly matches our expectation of the overall building stock. It is not clear how representative homes that take out a building permit for modifications might be. Owners of properties with existing code violations that might be expensive to fix may choose to forego a building permit as to avoid a building inspector visiting the house (and any casual investigation will reveal a lot of informal alterations in Vancouver).

Demolitions, as well as salvage/abatement permits (most demolitions require both types of permits) give an indication of the suited lots in the redevelopment process, although this may under-estimate the suites that are getting lost as unpermitted suites may not be declared prior to demolition. Homes that get demolished tend to be older. There is no information on how many homes with multiple suites get torn down, which would indicate a higher loss in suites than the data suggests at first sight. The jump in the distribution of units with and without suites between 2017 and 2018 looks suspiciously like a change in accounting methods, it will take more work to look into this.

Overall it would appear that new buildings tend to come with suites more often then not, and these suites will automatically be up to building code and easily permittable, if the owner wishes to rent them out. This indicates that the City of Vancouver is likely still gaining suites on net, but not at a high rate. Census data suggests that the addition of [suites, and laneway houses has barely managed to counteract the loss of population due to shrinking household sizes in many parts of Vancouver](https://doodles.mountainmath.ca/blog/2017/03/06/rs-population-change/).

Before wrapping up, let's quickly look at how the formalized addition of a suite relates to the building value of permitted houses constructed. 

```{r building-permit-values}
permits %>% 
  filter(specificusecategory %in% c("One-Family Dwelling","One-family Dwelling w/Sec Suite"),
         typeofwork=="New Building") %>%
  filter(projectvalue>300000) %>%
  ggplot(aes(x=year,y=projectvalue,fill=specificusecategory)) +
  #geom_violin() +
  scale_fill_manual(values=sanzo::duos$c033) +
  geom_boxplot() +
  scale_y_continuous(labels=scales::dollar) +
  labs(title="City of Vancouver new buildings",
       x=NULL,y="Median value of permits",fill=NULL,
       caption="City of Vancouver Open Data") +
  coord_cartesian(ylim=c(0,3000000))
```

Though the mismatch is diminishing, it's clear that SFH without suites tend to have more expensive building values, and are likely in more expensive parts of town.
This is something that we have generally seen in the distribution of suites, they tend to cluster on the East Side of Vancouver, with houses on the West Side remaining more exclusive and exclusionary.

## Upshot
Suites are great, and the flexibility of this type of housing adds relatively low-barrier options for households that can't find other places to live in low vacancy cities like Vancouver. For owners this means the ability to gain rental income to support their mortgage payments. For renters it means finding housing that - while often below building standards - often rents cheap. (The lower rent and sub-standard nature of suites are of course connected.)

This indicates that there is indeed a need and a market for housing that falls below the current building standards, where people are willing to trade centrality of location and lower rent for lower-standard dwelling units. Correspondingly, we may want to adapt our building code to: 

a. enable more secondary suites to become permitted and transition into the formal housing market, and
b. allow people to make some of these trade-offs in the formal housing market.

On the other hand, there is a large portion of suites that won't ever become part of the formal housing market because they still won't be permittable after reasonable modifications of the building code (and don't forget the fire code!). These suites exist because of a profound lack of alternatives, and policy should aim to build enough abundant rental options so that these units become unnecessary, as with similar strategies to replace [Single-Room Occupancy](https://vancouver.ca/files/cov/sro-revitalization-action-plan.pdf) with more livable, supportive, and affordable forms of housing. In the context of Metro Vancouver, this will likely be a long-term strategy. The housing shortage is too large to envision fully formalizing or replacing our informal housing stock anytime soon.

As usual, the code for this post is [available on GitHub](https://github.com/mountainMath/doodles/blob/master/content/posts/2021-06-08-basement-confidential-vancouver-s-informal-housing-stock/index.Rmarkdown).

<details><summary>Reproducibility receipt</summary>
```{r cache=FALSE}
## datetime
Sys.time()

## repository
git2r::repository()

## Session info
sessionInfo()
```
</details>