Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging layers #1172

Open
dicorynia opened this issue Oct 10, 2019 · 4 comments

Comments

@dicorynia
Copy link

@dicorynia dicorynia commented Oct 10, 2019

I used list-columns and map + unnest to merge multiple shapefiles into one sf object. But it recently doesn't work any longer (it worked in March 2019). I'm not sure if it's related to sf or tidyr.

Reprex :

  1. get 3 shapefiles :
    library(tidyverse)
    library(sf)
    library(fs)
    library(httr)
    
    # some shapefiles from https://fr.actualitix.com/blog/shapefiles-des-departements-de-france.html
    url <-  c("https://fr.actualitix.com/blog/actgeoshap/01-Ain.zip",
              "https://fr.actualitix.com/blog/actgeoshap/73-savoie.zip",
              "https://fr.actualitix.com/blog/actgeoshap/74-haute-savoie.zip")
    
    dep <- str_extract(url, "\\d{2}.*$")
    
    list(url, dep) %>% 
      pwalk(~ GET(.x, write_disk(.y)))
    
    walk(dep, unzip, junkpaths = TRUE, exdir = "shp")
  1. merge
    res <- dir_ls("shp", glob = "*.shp") %>% 
      tibble(fname = .) %>%
      mutate(data = map(fname, read_sf)) %>%
      unnest(data) %>%
      st_as_sf()
    # Error: No common type for `..1$data$geometry` <sfc_POLYGON> and `..2$data$geometry` <sfc_MULTIPOLYGON>.
    # Call `rlang::last_error()` to see a backtrace  
 
    rlang::last_error()
    # <error>
    #   message: No common type for `..1$data$geometry` <sfc_    # Error in rbind.data.frame(...) : 
    #   numbers of columns of arguments do not match> and `..2$data$geometry` <sfc_MULTIPOLYGON>.
    # class:   `vctrs_error_incompatible_type`
    # backtrace:
    #   1. fs::dir_ls("shp", glob = "*.shp")
    # 16. tibble::tibble(fname = .)
    # 17. dplyr::mutate(., data = map(fname, read_sf))
    # 18. tidyr::unnest(., data)
    # 19. vctrs::stop_incompatible_type(x, y, x_arg = x_arg, y_arg = y_arg)
    # 1. vctrs:::stop_incompatible(...)
    # 16. vctrs:::stop_vctrs(...)

I don't know the inner workings of vctrs... Should sf "advertise" in some way that POLYGON and MULTIPOLYGON are compatible (are they ?) ?

Any other method to easily merge multiple layers ?
Since the structures of the files are slightly different (column numbers) I can't use rbind :

    res <- dir_ls("shp", glob = "*.shp") %>% 
      map(read_sf) %>% 
      do.call(rbind, .)
    # Error in rbind.data.frame(...) : 
    #   numbers of columns of arguments do not match

Thanks.

@dicorynia dicorynia changed the title Merging files Merging layers Oct 10, 2019
@edzer

This comment has been minimized.

Copy link
Member

@edzer edzer commented Oct 10, 2019

even if we force them to be all read as MULTIPOLYGON, using type = 6 in st_read(), we get

    res <- dir_ls("shp", glob = "*.shp") %>% 
      tibble(fname = .) %>%
      mutate(data = map(fname, read_sf, type = 6)) %>%
      unnest(data)
# Error: No common type for `..1$data$geometry` <sfc_MULTIPOLYGON> and `..2$data$geometry` <sfc_MULTIPOLYGON>.

Removing the geometry column makes this work.

As this might indeed be a vctrs issue (or my lack of understanding it), do @lionel- or @DavisVaughan have any idea?

@lionel-

This comment has been minimized.

Copy link

@lionel- lionel- commented Oct 10, 2019

You need to write vec_ptype2() and vec_cast() methods for your classes, see the S3 vignettes on the vctrs website.

We're planning a drastic simplification of the way a common type is declared (ptype2 methods). In the mean time you'll need to write the double dispatch methods.

You might also need to define vec-proxy and vec-restore methods. The proxy is the raw data that vctrs will concatenate or subset, and then the restore method restores attributes based on the result. You might need to define your proxy to return a data frame if you have vectorised attributes (i.e. metadata in your attributes for each element of the S3 vector).

I've been meaning to look into sf-vctrs compatibility, sorry I didn't have time to get to it yet.

@edzer

This comment has been minimized.

Copy link
Member

@edzer edzer commented Oct 10, 2019

Thanks - I think I'll wait for your drastic simplification before diving into this!

@dicorynia

This comment has been minimized.

Copy link
Author

@dicorynia dicorynia commented Oct 30, 2019

Thanks for investigating...
I'll wait for an update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.