Efficient preallocation?

[cseveren]

Preallocation of simple features seems to still be pretty memory intensive, likely (?) because geometry size is unknown ex ante. Thus read-write with a large pre-allocated object is much slower than with a small pre-allocated object. Is there a way to make this more efficient? (Note that I cross-listed here because I'm not sure if this is an "issue" or something that just needs a workaround).

Here is a simple reprex illustrating this issue:

rm(list = ls())
library(sf)
library(tidyverse)

## Initialize size to 100 rows, populate with 100 features ##
start_time100 <- Sys.time()
rsize <- 100

route <- st_sf(id = 1:rsize, geometry = st_sfc(lapply(1:rsize, function(x) st_linestring() )), crs = 4326)

for (i in 1:100) {
  sln <- rbind(c(runif(1),runif(1)), c(runif(1),runif(1)), c(runif(1),runif(1)))
  route_temp <- st_linestring(sln, dim = "XY") %>% 
    st_sfc(crs = 4326)
  (route[i,] <- st_sf(id = i, geometry = route_temp))
}

route <- route[!st_is_empty(route),]
end_time100 <- Sys.time()

## Initialize size to 10000 rows, populate with 100 features ##
start_time10k <- Sys.time()
rsize <- 10000

route <- st_sf(id = 1:rsize, geometry = st_sfc(lapply(1:rsize, function(x) st_linestring() )), crs = 4326)

for (i in 1:100) {
  sln <- rbind(c(runif(1),runif(1)), c(runif(1),runif(1)), c(runif(1),runif(1)))
  
  route_temp <- st_linestring(sln, dim = "XY") %>% 
    st_sfc(crs = 4326)
  (route[i,] <- st_sf(id = i, geometry = route_temp))
}

route <- route[!st_is_empty(route),]
end_time10k <- Sys.time()

end_time100 - start_time100
end_time10k - start_time10k

Running on my machine gives:

> end_time100 - start_time100
Time difference of 0.2343311 secs
> end_time10k - start_time10k
Time difference of 9.813453 secs

[edzer]

This is about replacing blocks of rows in large data.frame's; although this might come up when working with sf objects, I don't think it has to do much with sf.

[cseveren]

Agreed. Note that the list approach speeds up results by a factor of 100 in my example (link below), it could be worth making a note somewhere in the documentation that preallocating empty features as here won't necessarily speed anything. Thanks for your work on this package

https://gis.stackexchange.com/questions/367859/efficient-preallocation-of-simple-features-in-r-sf-r