You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thank you for the package! It is perfect for what I am trying to do.
I am using your revgeo package on about 15,000 records.
When running the package by itself:
city_per_rec <- revgeo(NRM$lon[i], NRM$lat[i], provider = 'photon', API = NULL, output = 'frame', item = 'city’)
I get through about 1000 records and it hangs up. I first thought it was photon throttling but later realized it was not. I now believe it is a RAM issue so I wrote a separate rscript:
##Library
library(rstudioapi)
library(revgeo)
##Load iteration
load('iter.RData’)
##City Reverse Geocoding
for (j in 1:a)
{
for (i in (b+1):(b+100))
{
if (!is.na(NRM$TOWN[i]) && !is.na(NRM$LONGITUDE))
{
city_per_rec <- revgeo(NRM$lon[i], NRM$lat[i], provider = 'photon', API = NULL, output = 'frame', item = 'city’)
NRM$ChCity[i] <- toupper(as.character(city_per_rec$city))
rm(city_per_rec)
}
}
b <- (b+100)
gc()
print('Taking a Break!’)
Sys.sleep(30)
gc() }
##Update iteration
iter <- iter+1 save(iter, file = 'iter.RData’)
##Save Workspace Image
save.image()
##Run This Script Again
if(iter < a) { restartSession(command = 'source("City.R")') }
This should clear the workspace, release the RAM, and restart R to run the code on 100 records at a time however this still hangs up my computer after about 1000 records.
I got through the data by manually doing chunks of 500 and restarting my computer so I am thinking it is a RAM fragmentation issue (running on only 6GB).
Are you able to provide any advise?
Thank you,
Ethan
The text was updated successfully, but these errors were encountered:
What operating system & version of R are you using? 1,000 records shouldn't be enough to fill more than a couple hundred MBs of ram at a time (if that), so my best guess is that something else is going on.
Thank you for the package! It is perfect for what I am trying to do.
I am using your revgeo package on about 15,000 records.
When running the package by itself:
city_per_rec <- revgeo(NRM$lon[i], NRM$lat[i], provider = 'photon', API = NULL, output = 'frame', item = 'city’)
I get through about 1000 records and it hangs up. I first thought it was photon throttling but later realized it was not. I now believe it is a RAM issue so I wrote a separate rscript:
##Library
library(rstudioapi)
library(revgeo)
##Load iteration
load('iter.RData’)
##City Reverse Geocoding
for (j in 1:a)
{
for (i in (b+1):(b+100))
{
if (!is.na(NRM$TOWN[i]) && !is.na(NRM$LONGITUDE))
{
city_per_rec <- revgeo(NRM$lon[i], NRM$lat[i], provider = 'photon', API = NULL, output = 'frame', item = 'city’)
NRM$ChCity[i] <- toupper(as.character(city_per_rec$city))
rm(city_per_rec)
}
}
b <- (b+100)
gc()
print('Taking a Break!’)
Sys.sleep(30)
gc() }
##Update iteration
iter <- iter+1 save(iter, file = 'iter.RData’)
##Save Workspace Image
save.image()
##Run This Script Again
if(iter < a) { restartSession(command = 'source("City.R")') }
This should clear the workspace, release the RAM, and restart R to run the code on 100 records at a time however this still hangs up my computer after about 1000 records.
I got through the data by manually doing chunks of 500 and restarting my computer so I am thinking it is a RAM fragmentation issue (running on only 6GB).
Are you able to provide any advise?
Thank you,
Ethan
The text was updated successfully, but these errors were encountered: