Error running the "Real-world example" #34
Assignees
Labels
Comments
|
Extra info:
INFO [2019-11-28 08:38:52] found FAM2013ER.rda already downloaded
INFO [2019-11-28 08:38:52] found FAM2015ER.rda already downloaded
INFO [2019-11-28 08:38:52] everything already downloaded. Build dataset now
INFO [2019-11-28 08:38:52] psidR: Loading Family data from .rda files
INFO [2019-11-28 08:38:56] psidR: loaded individual file: /media/julien/TOSHIBA EXT/DATASETS/PSID_data/IND2015ER.rda
INFO [2019-11-28 08:38:56] psidR: total memory load in MB: 1400
INFO [2019-11-28 08:38:56] psidR: currently working on data for year 2013
INFO [2019-11-28 08:38:56] full 2013 sample has 80666 obs
INFO [2019-11-28 08:38:56] you selected 33940 obs belonging to SRC
INFO [2019-11-28 08:38:56] dropping non-heads leaves 5450 obs
INFO [2019-11-28 08:38:57] psidR: currently working on data for year 2015
INFO [2019-11-28 08:38:57] full 2015 sample has 80666 obs
INFO [2019-11-28 08:38:57] you selected 33940 obs belonging to SRC
INFO [2019-11-28 08:38:57] dropping non-heads leaves 5318 obs
INFO [2019-11-28 08:38:58] End of build.panel
> head(d)
faminc hours hvalue mortgage own state interview ID1968 pernum sequence relation.head age educ
1: 10950 40 0 0 5 4 1 860 1 1 10 72 14
2: 40942 0 148000 0 1 41 2 459 1 1 10 79 12
3: 52300 0 90000 35000 1 9 3 581 3 1 10 62 10
4: 26400 2096 90000 0 1 1 4 1438 187 1 10 34 12
5: 8520 0 0 0 8 42 5 1034 3 1 10 62 12
6: 43050 1520 0 0 5 48 6 691 33 1 10 23 14
empstat weight pid year
1: 4 54.070 860001 2013
2: 4 52.431 459001 2013
3: 4 86.861 581003 2013
4: 1 0.000 1438187 2013
5: 5 12.756 1034003 2013
6: 1 25.676 691033 2013 |
|
Another follow-up:
Edit: |
|
it's hard to debug without downloading the whole slew of data again (doing that now). what's the earliest year where it works (i.e. after 1969)? the problem is each time they update the data.table API, some functionality in here breaks... |
|
my guess is that the problem is the wealth supplement. this has changed in the PSID, so currently wrong here . the wealth variables have been moved to the family files from 1999 onwards, so you should just select those in your fam.vars data.frame. try setting
|
|
ping @SchroederAdrian |
I will try to do that.
I am on it. I will restrict the sample until it works. |
|
FYI #36 |
thanks! actually it's almost done (fast internet today!) will check later today. |
# Clear the workplace:
rm(list = ls())
#install.packages('devtools')
#require(devtools)
#install_github(repo = "https://github.com/floswald/psidR")
#------
# Paths
#------
# Path to main file:
# !Adjut this to you own setting!
path_to_main = "/home/julien/Documents/REPOSITORIES/PSIDPanelBuilder"
# Where the data is stored:
#data_dir = "/home/julien/MEGA/Dataset/PSID"
data_dir = "/media/julien/TOSHIBA EXT/DATASETS/PSID_data"
#--------
# Options
#--------
# Set to true to add wealth data:
wealth = FALSE
# Set to true to work with the small dataset
small = FALSE
first_year = 1970
# ipak function: install and load multiple R packages.
# check to see if packages are installed. Install them if they are not, then load them into the R session.
# source: https://gist.github.com/stevenworthington/3178163
ipak <- function(pkg){
new.pkg <- pkg[!(pkg %in% installed.packages()[, "Package"])]
if (length(new.pkg))
install.packages(new.pkg, dependencies = TRUE)
sapply(pkg, require, character.only = TRUE)
}
# Load the required packages
# If you don't have them already installed, it may take a while
packages <- c("psidR")
ipak(packages)
require(psidR)
require(rjson)
require(data.table)
# Let's read PSID data from json file
login <- fromJSON(file = paste0(path_to_main, "/login_psid.json"))
# ################################################
# Real-world example: not run because takes long.
# Build panel with income, wage, age and education
# optionally: add wealth supplements!
# ################################################
# The package is installed with a list of variables
# Alternatively, search for names with \\code{\\link{getNamesPSID}}
# This is the body of function build.psid()
# (so why not call build.psid() and see what happens!)
r = system.file(package="psidR")
if (small){
f = fread(file.path(r,"psid-lists","famvars-small.txt"))
i = fread(file.path(r,"psid-lists","indvars-small.txt"))
if (wealth){
w = fread(file.path(r,"psid-lists","wealthvars-small.txt"))
}
} else {
f = fread(file.path(r,"psid-lists","famvars.txt"))
i = fread(file.path(r,"psid-lists","indvars.txt"))
if (wealth){
w = fread(file.path(r,"psid-lists","wealthvars.txt"))
}
}
# Selected years only
i <- i[which(i$year>=first_year), ]
f <- f[which(f$year>=first_year), ]
setkey(i,"name")
setkey(f,"name")
if (wealth) setkey(w,"name")
i = dcast(i[,list(year,name,variable)],year~name)
f = dcast(f[,list(year,name,variable)],year~name)
if (wealth) {
w = dcast(w[,list(year,name,variable)],year~name)
d = build.panel(datadir=data_dir,fam.vars=f,
ind.vars=i,wealth.vars=w,
heads.only =TRUE,sample="SRC",
design="all")
save(d, file= paste0(data_dir, "/psid.RData"))
} else {
d = build.panel(datadir=data_dir,fam.vars=f,
ind.vars=i,
heads.only =TRUE,sample="SRC",
design="all")
save(d, file= paste0(data_dir, "/psid_no_wealth.RData"))
}
head(d)
|
|
excellent, that means my lead in #36 is correct. so: build without wealth supplements for now (select the wealth vars from family files for 1999 onwards!) |
|
Thanks for being on this guys! I manually compiled the files using lodown and your psidR helper functions but definitely handy to have a second dataset to compare to now. Great package besides that! The PSID is only useful when its easily accessible and your functions saved me a lot of time already. |
|
the 1969 and wealth issues are separate bugs I think. there is no wealth supplement in 1969, so that cannot be the reason of the error. For completeness, here is full output |
Merged
|
well, took me only 3 months, but that should work now. @SchroederAdrian @JulienPascal |
Hi, I tried put the wealth variables in the famvars.txt documents for a 1999-2009 panel. However, it keeps warning me : "Error in |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
Thanks for great package. It's a big time-saver. I have tried to run the Real-world example from the documentation:
I encountered the following error message
Is there any easy fix for this problem?
This issue is related to #32
Thanks.
The text was updated successfully, but these errors were encountered: