Unknown variables error: Unemployment Rates

[johnziebro]

Variables listed below can be looked up with variable_info(), but not downloaded. Other required variables can be downloaded. Data.census.gov will also show all variables as valid on the website.
Similar issue: #30 was resolved by updating source Census Bureau data due to changes they've made.

Having trouble accessing:
'S2301_C02_008E': 'Estimate!!Labor Force Participation Rate!!Population 16 years and over!!AGE!!55 to 59 years'
'S2301_C02_001E': 'Estimate!!Labor Force Participation Rate!!Population 16 years and over'

import censusdata as cd

# define census variable
variables = ['S2301_C02_008E']

# get variable info
display( cd.variable_info.censusvar('acs1', 2019, variables) )

# define geo index
geo = cd.censusgeo([('state', '06')])

# download data
cd.download('acs1', 2019, geo, variables)

'S2303_C02_008E: Estimate!!Percent!!Population 16 to 64 years!!WEEKS WORKED!!Did not work'
'S2301_C02_001E: Estimate!!Labor Force Participation Rate!!Population 16 years and over'
ValueError: Unexpected response (URL: https://api.census.gov/data/2019/acs/acs1?get=NAME,S2301_C02_008E&for=state:06): error: error: unknown variable 'S2301_C02_008E' 

As well as:
'DP03_0009PE: Percent!!EMPLOYMENT STATUS!!Civilian labor force!!Unemployment Rate'

import censusdata as cd

# define census variable
variables = ['DP03_0009PE']

# get variable info
display( cd.variable_info.censusvar('acs5', 2019, variables) )

# define geo index
geo = cd.censusgeo([('state', '06')])

# download data
cd.download('acs5', 2019, geo, variables)

{'DP03_0009PE': ['SELECTED ECONOMIC CHARACTERISTICS',
  'Percent!!EMPLOYMENT STATUS!!Civilian labor force!!Unemployment Rate',
  'float']}
ValueError: Unexpected response (URL: https://api.census.gov/data/2019/acs/acs5?get=NAME,DP03_0009PE&for=state:06): error: error: unknown variable 'DP03_0009PE' 

[johnziebro]

As described in issue 30's fix, the test code does not work for table DP03:

import censusdata as cd
tableInfo = cd.censustable("acs5", 2019, "DP03")
fullTable = cd.download("acs5", 2019, cd.censusgeo([('state', '51'), ('county', '059')]),
list(tableInfo.keys()), tabletype='subject')

ValueError: Unexpected response (URL: https://api.census.gov/data/2019/acs/acs5/subject?get=NAME,DP03_0001E,DP03_0001PE,DP03_0002E,DP03_0002PE,DP03_0003E,DP03_0003PE,DP03_0004E,DP03_0004PE,DP03_0005E,DP03_0005PE,DP03_0006E,DP03_0006PE,DP03_0007E,DP03_0007PE,DP03_0008E,DP03_0008PE,DP03_0009E,DP03_0009PE,DP03_0010E,DP03_0010PE,DP03_0011E,DP03_0011PE,DP03_0012E,DP03_0012PE,DP03_0013E,DP03_0013PE,DP03_0014E,DP03_0014PE,DP03_0015E,DP03_0015PE,DP03_0016E,DP03_0016PE,DP03_0017E,DP03_0017PE,DP03_0018E,DP03_0018PE,DP03_0019E,DP03_0019PE,DP03_0020E,DP03_0020PE,DP03_0021E,DP03_0021PE,DP03_0022E,DP03_0022PE,DP03_0023E,DP03_0023PE,DP03_0024E,DP03_0024PE,DP03_0025E&for=county:059&in=state:51): error: error: unknown variable 'DP03_0001E'

As a workaround the test code will work for table S2301. S2301_C02_008E and S2301_C02_001E can be extracted in this manner, albeit with much increased bandwidth usage due to downloading a complete table for each required administration boundary.

tableInfo = cd.censustable("acs5", 2019, "S2301")
fullTable = cd.download("acs5", 2019, cd.censusgeo([('state', '51'), ('county', '059')]),
list(tableInfo.keys()), tabletype='subject')
fullTable.T[fullTable.T.index.str.contains('_C02_008E')]

	Fairfax County, Virginia: Summary level: 050, state:51> county:059
S2301_C02_008E	82.3

[johnziebro]

As a side note, what I find very strange about this issue is that the Census API responds with 'S2301_C02_008E' variable does not exist, but if the full table S2301 is downloaded directly from the Census Bureau, it does exist in the table.

[jtleider]

Hi,

The errors are because tabletype was not properly specified (see documentation). For subject variables (those starting in S), tabletype='subject' needs to be passed as an argument for download. Similarly, for data profile variables (those starting in DP), tabletype='profile' needs to be specified. By specifying those arguments I was able to fix both errors you mentioned. This is not related to whether the whole table is downloaded or only a part.

If there are any errors this does not address please let me know.

[johnziebro]

That explains it, thank you for the correction and clarification! Somehow I missed this in the docs. Closing as resolved.

Relevant docs:
https://jtleider.github.io/censusdata/variables.html
https://jtleider.github.io/censusdata/api.html?highlight=tabletype