Skip to content

Dataset documentation and questions

Jump to bottom Edit New page
CourtneyEpstein edited this page Nov 3, 2015 · 38 revisions

CMS Open Payments

Elhadad

Data Source

The goal was to predict when an article is newsworthy. To do that they built two datasets:

  1. Articles from a diverse range of biomedical and health journal and kept the articles that were considered newsworthy because they were covered by Reuters. (1,431 rows)
  2. Articles from JAMA journals and kept the one that were they were given press release by the journal editors. (1,007 rows)

Columns for *_article_info.csv:

  • PubMed ID,
  • citation title
  • journal,
  • authors,
  • affiliation (or missing),
  • abstract,
  • MeSH terms: Medical Subject Headings (MeSH) is a comprehensive controlled vocabulary for the purpose of indexing journal articles and books in the life sciences; it serves as a thesaurus that facilitates searching.

To add columns to the dataframe create by pd.read CSV do the following: reuters.columns=['PubMed_ID', 'citation_title', 'journal', 'authors', 'affiliation', 'abstract', 'MeSH terms']

There are 2 other datasets that they call negative instances. These contain articles having some similarities but not considered newsworthy. The two datasets are :

  1. Negative instances linked to the article from Reuters. (28,359 rows)
  2. Negative instances linked to articles from JAMA. (10,026 rows)

These datasets are the datasets jama_article_info.csv and all_reuters_artcile_info.csv.

Columns for matched_articles_.csv:

  • PubMed ID,[ This is the pmid of the corresponding article for which the row was 'matched']
  • PubMed ID,
  • citation title
  • journal,
  • authors,
  • affiliation (or missing),
  • abstract,
  • MeSH terms

There are 2 other datasets that they call negative instances. These contain articles having some similarities but not considered newsworthy. The two datasets are :

  1. Negative instances linked to the article from Reuters. (28,359 rows)
  2. Negative instances linked to articles from JAMA. (10,026 rows)

These are the datasets all_reuters_matched_articles_filtered.csv and jama_pmids.txt_matched_articles_filtered.csv. Negative instances contain X articles for each positive article (1) or (2) published in the same journal in the same year that did not receive coverage in the Reuters corpus or no press release was issued for JAMA. They also used several filtering heuristics to do this « matched sampling ». Not more explanations on how they matched this negative instances.

The columns are the same as (1) and (2) except that the first column corresponds to the PubMed ID from the newsworthy corresponding article.

What they did in the paper

They built citation features: journal name, institution of first author, extract words (uni and bi-grams) from titles, abstracts and MeSH terms. At the end they had 14,614 features. Then they used a L2 regularized regression to predict the newsworthiness.

Idea:

You need to find important words from the abstract and the MeSH terms to have features to build your prediction model. Look at what are the diseases in the top (50-100) features in the prediction. Try to link these diseases that have a good coverage in the media to the healthcare cost. The articles are from 09.2012-now? so maybe look at the results from different time periods.

Chronic Conditions

Each cell is a record defined by all combinations of age category, gender, various chronic conditions, and dual-eligibility status (eligible for both Medicare and Medicaid) of the beneficiaries.

2010_Chronic_Conditions_PUF.csv 3.6 MB, 22003 rows, 55 columns - see data dictionary

COLUMNS

###Profile features:

  • BENE_SEX_IDENT_CD: 1 or 2
  • BENE_AGE_CAT_CD: 6 groups

###Conditions:

  • CC_ALZHDMTA

  • CC_CANCER

  • CC_CHF

  • CC_CHRNKIDN

  • CC_COPD

  • CC_DEPRESSN

  • CC_DIABETES

  • CC_ISCHMCHT

  • CC_OSTEOPRS

  • CC_RA_OA

  • CC_STRKETIA

  • CC_2_OR_MORE

  • DUAL_STUS

###Claims variables (in the form of counts and averages): ####Part A: less than 12 months

  • BENE_COUNT_PA_LT_12
  • AVE_MO_EN_PA_LT_12
  • AVE_PA_PAY_PA_LT_12
  • AVE_IP_PAY_PA_LT_12
  • AVE_SNF_PAY_PA_LT_12
  • AVE_OTH_PAY_PA_LT_12
  • AVE_IP_ADM_PA_LT_12
  • AVE_SNF_DAYS_PA_LT_12

####Part A: equal to 12 months

  • BENE_COUNT_PA_EQ_12
  • AVE_PA_PAY_PA_EQ_12
  • AVE_IP_PAY_PA_EQ_12
  • AVE_SNF_PAY_PA_EQ_12
  • AVE_OTH_PAY_PA_EQ_12
  • AVE_IP_ADM_PA_EQ_12
  • AVE_SNF_DAYS_PA_EQ_12 ####Part B: less than 12 months
  • BENE_COUNT_PB_LT_12
  • AVE_MO_EN_PB_LT_12
  • AVE_PB_PAY_PB_LT_12
  • AVE_CA_PAY_PB_LT_12
  • AVE_OP_PAY_PB_LT_12
  • AVE_OTH_PAY_PB_LT_12
  • AVE_CA_VST_PB_LT_12
  • AVE_OP_VST_PB_LT_12 ####Part B: equal to 12 months
  • BENE_COUNT_PB_EQ_12
  • AVE_PB_PAY_PB_EQ_12
  • AVE_CA_PAY_PB_EQ_12
  • AVE_OP_PAY_PB_EQ_12
  • AVE_OTH_PAY_PB_EQ_12
  • AVE_CA_VST_PB_EQ_12
  • AVE_OP_VST_PB_EQ_12 ####Part C:
  • BENE_COUNT_PC_LT_12
  • AVE_MO_EN_PC_LT_12
  • BENE_COUNT_PC_EQ_12 ####Part D:
  • BENE_COUNT_PD_LT_12
  • AVE_MO_EN_PD_LT_12
  • AVE_PDE_CST_PD_LT_12
  • AVE_PDE_PD_LT_12
  • BENE_COUNT_PD_EQ_12
  • AVE_PDE_CST_PD_EQ_12
  • AVE_PDE_PD_EQ_12

Here is a useful read about how the data can be used.

####About Medicare Part A, B, C, D Different parts of Medicare cover different services. Original Medicare, made up of Parts A and B, is administered by the federal government:

  1. Part A (Hospital Insurance) covers most medically necessary hospital, skilled nursing facility, home health and hospice care. No monthly premium if you’ve worked more than 10 years.
  2. Part B (Medical Insurance) covers most medically necessary doctors’ services, preventive care, durable medical equipment, hospital outpatient services, laboratory tests, x-rays, mental health care, and some home health and ambulance services. You pay a monthly premium for this coverage.
  3. Part C is the part of Medicare policy that allows private health insurance companies to provide Medicare benefits.
  4. Part D provides prescription drug coverage and is provided only through private insurance.

###Questions for modeling:

  1. What predicts average payment?
  2. Which are the most costly chronic conditions, in terms of total dollar amount (avg payment * bene count) and aggregated by chronic condition?
  3. Are there any conditions and age group combinations that are expensive to treat?
  4. How do Part A and B differ?
  5. How do Part A and B interact?
  6. How much did payments increase from 2008 to 2010?

Notes: payments distribution is very skewed (90% of cost comes from 1% of patients). Consider taking logs.

NPDB Malpractice Claims

Medicare Provider Utilization and Payment Data

SPARCS

Add a custom footer
Add a custom sidebar

Clone this wiki locally