Error when merging extension in modeler CPA when running examples

[pajachiet]

• SDV version: master branch
• Python version: 3.6.5
• Operating System: macOS

Description

Running the following command, with airbnb_demo or biodegradability

 python examples/demo.py airbnb_demo

I get an error at the following line (traceback below). Indeed, neither extended_table or extension.reset_index() have the primarey key column.

Traceback (most recent call last):
  File "…/SDV/examples/demo.py", line 69, in <module>
    run_demo(sys.argv[1])
  File "…/SDV/examples/demo.py", line 53, in run_demo
    sdv.fit()
  File "…/SDV/sdv/sdv.py", line 45, in fit
    self.modeler.model_database()
  File "…/SDV/sdv/modeler.py", line 243, in model_database
    self.RCPA(table)
  File "…/SDV/sdv/modeler.py", line 237, in RCPA
    self.CPA(table)
  File "…/SDV/sdv/modeler.py", line 222, in CPA
    extended_table = extended_table.merge(extension, how='left', on=pk)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/frame.py", line 5370, in merge
    copy=copy, indicator=indicator, validate=validate)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/reshape/merge.py", line 57, in merge
    validate=validate)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/reshape/merge.py", line 565, in __init__
    self.join_names) = self._get_merge_keys()
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/reshape/merge.py", line 824, in _get_merge_keys
    right_keys.append(right[rk]._values)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/frame.py", line 2139, in __getitem__
    return self._getitem_column(key)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/frame.py", line 2146, in _getitem_column
    return self._get_item_cache(key)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/generic.py", line 1842, in _get_item_cache
    values = self._data.get(item)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/internals.py", line 3843, in get
    loc = self.items.get_loc(item)
  File "…/SDV/venv/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2527, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 117, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 139, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1265, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1273, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'id'

[ManuelAlvarezC]

Thanks for reporting @pajachiet.

Do you mind adding the commit sha? This repository is under active development and having the commit this issue is related to will make helping you easier.

[pajachiet]

Please find more information below

Step to reproduce the issue.

Maybe we could automate this with CI, I would know how to do it with Gitlab, but i don't know well Jenkins or Circle-CI to do it efficiently.

git clone https://github.com/HDI-Project/SDV.git
cd SDV
virtualenv venv
source venv/bin/activate
pip install -e .
python examples/demo.py airbnb_demo

$ git rev-parse HEAD
04b3a34bfd680a02fcc038721f2fff471c3db4b1

$ pip freeze        
boto3==1.9.87
botocore==1.12.87
copulas==0.2.1
cycler==0.10.0
docutils==0.14
exrex==0.10.5
Faker==1.0.2
jmespath==0.9.3
kiwisolver==1.0.1
matplotlib==3.0.2
numpy==1.16.1
pandas==0.24.1
pyparsing==2.3.1
python-dateutil==2.7.5
pytz==2018.9
rdt==0.1.2
s3transfer==0.1.13
scikit-learn==0.20.2
scipy==1.2.0
-e git+https://github.com/HDI-Project/SDV.git@04b3a34bfd680a02fcc038721f2fff471c3db4b1#egg=sdv
six==1.12.0
text-unidecode==1.2
urllib3==1.24.1