Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stack overflow when running DFS as part of larger project #267

Closed
ian-perry-ironnet opened this issue Sep 24, 2018 · 1 comment
Closed

Stack overflow when running DFS as part of larger project #267

ian-perry-ironnet opened this issue Sep 24, 2018 · 1 comment

Comments

@ian-perry-ironnet
Copy link

ian-perry-ironnet commented Sep 24, 2018 •
edited

I'm using featuretools as one step in a data ingestion pipeline to create extra features as part of a larger project. After doing some code refactoring and organizing, I started seeing this error for a few of my datasets while ft.dfs was running:

Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fffa95b0380 (most recent call first):
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/_weakrefset.py", line 75 in __contains__
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/abc.py", line 190 in __instancecheck__
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/dtypes/inference.py", line 283 in is_list_like
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 975 in putmask
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 431 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 453 in fillna
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 451 in f
  File "/Users/ian.perry/anaconda3/envs/cogsys/lib/python3.6/site-packages/pandas/core/internals.py", line 495 in split_and_operate
  ...
Abort trap: 6

Unfortunately because it's a stack-related issue, I can't reproduce the problem in a notebook and my data is sensitive. After a lot of debugging I managed to fix the problem by setting the python recursion limit much higher than its default value of 1000 (I found 5000) to work for all my datasets (sys.setrecursionlimit(5000)).

Just wanted to make you guys aware and possibly provide help for those that encounter the same problem. Internally to Featuretools and DFS I don't know what's blowing up the stack or if it's even a bug.
@rwedge
Copy link
Contributor

rwedge commented May 14, 2020

Closing until we can get a reproducible example

@rwedge rwedge closed this as completed May 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rwedge @ian-perry-ironnet