Overall slow performance of feature calculation #701
Comments
|
The issue probably is that FT have to check time index column against cutoff_date for every row over which you aggregate. Moreover for custom features you are doing these expensive calculations independently for each custom feature you calculate. When you use your tool you probably "manually" drop the future information and info outside of training window and then do the aggregations, which is obviously much faster. Where it comes to check for future information you can theoretically drop it manually before running the calculation and somehow instruct FT not to check if cutoff_date is prior to the time of the event. As far as I know FT do not offer such option (maybe option approximate in dfs function can be used for this?). The training window check is bit more tricky, very inelegant solution would be to drop info outside of training window and then do the calculation independently for all time windows you want to consider. This could be solved by adding the following features to FT (which I believe would be very helpful in general):
|
|
@johnnyheineken I realize this issue is old, but are you able to run this on the latest version of featuretools? If so, we'd be curious to see how it runs now. Based on that we can come up with some actions that we can take if things aren't fixed. I'm going to close this issue but feel free to open a new one if you have a chance to run! Thanks |
Hi,
I've been testing feature tools extensively. I do not have any specific bug report, but I still would like to report the performance issues, which we encountered with FeatureTools.
Our typical usecase looks like this. We usually just aggregate data from transactional level to application level.
Entity set + transaction entity
and
In our specific usecase, we usually do following features:
a) simple aggregations
b) simple aggregations over time windows
c) ratio of aggregations - different aggregations
d) ratio of aggregations - different time windows
e) time since first/last event
All of these are sometimes segmented per another features.
Out of these features, default FeatureTools are good for use case a) and e).
If we consider 'TREND" primitive, we can also calculate d).
If we write custom primitives, use case b) and c) is also managable.
However, this does not solve the problem that the calculation is slow, and this is also amplified when we use the custom primitives (which are fyi using numpy, not pandas).
These are the features to be calculated (~220 features):
Calculation of these features takes 2 hours, 30 minutes.
If we calculate similar features using pandas (I wrote a specialised tool for this use case), calculation of the same features on a single core takes 10 seconds + more of the ratio features (450 features total).
However, even if we omit the custom features, estimated calculation time is still in hours (97 features)
This is not a single usecase with this issue. We encountered this problem on several occasions throughout the company (different data, different setups, different features).
It is true that FeatureTools can be parallelised. But they become very resource hungry and still, even 100 cores will not decrease the computation time to seconds.
Feel free to contact me to specify this problem any further.
P.S. I am using FT 0.9.1, Python 3.7.4, pandas 0.25rc, numpy 1.17
The text was updated successfully, but these errors were encountered: