python
import featuretools as ft import pandas as pd
es = ft.demo.load_mock_customer(return_entityset=True)
Featuretools groups features into four general types:
Identity features <feature-types.identity>Transform <feature-types.transform>andCumulative Transform features <feature-types.cumulative_transform>Aggregation features <feature-types.aggregation>Direct features <feature-types.direct>
In Featuretools, each feature is defined as a combination of other features. At the lowest level are IdentityFeature <.feature_base.IdentityFeature> features which are equal to the value of a single variable.
Most of the time, identity features will be defined transparently for you, such as in the transform feature example below. They may also be defined explicitly:
python
time_feature = ft.Feature(es["transactions"]["transaction_time"]) time_feature
Direct features are used to "inherit" feature values from a parent to a child entity. Suppose each event is associated with a single instance of the entity products. This entity has metadata about different products, such as brand, price, etc. We can pull the brand of the product into a feature of the event entity by including the event entity as an argument to Feature. In this case, Feature is an alias for primitives.DirectFeature:
python
brand = ft.Feature(es["products"]["brand"], entity=es["transactions"]) brand
Transform features take one or more features on an .Entity and create a single new feature for that same entity. For example, we may want to take a fine-grained "timestamp" feature and convert it into the hour of the day in which it occurred.
python
from featuretools.primitives import Hour ft.Feature(time_feature, primitive=Hour)
Using algebraic and boolean operations, transform features can combine other features into arbitrary expressions. For example, to determine if a given event event happened in the afternoon, we can write:
python
hour_feature = ft.Feature(time_feature, primitive=Hour) after_twelve = hour_feature > 12 after_twelve at_twelve = hour_feature == 12 before_five = hour_feature <= 17 is_afternoon = after_twelve & before_five is_afternoon
Aggregation features are used to create features for a parent entity by summarizing data from a child entity. For example, we can create a Count <.primitives.Count> feature which counts the total number of events for each customer:
python
from featuretools.primitives import Count total_events = ft.Feature(es["transactions"]["transaction_id"], parent_entity=es["customers"], primitive=Count) fm = ft.calculate_feature_matrix([total_events], es) fm.head()
Note
For users who have written aggregations in SQL, this concept will be familiar. One key difference in featuretools is that GROUP BY and JOIN are implicit. Since the parent and child entities are specified, featuretools can infer how to group the child entity and then join the resulting aggregation back to the parent entity.
Often times, we only want to aggregate using a certain amount of previous data. For example, we might only want to count events from the past 30 days. In this case, we can provide the use_previous parameter:
python
- total_events_last_30_days = ft.Feature(es["transactions"]["transaction_id"],
parent_entity=es["customers"], use_previous="30 days", primitive=Count)
fm = ft.calculate_feature_matrix([total_events_last_30_days], es) fm.head()
Unlike with cumulative transform features, the use_previous parameter here is evaluated relative to instances of the parent entity, not the child entity. The above feature translates roughly to the following: "For each customer, count the events which occurred in the 30 days preceding the customer's timestamp."
Find the list of the supported aggregation features here <api_ref.aggregation_features>.
When defining aggregation or cumulative transform features, we can provide a where parameter to filter the instances we are aggregating over. Using the is_afternoon feature from earlier <feature-types.transform>, we can count the total number of events which occurred in the afternoon:
python
- afternoon_events = ft.Feature(es["transactions"]["transaction_id"],
parent_entity=es["customers"], where=is_afternoon, primitive=Count).rename("afternoon_events")
fm = ft.calculate_feature_matrix([afternoon_events], es) fm.head()
The where argument can be any previously-defined boolean feature. Only instances for which the where feature is True are included in the final calculation.
Composing multiple feature types is an extremely powerful abstraction that Featuretools makes simple. For instance, we can aggregate direct features on a child entity from a different parent entity. For example, to calculate the most common brand a customer interacted with:
python
from featuretools.primitives import Mode brand = ft.Feature(es["products"]["brand"], entity=es["transactions"]) favorite_brand = ft.Feature(brand, parent_entity=es["customers"], primitive=Mode) fm = ft.calculate_feature_matrix([favorite_brand], es) fm.head()
Because we can check if two features are equal (or a feature is equal to a value), we override Python's equals (==) operator. This means to check if two feature objects are equal (instead of their computed values in the feature matrix), we need to compare their hashes:
python
hour_feature.hash() == hour_feature.hash() hour_feature.hash() != hour_feature.hash()
dictionaries and sets use equality underneath, so those keys need to be hashes as well
python
myset = set() myset.add(hour_feature.hash()) hour_feature.hash() in myset mydict = dict() mydict[hour_feature.hash()] = hour_feature hour_feature.hash() in mydict