@@ -61,7 +61,7 @@ Potential benefits include:
61
61
* Less noise in predictions; better generalization
62
62
* More control to the user on what the model can fit. For example, the user may
63
63
want to exclude some interactions even if they perform well due to regulatory
64
- constraints
64
+ constraints.
65
65
66
66
****************
67
67
A Simple Example
@@ -127,11 +127,19 @@ first and second constraints (``[0, 1]``, ``[2, 3, 4]``).
127
127
Source(source, format='png').render('../_static/feature_interaction_illustration3', view=False)
128
128
Source(source, format='svg').render('../_static/feature_interaction_illustration3', view=False)
129
129
130
+ .. |fig1 | image :: ../_static/feature_interaction_illustration2.svg
131
+ :scale: 7%
132
+ :align: middle
130
133
131
- .. figure :: ../_static/feature_interaction_illustration2.svg
132
- :align: left
134
+ .. |fig2 | image :: ../_static/feature_interaction_illustration3.svg
135
+ :scale: 7%
136
+ :align: middle
133
137
134
- .. figure :: ../_static/feature_interaction_illustration3.svg
138
+ +-----------+---------+
139
+ | |fig1 | | |fig2 | |
140
+ +-----------+---------+
141
+ | forbidden | allowed |
142
+ +-----------+---------+
135
143
136
144
137
145
****************************************************
@@ -175,15 +183,15 @@ to set the ``tree_method`` parameter to one of the following: ``exact``, ``hist`
175
183
Advanced topic
176
184
**************
177
185
178
- The intuition behind interaction constraint is simple. User have prior knowledge about
186
+ The intuition behind interaction constraints is simple. Users may have prior knowledge about
179
187
relations between different features, and encode it as constraints during model
180
188
construction. But there are also some subtleties around specifying constraints. Take
181
- constraint ``[[1, 2], [2, 3, 4]] `` as an example, the second feature appears in two
182
- different interaction sets ``[1, 2] `` and ``[2, 3, 4] ``, so the union set of features
183
- allowed to interact with ``2 `` is ``{1, 3, 4} ``. In following diagram, root splits at
184
- feature ``2 ``. because all its descendants should be able to interact with it, so at the
185
- second layer all 4 features are legitimate split candidates for further splitting,
186
- disregarding specified constraint sets.
189
+ the constraint ``[[1, 2], [2, 3, 4]] `` as an example. The second feature appears in two
190
+ different interaction sets, ``[1, 2] `` and ``[2, 3, 4] ``. So the union set of features
191
+ allowed to interact with ``2 `` is ``{1, 3, 4} ``. In the following diagram, the root splits at
192
+ feature ``2 ``. Because all its descendants should be able to interact with it, all 4 features
193
+ are legitimate split candidates at the second layer. At first sight, this might look like
194
+ disregarding the specified constraint sets, but it is not .
187
195
188
196
.. plot ::
189
197
:nofigs:
@@ -221,17 +229,17 @@ disregarding specified constraint sets.
221
229
This has lead to some interesting implications of feature interaction constraints. Take
222
230
``[[0, 1], [0, 1, 2], [1, 2]] `` as another example. Assuming we have only 3 available
223
231
features in our training datasets for presentation purpose, careful readers might have
224
- found out that the above constraint is same with ``[0, 1, 2] ``. Since no matter which
225
- feature is chosen for split in root node, all its descendants have to include every
226
- feature as legitimate split candidates to avoid violating interaction constraints.
232
+ found out that the above constraint is the same as simply ``[[ 0, 1, 2] ] ``. Since no matter which
233
+ feature is chosen for split in the root node, all its descendants are allowd to include every
234
+ feature as legitimate split candidates without violating interaction constraints.
227
235
228
236
For one last example, we use ``[[0, 1], [1, 3, 4]] `` and choose feature ``0 `` as split for
229
- root node. At the second layer of built tree, ``1 `` is the only legitimate split
237
+ the root node. At the second layer of the built tree, ``1 `` is the only legitimate split
230
238
candidate except for ``0 `` itself, since they belong to the same constraint set.
231
- Following the grow path of our example tree below, the node at second layer splits at
239
+ Following the grow path of our example tree below, the node at the second layer splits at
232
240
feature ``1 ``. But due to the fact that ``1 `` also belongs to second constraint set ``[1,
233
- 3, 4] ``, at third layer, we need to include all features as candidates to comply with its
234
- ascendants.
241
+ 3, 4] ``, at the third layer, we are allowed to include all features as split candidates and
242
+ still comply with the interaction constraints of its ascendants.
235
243
236
244
.. plot ::
237
245
:nofigs:
0 commit comments