Booster slicing doesn't take all valid increasing sequences

[david-cortes]

When trying to slice a booster, it will not accept all formats that python would otherwise accept, and it's not clear what exactly it considers acceptable ranges - for example, if the step is not equal to one, it requires that the end be step more than the last returned round instead of just +1.

Example:

import numpy as np, xgboost as xgb
rng = np.random.default_rng(seed=123)
y = rng.standard_normal(size=100)
X = rng.standard_normal(size=(100,10))
dm = xgb.DMatrix(X, y)
model = xgb.train(
    dtrain=dm,
    params={},
    num_boost_round=20
)

Fails:

model[0:4:3]

Succeeds:

model[0:6:3]

In contrast, python lists and numpy arrays would take both of those.

[david-cortes]

Additionally:

• If the end of the slice (which is not included in the output) goes past the number of rounds in the model, regardless of what's the last number that the sequence would involve, it will throw an out-of-bounds error.

Here it has 20 rounds, but cannot slice like this:

import numpy as np, xgboost as xgb
rng = np.random.default_rng(seed=123)
y = rng.standard_normal(size=100)
X = rng.standard_normal(size=(100,10))
dm = xgb.DMatrix(X, y)
model = xgb.train(
    dtrain=dm,
    params={},
    num_boost_round=20
)
model[0:21:3]

Note that this slice would stop at 18, which is within the range of the model's rounds:

np.arange(20)[0:21:3]

array([ 0,  3,  6,  9, 12, 15, 18])

• When the underlying C function XGBoosterSlice returns code -2 (index out of bound), it will not set a message in XGBGetLastError(). The case above has an error message, but that error is thrown from the python wrapper, not from C++.

[trivialfis]

The slicing is not as rich as Python or numpy. We at the time thought this might be a bit over-engineering for a booster class. It seems we should implement the full slicing just for less confusion. The easiest way to do it is just to convert a booster into a list of boosters and then let Python do the rest of the indexing. However, the conversion can be expensive.

[david-cortes]

The slicing is not as rich as Python or numpy. We at the time thought this might be a bit over-engineering for a booster class. It seems we should implement the full slicing just for less confusion. The easiest way to do it is just to convert a booster into a list of boosters and then let Python do the rest of the indexing. However, the conversion can be expensive.

I don't think that'd be necessary, since it's possible to e.g. create a sequence with np.arange, slice it using the user' input, and pass those integers as an array instead.

Or, since this functionality is only used in the early stopping callback, could also be possible to remove the 'step' part altogether.

[trivialfis]

It's used inside xgboost for early stopping, but outside of xgboost, it's also used for people to explore the individual trees' impact on the final output. There are some quirks in gradient boosting, like the first few trees are usually shallower than the latter trees, sometimes people customize their models for exploratory purposes, sometimes used for extracting a submodel then stacking with other models.