Top-level API for time grouping

[jhamman]

This package has a growing number of methods that rely on a DateTimeIndex inside the estimators. I'm beginning to think this design decision is complicating the implementation of individual methods (see @dgergel's great and persistent attempts in #28 as an example) and that we should consider stepping back from the current approach. The use DateTimeIndexes inside the estimators is also the primary divergence from full scikit-learn integration. In this issue, I'll propose a new top-level API that supports using time indexes for grouped training/prediction outside individual regressors/transformers.

I'll start by outlining a few common time-grouping approaches:

• Group-by day-of-year or month - These are the simplest to implement using out-of-the-box functionality in both Xarray and Pandas
• Padded and/or windowed groups based on day-of-year or month - @jukent's Zscore method and @dgergel's NASA-NEX BCSD (Feature/implement daily bcsd #28) method use variations on padded day-of-year groupers. Additionally, the reference implementation of ARRM (see Add asynchronous regional regression model #42) also uses a +/- 15 day window around month boundaries.

My proposal is that we develop (or in some cases, continue to develop) a series of grouper classes that perform these (sometimes) esoteric grouping operations and that we utilize these grouper classes with an API object that supports training and prediction.

The grouper concept may look something like this:

index = ds.indexes['time']
# or
index = df.index

group_iter = PaddedDOYGrouper(index, window=15)

under the hood, these groupers would support iteration like this...

# pandas
for inds in group_iter:
    df_group = df.iloc[inds]

# xarray
for inds in group_iter:
    ds_group = ds.isel(time=inds)

We could then write a simple model API that combines the grouper object with a proper sklearn-compatible regressors/pipelines:

arrm_model = GroupedRegressor(estimator=PiecewiseLinearRegression, grouper=PaddedDOYGrouper)

arrm_model.fit(X_df, y_df)  # -> fits multiple PiecewiseLinearRegression for each group produced by the grouper
...

If done correctly, I think we can share the Groupers between Pandas and Xarray applications allowing us to use these either at the PointWiseDownscaler level or for individual points.

---

@jukent and @dgergel - I'm curious to hear from you on the potential feasibility of this approach for the methods you have developed here. Am I missing anything in that would keep us from executing on this sort of API?

[jukent]

I think this is a great idea and doable. Maybe we should set up a meeting at the end of next week?

[dgergel]

I agree with @jukent, I think this is a great idea and a good way forward for dealing with the DatetimeIndex issues that have been getting increasingly complicated with the newer grouper classes. Maybe we can aim to schedule a call next Thursday or Friday?

[dgergel]

@jhamman @jukent dropping this when2meet in for finding a time to discuss this either later this week or sometime next week.

[jhamman]

I'm out of commission this week but would be happy to chat about this next week. I filled out your poll. Thanks for setting it up.

[jukent]

I also filled out the poll. Thanks for putting that together.