DOC Update IncrementalPCA example to actually use batches

[dsblank]

What does this implement/fix? Explain your changes.

It wasn't clear to be how to actually use the API to train an IncrementalPCA. This updates the example, and gives the same results as before.

[jeremiedbb]

Hi @dsblank, I'm not sure to understand what you mean by "to actually use batches". Internally fit does use batches. This is controlled by the batch_size parameter. It's equivalent to calling partial_fit in a row, which is what fit does actually. Calling partial_fit manually is for advanced use cases like out of core, or when you don't have all the data from the start (as described here https://scikit-learn.org/stable/computing/scaling_strategies.html#incremental-learning), so I would keep this example as is.

[dsblank]

@jeremiedbb :

I'm not sure to understand what you mean by "to actually use batches". Internally fit does use batches.

Yes, but if you can use that interface, why not use the non-incremental version?

It seems the main use for IncrementalPCA is when you don't want to use the auto-batching API, and manually provide the batch yourself (by, say, loading batches from files).

Calling partial_fit manually is for advanced use cases like out of core, or when you don't have all the data from the start

I think it is worthwhile to have such an example. I can update the code to show all three variations if that would help.

[ogrisel]

I don't understand the choice of X_ipca as a variable name for the estimator instance. I think it should stay named ipca.

X_* variable names should be reserved for 2D arrays (original data or transformed data) in this example.

[jeremiedbb]

Yes, but if you can use that interface, why not use the non-incremental version?

When you have a large dataset (that still fits in memory), using the minibatch version can converge a lot faster. While you need many iterations over the full dataset, with the full batch version, to converge, you can often reach almost the same quality of results within only a few epochs with the minibatch version because you update the learned parameters more often and by smaller steps.

I think it is worthwhile to have such an example. I can update the code to show all three variations if that would help.

There's already an example that explains how to use the partial_fit API when all the data is not available at once https://scikit-learn.org/stable/auto_examples/applications/plot_out_of_core_classification.html

[ogrisel]

It would be both simple and more efficient to slice batches:

for idx in range(0, len(X), batch_size):
    X_batch = X[idx:idx + batch_size]
    ipca.partial_fit(X_batch)

...

[dsblank]

Yes, that makes sense!

[ogrisel]

There's already an example that explains how to use the partial_fit API when all the data is not available at once https://scikit-learn.org/stable/auto_examples/applications/plot_out_of_core_classification.html

I personally wouldn't mind showing that the IncrementalPCA model also supports the partial_fit API in the plot_incremental_pca.py example, maybe with a comment that explains that each batch could be loaded progressively from a disk-based storage instead of slicing an in-memory X matrix.

[jeremiedbb]

@dsblank are you still interested in finalizing this PR ?