Tidyclust

[kbodwin]

This is my writeup with some details about clustering (unsupervised learning) and how I envision that fitting into the tidymodels framework.

[juliasilge]

This is super exciting to see! 🙌

[juliasilge]

I might take out "obviously" here because if a goal is for folks to be able to use these as preprocessing steps, then building this out using a recipes framework might be worth considering (rather than parsnip).

[kbodwin]

I edited the sentence, but I disagree with using the recipes framework.

Even if an unsupervised procedure informs a preprocessing, the application of the clustering method is not itself a preprocessing step. The data is not altered by the search procedure.

Besides which, I think things could get a little confusing when clustering is the end point in and of itself - would it appear as a recipe with no "baking" step? That feels strange to me.

[alexpghayes]

Just wanted to comment that for unsupervised methods there is often both a forward and a backward transformation. Less so with clustering, but for many PCA-like tools. I previously brought this up at little in tidymodels/recipes#264.

Another possible consideration before starting to prototype things is the difference between inductive/transductive models, or methods that can be applied to a new dataset versus those that cannot.

[michaelgaunt404]

Heyo! really excited to see this developing. I've recently fallen into the world of unsupervised clustering (via some gnar text projects) and have been having a hard time understanding the literature and not being able to find certain methods in tidymodels.

[brshallo]

Very cool! Looking forward to developments here and this functionality coming to tidymodels!

I posted a toy solution on SO for validating kmeans cluster partition stability on a holdout set: https://stackoverflow.com/a/68845111/9059865 . (For anyone stumbling onto this thread and looking for something simple in the interim before {celery} 😊 gets implemented in tidymodels.)

[topepo]

Overall it is great. My comments mostly reflect the idea that a different, but very similar, clas sof objects should be used to represent a clustering object.

[topepo]

I agree that a parsnip-like interface would be a great idea. I don't think that the same exact object structure and class are needed though. For example, the mode of "partition" is only required if yo fit it into an object of class model_spec.

I think that an alternate class name (cluster_spec?) would be good with some minor difference. For example, the distance component is one that doesn't fit with a model spec.

In other words, the api is good but using the entirety of the existing model specification format will be ungainly or kludgy.

[topepo]

Is bootstrapping used at all in cluster validation? Would something like this come into play with the potential for tuning mentioned below?

[topepo]

Perhaps we could change from the nomenclature of "performance metrics" to something like "partition characteristics", the former being more of a supervised learning thing.