Add shuffle, shuffle! functions

[rana]

Hi,

Would be helpful to see shuffle, shuffle! functions in DataFrames. Used in randomizing machine learning mini batches.

What do you think?

[bkamins]

Now you can do shuffle via df[shuffle(axes(df, 1)), :] but I agree we could add it.

@nalimilan - given we have settled to treat a DataFrame as a collection of rows I think it is OK to add it. If you agree, then I can make a PR.

[rana]

Thanks, I didn't know about df[shuffle(axes(df, 1)), :]. I will start using that in the mean time.

[bkamins]

A bit less efficient (but more aesthetic) way to do it is DataFrame(shuffle(eachrow(df))).

[rana]

Maybe also consider offering column shuffling?

shuffle(;cols=false)

shuffle!(;cols=false)

[bkamins]

We treat DataFrame as row oriented, so I would not implement column shuffling directly, rather this:

select(df, randperm(ncol(df)))

or this:

df[:, randperm(ncol(df))]

should be used

[nalimilan]

Reminds me of a similar discussion about sample. Maybe better leave this for post-1.0.

Shuffling columns doesn't sound too common, is it?

[bkamins]

Also another pattern that can be used to shuffle columns is df[randperm(nrow(df)), :].

An in-place operation is more challenging and will require a careful design.

OK - leaving this decision post 1.0 (mostly because it is easy to do this without this function).

[rana]

I haven't seen many column permutation examples, though I use it in my work. Appreciate the pointer on how to do it. When I'm deep in a language it is obvious. In this case I'm in multiple languages and frameworks and looking for convenience functions.

[bkamins]

Sure. I guess the point of @nalimilan is that we want to move towards 1.0 pretty soon.

In general - as we try to look at DataFrame as a collection of rows now I would be OK with adding shuffle and sample to it now. But @nalimilan is a kind of "ecosystem curator" (as it has to be consistent) so I prefer to delegate the final word to him 😄.