FEA Fuzzy joining on datetime

[jovan-stojanovic]

Adds support for fuzzy joining tables on datetime columns.

The datetime columns in the table must be recognizable with pandas.DataFrame.select_dtypes('datetime').

[jovan-stojanovic]

I applied a StandardScaler on datetime values to avoid issue #547.

I also added an 'auto' option for the numerical_match parameter: in this case, we will use the column types to identify how to encode it.

What remains is to differentiate this numerical_match='auto' approach from the 'numbers' and 'time', where I am unsure what to do. I guess the idea would be to force certain columns to either have the appropriate type (int, float or datetime) suitable for encoding or raise an error?
Or to just enforce a check as if there is this type, numeric or datetime, and raise an error if it's missing, WDYT?

[Vincent-Maladiere]

Hi Jovan, here are some feedbacks on this PR :)

My main remark is that I don't understand the point of numerical_match anymore: we extract embeddings from numerical and times by using the StandardScaler, and from string by using TFIDF.

Therefore we use the Euclidean Distance in the NearestNeighbor in all cases.

If users want to encode numerical values as strings, shouldn't they transform their numerical values to string before running the fuzzy_join? Otherwise, I feel it's very error-prone.

Also, removing the numerical_match would simplify the logic a lot.

Am I missing something? WDYT?

[LilianBoulard]

Thanks for the PR!
I have nothing of significance to add, as I'm not super familiar with the workings of fuzzy_join 😅

[Vincent-Maladiere]

One last comment before it LGTM :)

[Vincent-Maladiere]

also "string" and "category", so include=["string", "category", "object"] ?

[jovan-stojanovic]

As the test passed, I though object was sufficient. But if we want to be sure, I'll add all

[Vincent-Maladiere]

LGTM, waiting for the CI to be green :)