case sensitive merge

[shantanuo]

Problem description

By default Pandas merge method is case-sensitive.
There should be a way to merge 2 dataframes without considering upper/ lower case just like SQL joins.

Expected Output

May be a parameter case=True (default) that can be changed to case=False

[toobaz]

I don't think we want to get into this, for several reasons: in pandas (differently from SQL), column names are not necessarily strings; when possible, we want indexing to work the same everywhere (so we would need to support case=False in .loc, concat too...); I guess we don't want to overload the API with ad hoc parameters that are easy to emulate (df.rename({l : l.lower() for l in df}, axis=1).

[TomAugspurger]

Was the proposal for case-insensitive merges on the column names, or on the values? Could you show an example @shantanuo could you clarify? If it's on the values, does calling str.lower() on both frames first suite your needs?

[shantanuo]

Was the proposal for case-insensitive merges on the column names, or on the values?

on the values.

does calling str.lower() on both frames first suite your needs?

It does. But I thought having a parameter for this would be elegant.

[toobaz]

on the values.

I might have misunderstood. Could you show an example?

[shantanuo]

https://stackoverflow.com/questions/45519132/merge-method-compared-with-sql-query

I guess case is not important when we compare 2 columns for merging. There should be an option (built-in) to disable case-sensitivity.

[TomAugspurger]

It does. But I thought having a parameter for this would be elegant.

I'd prefer to keep things from getting too complex right now. I would like to see a small library that implements this, along with fuzzy joins, similar to http://varianceexplained.org/fuzzyjoin/

[toobaz]

https://stackoverflow.com/questions/45519132/merge-method-compared-with-sql-query

Right, the workaround suggested there is also simpler than mine

[shantanuo]

I guess it should be mentioned in the documentation somewhere so that developers time can be saved.

[TomAugspurger]

Sure. Added a couple sentences to doc/source/merging.rst explaining that would be welcome. Do you have time to make a pull request?

…

On Fri, May 4, 2018 at 5:51 AM, Shantanu Oak ***@***.***> wrote: I guess it should be mentioned in the documentation somewhere so that developers time can be saved. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20896 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIhUozkNRxeZRGzVSeBAFdicDSJLCks5tvDK7gaJpZM4Ttysy> .

[Ekillz]

Comment on the official pandas.merge page would be even better.

[TomAugspurger]

Sure, and we're able to link between the two pages. Can you make a PR?

…

On Thu, May 31, 2018 at 3:28 AM, Em ***@***.***> wrote: Comment on the official pandas.merge page would be even better. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20896 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIukTefSsg8o8FOSn1_C26uT3QmkQks5t38XOgaJpZM4Ttysy> .

[SaneshmonSabu]

What if I want to retain former cases after merging? str.lower would require at least one side to be in lower case.

What I am trying to achieve - merge "Pandas" with "PANDAS"