BUG: Concat with inner join and empty DataFrame #15328

Closed
jesrael opened this Issue Feb 7, 2017 · 3 comments

Comments

Projects
None yet
3 participants

jesrael commented Feb 7, 2017 edited by jorisvandenbossche

Function concat with parameter join='inner' not return empty DataFrame:

DF_empty = pd.DataFrame()
DF_a = pd.DataFrame({'a': [1, 2]}, index=[0, 1])
print (DF_empty)
Empty DataFrame
Columns: []
Index: []

print (DF_a)
   a
0  1
1  2

print(pd.concat([DF_empty, DF_a], axis=1, join='inner'))
   a
0  1
1  2

But merge works nice:

print (pd.merge(DF_empty, DF_a, left_index=True, right_index=True))
Empty DataFrame
Columns: [a]
Index: []

SO question


print (pd.show_versions())
INSTALLED VERSIONS ------------------ commit: None python: 3.5.1.final.0 python-bits: 64 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: sk_SK LOCALE: None.None
pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 20.3
Cython: 0.23.4
numpy: 1.11.0
scipy: 0.17.0
statsmodels: 0.6.1
xarray: None
IPython: 4.1.2
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.5.1
pytz: 2016.2
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.5.2
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.6.0
bs4: 4.4.1
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0
pandas_datareader: 0.2.1
None

<\details>

Contributor

jreback commented Feb 7, 2017

ok, this is a bug, we normally filter out completely empty frames, but in this case we need to skip that condition

index 3fbd83a..5bf0546 100644
--- a/pandas/tools/merge.py
+++ b/pandas/tools/merge.py
@@ -1689,7 +1689,7 @@ class _Concatenator(object):
                            if sum(obj.shape) > 0 or isinstance(obj, Series)]
 
             if (len(non_empties) and (keys is None and names is None and
-                                      levels is None and join_axes is None)):
+                                      levels is None and join_axes is None and not self.intersect)):
                 objs = non_empties
                 sample = objs[0]
 

pull-request to add some tests (there are very little tests for using join= with concat) and fix would be great.

jreback added this to the 0.20.0 milestone Feb 7, 2017

jreback changed the title from Concat with inner join and empty DataFrame not works to BUG: Concat with inner join and empty DataFrame Feb 7, 2017

happy to work on that. does it amount to fixing /pandas/tools/merge.py and writing some related tests in pandas/pandas/tests/types/test_concat.py?

Contributor

jreback commented Feb 9, 2017

tests are in pandas/tools/tests/test_concat.py

and (shortly) this code is in pandas/tools/concat.py (just moving it).

jreback closed this in c7300ea Feb 16, 2017

@AnkurDedania AnkurDedania added a commit to AnkurDedania/pandas that referenced this issue Mar 21, 2017

@abaldenko @AnkurDedania abaldenko + AnkurDedania BUG: Concat with inner join and empty DataFrame
closes #15328

Author: abaldenko <abaldenko@massmutual.com>

Closes #15397 from abaldenko/concat_empty_dataframe and squashes the following commits:

47c8735 [abaldenko] BUG: Concat with inner join and empty DataFrame
fc473b7 [abaldenko] BUG: Concat with inner join and empty DataFrame
b86dcb6 [abaldenko] BUG: Concat with inner join and empty DataFrame
23401c1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment