BUG: concat of Series w/o names #10698 #10723

IamGianluca · 2015-08-02T11:42:29Z

Let the result of 'concat' to inherit the parent Series' names. The Series' name (if present) will be used as the resulting DataFrame column name. When only one of the Series has a valid name, the resulting DataFrame will inherit the name only, and use a column name for the other columns the column index value.

jreback · 2015-08-02T11:43:56Z

pandas/tools/merge.py

+                columns = []
+                for i in range(len(data)):
+                    columns.append(Series(data[i]).name if Series(data[i]).name is not None else i)
+                tmpdf.columns = columns


use a counter here - first missing should be 0

Hi jreback. I see your point. I've erroneously believed we wanted to use the column index as the column name in case the parent Series didn't have a valid name. I'm going to update the pull request to include the change you suggested.

Apologies in advance if my code is not always the neatest. I'm new to software development, although I use pandas everyday for data analysis. I thought actively participating to the project could be a good way for me to learn how to code properly and become a better programmer.

jreback · 2015-08-02T15:28:40Z

here are some helpful hints FYI: http://pandas.pydata.org/pandas-docs/stable/contributing.html

IamGianluca · 2015-08-02T20:22:03Z

Apologies I pushed a few "fixes" for some unit tests which were failing. Looking at the reason why those tests were failing I've noticed it was because of the use of the argument 'keys' in the function 'concat'. Most people are using it to pass the column names, which is something I personally don't do and therefore I didn't consider when writing the code. I'm going to reset my local branch to commit 'b9cba86ec1c11c200c44c62d0a300b7f326a12e4' and make the necessary changes to allow this case. Before sending the next pull request I'm going to make sure all nosetests pass correctly. This should also fix the build issue. Apologies again, this is due to lack of experience. I'll make sure to learn from my mistake.

jreback · 2015-08-04T22:36:41Z

pandas/tools/tests/test_merge.py

@@ -1797,6 +1797,15 @@ def test_concat_dataframe_keys_bug(self):
        self.assertEqual(list(result.columns), [('t1', 'value'),
                                                ('t2', 'value')])

+    def test_concat_series_partial_columns_names(self):


add the issue number as a comment here

I've added the reference to the GitHub issue

IamGianluca · 2015-08-08T14:14:08Z

@jreback Thanks for spending the time to review my pull request. I'm going to start working on another bug now.

jreback · 2015-08-10T11:08:06Z

pls add a whatsnew note. This is just a bug fix, but I think should have a mini-example. pls have squash as well.

IamGianluca · 2015-08-11T12:53:39Z

I think I made a mistake. I rebased my local version because was missing some of the recent updates in the upstream version and (after solving some merging conflicts) I pushed. I see there is an alert message saying "This branch has conflicts that must be resolved" on my pull request in GitHub. Apologies for that. Is there a way I can solve this?

jreback · 2015-08-11T13:16:24Z

you need to rebase / squash. If you have a conflict it will show up when you do this. Conflicts are normal and happen because others have changed code that you are changing. See the contributing docs here

IamGianluca · 2015-08-11T14:06:09Z

Okay! So it should be fine right?

max-sixty · 2015-08-11T14:22:59Z

Conflicts are fine, as in they're not a sign you've done anything wrong. But they do need to be resolved before merge - the docs describe how to squash & merge. Cheers

IamGianluca · 2015-08-11T20:24:01Z

Okay guys, I think I did it. Please let me know if I did something wrong. Apologies again for my lack of experience.

jreback · 2015-08-15T17:28:58Z

doc/source/whatsnew/v0.17.0.txt

@@ -152,6 +153,30 @@ Other enhancements
   s.drop_duplicates(keep=False)


+- ``concat`` will now inherit the existing series names (even when some are missing), if new ones are not provided through the ``keys`` argument (:issue:`10698`).


I would say something like: will use existing Series names if provided.

jreback · 2015-08-15T17:30:24Z

might need a doc-example http://pandas.pydata.org/pandas-docs/stable/merging.html#more-concatenating-with-group-keys to explain how keys overrides this (and add a test as well)

jreback · 2015-08-18T10:51:21Z

can you update and add a doc example?

IamGianluca · 2015-08-18T11:29:45Z

Sure! I think I messed up with my branch because lots of test are failing after I rebased last day. I'll fix this ASAP.

IamGianluca · 2015-08-19T07:53:37Z

I see the branch has conflicts. I'm assuming in this sort of situations I need to fix those myself right?

In the effort of solving them using git mergetool I've noticed that sometimes there are pieces of code in my LOCAL branch which are not present in the REMOTE, but I haven't create them myself. What I usually do is to update my master branch:

# go to the master branch
git checkout master
# pull changes from github
git fetch upstream
# update the master branch
git rebase upstream/master
# push it to your Github repo
git push

Then I update the local branch:

# go to the feature branch
git checkout my-new-feature
# make a backup in case you mess up
git branch tmp my-new-feature
# rebase on master
git rebase master

The rebase will fail and suggest me to use mergetool to solve the conflicts manually. Now, how should I behave in situations like the one in the attached picture?

In the picture above the document on the left is my local, the one in the middle is the result of the merge, and the one on the right is the remote. In this occasions I tend to pick from the remote everything I don't have in my local branch, because I assume these are coming from other submitted pull requests or code merged while I was working on my local branch. What about the code which is in my local, hasn't been written by me, and is not in the remote? Should I discard it?

jreback · 2015-08-20T13:00:50Z

doc/source/whatsnew/v0.17.0.txt

+
+  .. ipython:: python
+
+    foo = pd.Series([1,2], name='foo')


just create all of these in a separate python block above previous behavior. Then no need to show them twice. (obviously show the results in the previous behavior section as a code-block then in an ipython block in new behavior).

jreback · 2015-08-20T13:02:20Z

need to rebase / squash. generally you won't have conflicts. other people add code and should be straightforward to accept.

jorisvandenbossche · 2015-08-29T23:53:07Z

doc/source/merging.rst

+
+.. ipython:: python
+
+   s3 = Series([0, 1, 2, 3], name='foo')


Can you use pd. before all Series and concat calls?

We are converting the docs (already partly, but maybe this one not)

jorisvandenbossche · 2015-08-29T23:55:10Z

Seems you have a bit too much changes in the whatsnew file?

IamGianluca · 2015-08-30T09:59:03Z

Joris, my bad, I messed up when merging. I think I've solved all the issues now.

jreback · 2015-08-30T11:45:40Z

pandas/tools/merge.py


 import pandas.core.common as com

+import pandas.lib as lib
 import pandas.algos as algos


extra import?

@jreback You're right. I've fixed!

jreback · 2015-09-01T12:02:16Z

can you update according to comments

BUG: concat of Series w/o names #10698

jreback · 2015-09-02T11:54:22Z

@IamGianluca awesome job! thanks!

jreback reviewed Aug 2, 2015
View reviewed changes

jreback changed the title ~~Issue 10698 fix~~ BUG: concat of Series w/o names #10698 Aug 2, 2015

jreback added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Aug 2, 2015

jreback added this to the 0.17.0 milestone Aug 2, 2015

jreback reviewed Aug 4, 2015
View reviewed changes

jreback reviewed Aug 15, 2015
View reviewed changes

jreback reviewed Aug 20, 2015
View reviewed changes

jorisvandenbossche reviewed Aug 29, 2015
View reviewed changes

jreback reviewed Aug 30, 2015
View reviewed changes

BUG: Concat of Series w/o names. Closes #10698

fa29a13

jreback added a commit that referenced this pull request Sep 2, 2015

Merge pull request #10723 from IamGianluca/issue_10698_fix

207efc2

BUG: concat of Series w/o names #10698

jreback merged commit 207efc2 into pandas-dev:master Sep 2, 2015

jreback mentioned this pull request Sep 12, 2015

BUG: empty Series concat has no effect #11082

Closed

sinhrks mentioned this pull request Apr 10, 2016

BUG: empty Series concat has no effect #12846

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: concat of Series w/o names #10698 #10723

BUG: concat of Series w/o names #10698 #10723

IamGianluca commented Aug 2, 2015

jreback Aug 2, 2015

IamGianluca Aug 2, 2015

jreback commented Aug 2, 2015

IamGianluca commented Aug 2, 2015

jreback Aug 4, 2015

IamGianluca Aug 30, 2015

IamGianluca commented Aug 8, 2015

jreback commented Aug 10, 2015

IamGianluca commented Aug 11, 2015

jreback commented Aug 11, 2015

IamGianluca commented Aug 11, 2015

max-sixty commented Aug 11, 2015

IamGianluca commented Aug 11, 2015

jreback Aug 15, 2015

jreback commented Aug 15, 2015

jreback commented Aug 18, 2015

IamGianluca commented Aug 18, 2015

IamGianluca commented Aug 19, 2015

jreback Aug 20, 2015

jreback commented Aug 20, 2015

jorisvandenbossche Aug 29, 2015

jorisvandenbossche Aug 29, 2015

jorisvandenbossche commented Aug 29, 2015

IamGianluca commented Aug 30, 2015

jreback Aug 30, 2015

IamGianluca Aug 30, 2015

jreback commented Sep 1, 2015

jreback commented Sep 2, 2015

		@@ -152,6 +153,30 @@ Other enhancements
		s.drop_duplicates(keep=False)


		- ``concat`` will now inherit the existing series names (even when some are missing), if new ones are not provided through the ``keys`` argument (:issue:`10698`).

BUG: concat of Series w/o names #10698 #10723

BUG: concat of Series w/o names #10698 #10723

Conversation

IamGianluca commented Aug 2, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Aug 2, 2015

IamGianluca commented Aug 2, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IamGianluca commented Aug 8, 2015

jreback commented Aug 10, 2015

IamGianluca commented Aug 11, 2015

jreback commented Aug 11, 2015

IamGianluca commented Aug 11, 2015

max-sixty commented Aug 11, 2015

IamGianluca commented Aug 11, 2015

Choose a reason for hiding this comment

jreback commented Aug 15, 2015

jreback commented Aug 18, 2015

IamGianluca commented Aug 18, 2015

IamGianluca commented Aug 19, 2015

Choose a reason for hiding this comment

jreback commented Aug 20, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorisvandenbossche commented Aug 29, 2015

IamGianluca commented Aug 30, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Sep 1, 2015

jreback commented Sep 2, 2015