New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GOTCHA: You can't use dot notation to add a column to a DataFrame #7175

Closed
AllenDowney opened this Issue May 19, 2014 · 21 comments

Comments

Projects
None yet
6 participants
@AllenDowney
Contributor

AllenDowney commented May 19, 2014

I ran into a nasty gotcha I have not seen mentioned in the documentation. In general, you can refer to DataFrame columns like a dictionary key or like an object attribute. But the behavior is inconsistent. In particular, you can modify an existing column using dot notation, but you can't add a new column.

The following example demonstrates.

d = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']),
     'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}
df = pd.DataFrame(d)

# this adds a column to df
df['three'] = df.two + 1

# this successfully modifies an existing column
df.one += 1

# this seems to work but does not add a column to df
df.four = df.two + 2

print df.three
print df.four
print df

Is this behavior unavoidable?

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 19, 2014

Contributor

see docs here: http://pandas-docs.github.io/pandas-docs-travis/indexing.html#attribute-access
I guess an additional warning to that effect could be added.

This whole attribute access bizness if a bit of a hybrid, e.g. it WILL work if the column already exists, but not for a completely new column.

The reason is that it is VERY error prone; e.g. very easy to overwite an existing method (though this could raise). Furthermore people often want to simply assign an attribute e.g. df.name = 'foo' or whatever, so very tricky to figure out what is needed (e.g. is that an additional column?)

IMHO the assignment should be completely banned and only allow read-only access, but I may be in a minority view.

dupe of this issue: #6160

Contributor

jreback commented May 19, 2014

see docs here: http://pandas-docs.github.io/pandas-docs-travis/indexing.html#attribute-access
I guess an additional warning to that effect could be added.

This whole attribute access bizness if a bit of a hybrid, e.g. it WILL work if the column already exists, but not for a completely new column.

The reason is that it is VERY error prone; e.g. very easy to overwite an existing method (though this could raise). Furthermore people often want to simply assign an attribute e.g. df.name = 'foo' or whatever, so very tricky to figure out what is needed (e.g. is that an additional column?)

IMHO the assignment should be completely banned and only allow read-only access, but I may be in a minority view.

dupe of this issue: #6160

@jreback jreback closed this May 19, 2014

@jreback jreback added the Dupe label May 19, 2014

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 19, 2014

Contributor

actually....going to reopen this issue; I think what you are doing should raise, e.g. assigning a pandas object to a non-existant column

Contributor

jreback commented May 19, 2014

actually....going to reopen this issue; I think what you are doing should raise, e.g. assigning a pandas object to a non-existant column

@jreback jreback reopened this May 19, 2014

@jreback jreback added Error Reporting and removed Dupe labels May 19, 2014

@jreback jreback added this to the 0.15.0 milestone May 19, 2014

@jreback jreback changed the title from GOTCHA: You can't use dot notation to add a column to a DataFram to GOTCHA: You can't use dot notation to add a column to a DataFrame May 19, 2014

@AllenDowney

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

AllenDowney May 19, 2014

Contributor

Thanks for considering the issue. The documentation you linked says that
"Setting is allowed as well", but does not distinguish modifying an
existing column from creating a new one (unless I am not reading carefully
enough).

On Mon, May 19, 2014 at 11:52 AM, jreback notifications@github.com wrote:

Reopened #7175 #7175.


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#event-122520592
.

Contributor

AllenDowney commented May 19, 2014

Thanks for considering the issue. The documentation you linked says that
"Setting is allowed as well", but does not distinguish modifying an
existing column from creating a new one (unless I am not reading carefully
enough).

On Mon, May 19, 2014 at 11:52 AM, jreback notifications@github.com wrote:

Reopened #7175 #7175.


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#event-122520592
.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback May 19, 2014

Contributor

yes I think that is an oversite. would you like to do a PR to put in a warning about the setting part? (e.g. that can only set an existing column), as this won't be fixed till next release at the earliest (docs can put in now).

Contributor

jreback commented May 19, 2014

yes I think that is an oversite. would you like to do a PR to put in a warning about the setting part? (e.g. that can only set an existing column), as this won't be fixed till next release at the earliest (docs can put in now).

@AllenDowney

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

AllenDowney May 20, 2014

Contributor

Sure, but I don't know where the source lives for the docs.

On Mon, May 19, 2014 at 12:07 PM, jreback notifications@github.com wrote:

yes I think that is an oversite. would you like to do a PR to put in a
warning about the setting part? (e.g. that can only set an existing
column), as this won't be fixed till next release at the earliest (docs can
put in now).


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#issuecomment-43523616
.

Contributor

AllenDowney commented May 20, 2014

Sure, but I don't know where the source lives for the docs.

On Mon, May 19, 2014 at 12:07 PM, jreback notifications@github.com wrote:

yes I think that is an oversite. would you like to do a PR to put in a
warning about the setting part? (e.g. that can only set an existing
column), as this won't be fixed till next release at the earliest (docs can
put in now).


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#issuecomment-43523616
.

@jreback

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

AllenDowney May 20, 2014

Contributor

PR submitted

On Tue, May 20, 2014 at 2:25 PM, jreback notifications@github.com wrote:

pandas/docs/source/indexing.rst


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#issuecomment-43664480
.

Contributor

AllenDowney commented May 20, 2014

PR submitted

On Tue, May 20, 2014 at 2:25 PM, jreback notifications@github.com wrote:

pandas/docs/source/indexing.rst


Reply to this email directly or view it on GitHubhttps://github.com//issues/7175#issuecomment-43664480
.

@onesandzeroes

This comment has been minimized.

Show comment
Hide comment
@onesandzeroes

onesandzeroes Sep 22, 2014

Contributor

@jreback Just came across this on StackOverflow with this question. This seems like pretty weird/surprising behaviour to me, is there any good reason why this shouldn't raise or at least warn? I might try changing this to a raise and see how many tests it breaks.

Contributor

onesandzeroes commented Sep 22, 2014

@jreback Just came across this on StackOverflow with this question. This seems like pretty weird/surprising behaviour to me, is there any good reason why this shouldn't raise or at least warn? I might try changing this to a raise and see how many tests it breaks.

@AllenDowney

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

AllenDowney Sep 22, 2014

Contributor

I would argue for a warning rather than an exception. It is not illegal to
attach a new attribute to a DataFrame (although it is probably not great
style).

Allen

On Mon, Sep 22, 2014 at 3:09 AM, onesandzeroes notifications@github.com
wrote:

@jreback https://github.com/jreback Just came across this on
StackOverflow with this question
http://stackoverflow.com/questions/25968118/how-to-delete-an-item-from-a-pandas-dataframe.
This seems like pretty weird/surprising behaviour to me, is there any good
reason why this shouldn't raise or at least warn? I might try changing
this to a raise and see how many tests it breaks.


Reply to this email directly or view it on GitHub
#7175 (comment).

Contributor

AllenDowney commented Sep 22, 2014

I would argue for a warning rather than an exception. It is not illegal to
attach a new attribute to a DataFrame (although it is probably not great
style).

Allen

On Mon, Sep 22, 2014 at 3:09 AM, onesandzeroes notifications@github.com
wrote:

@jreback https://github.com/jreback Just came across this on
StackOverflow with this question
http://stackoverflow.com/questions/25968118/how-to-delete-an-item-from-a-pandas-dataframe.
This seems like pretty weird/surprising behaviour to me, is there any good
reason why this shouldn't raise or at least warn? I might try changing
this to a raise and see how many tests it breaks.


Reply to this email directly or view it on GitHub
#7175 (comment).

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Sep 22, 2014

Contributor

I think could deal with this in several ways:

  • add an option (mode.attribute_assignment).that by default warns (but could alsoe None or raise) to control this: that way everyone/noone is happy!
  • invoke the option if a method is attempting to be overwritten (and if you actually DO want to monkey-patch you would then simply do this in a context manager to remove the warning), so that doing df.max = 'foo' will warn
  • still it remains ambiguous whether df.Age = 'foo' is meant for df['Age'] = 'foo' or really an attribute assignement. I think you could allow the former, then need a way to do the latter
    (like maybe df.set_attribute('Age','foo')
Contributor

jreback commented Sep 22, 2014

I think could deal with this in several ways:

  • add an option (mode.attribute_assignment).that by default warns (but could alsoe None or raise) to control this: that way everyone/noone is happy!
  • invoke the option if a method is attempting to be overwritten (and if you actually DO want to monkey-patch you would then simply do this in a context manager to remove the warning), so that doing df.max = 'foo' will warn
  • still it remains ambiguous whether df.Age = 'foo' is meant for df['Age'] = 'foo' or really an attribute assignement. I think you could allow the former, then need a way to do the latter
    (like maybe df.set_attribute('Age','foo')
@AllenDowney

This comment has been minimized.

Show comment
Hide comment
@AllenDowney

AllenDowney Sep 22, 2014

Contributor

I think there are three cases to consider for

df.foo = bar

  1. If foo is the name of an existing column

Current behavior: replace the contents of the column with bar
Desired behavior: same

  1. If foo is the name of a DataFrame method

Current behavior: clobbers the existing method
Desired behavior: warning

  1. If foo doesn't exist

Current behavior: adds an attribute to the DataFrame (not a column)
Desired behavior: warning

Does that sound right?

If so, then I think the warning message should give instructions for
suppressing the warning. Now there are two options for suppressing the
warning:

  1. Provide an option. Pro: you can suppress once and never worry again.
    Con: you can suppress once and forget about it.

  2. Require the programmer to catch the warning. Pro: You have to actively
    suppress every time. Con: You have to actively suppress every time.

Jeff, you suggested that there could be a way to set an attribute named foo
even if there is a column named foo. I assume you were kidding ;)

Allen

On Mon, Sep 22, 2014 at 9:33 AM, jreback notifications@github.com wrote:

I think could deal with this in several ways:

  • add an option (mode.attribute_assignment).that by default warns (but
    could alsoe None or raise) to control this: that way everyone/noone is
    happy!
  • invoke the option if a method is attempting to be overwritten (and
    if you actually DO want to monkey-patch you would then simply do this in a
    context manager to remove the warning), so that doing df.max = 'foo'
    will warn
  • still it remains ambiguous whether df.Age = 'foo' is meant for df['Age']
    = 'foo' or really an attribute assignement. I think you could allow
    the former, then need a way to do the latter (like maybe
    df.set_attribute('Age','foo')


Reply to this email directly or view it on GitHub
#7175 (comment).

Contributor

AllenDowney commented Sep 22, 2014

I think there are three cases to consider for

df.foo = bar

  1. If foo is the name of an existing column

Current behavior: replace the contents of the column with bar
Desired behavior: same

  1. If foo is the name of a DataFrame method

Current behavior: clobbers the existing method
Desired behavior: warning

  1. If foo doesn't exist

Current behavior: adds an attribute to the DataFrame (not a column)
Desired behavior: warning

Does that sound right?

If so, then I think the warning message should give instructions for
suppressing the warning. Now there are two options for suppressing the
warning:

  1. Provide an option. Pro: you can suppress once and never worry again.
    Con: you can suppress once and forget about it.

  2. Require the programmer to catch the warning. Pro: You have to actively
    suppress every time. Con: You have to actively suppress every time.

Jeff, you suggested that there could be a way to set an attribute named foo
even if there is a column named foo. I assume you were kidding ;)

Allen

On Mon, Sep 22, 2014 at 9:33 AM, jreback notifications@github.com wrote:

I think could deal with this in several ways:

  • add an option (mode.attribute_assignment).that by default warns (but
    could alsoe None or raise) to control this: that way everyone/noone is
    happy!
  • invoke the option if a method is attempting to be overwritten (and
    if you actually DO want to monkey-patch you would then simply do this in a
    context manager to remove the warning), so that doing df.max = 'foo'
    will warn
  • still it remains ambiguous whether df.Age = 'foo' is meant for df['Age']
    = 'foo' or really an attribute assignement. I think you could allow
    the former, then need a way to do the latter (like maybe
    df.set_attribute('Age','foo')


Reply to this email directly or view it on GitHub
#7175 (comment).

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Sep 22, 2014

Contributor

@AllenDowney good summary. pull-request?

I had meant that df.a = 'foo' could translate to df['a'] = 'foo', but then you would have to have a method df.set_attribute('a2','foo') that could then set an actual attribute. But on 2nd thought this is really a big change. So let's go with your summary.

I think an option for the warning is the best as it will happend EVERY time (unless you turn of the option entirely, and can be worked around by using pd.option_context to set the context

Contributor

jreback commented Sep 22, 2014

@AllenDowney good summary. pull-request?

I had meant that df.a = 'foo' could translate to df['a'] = 'foo', but then you would have to have a method df.set_attribute('a2','foo') that could then set an actual attribute. But on 2nd thought this is really a big change. So let's go with your summary.

I think an option for the warning is the best as it will happend EVERY time (unless you turn of the option entirely, and can be worked around by using pd.option_context to set the context

@jtratner

This comment has been minimized.

Show comment
Hide comment
@jtratner

jtratner Sep 22, 2014

Contributor

+1 for warning. Also if you make the warning message generic (i.e., don't
include the attribute name), you'll only see the warning once per script,
because I believe the default is to suppress warnings that are generated
multiple times.

On Mon, Sep 22, 2014 at 7:28 AM, jreback notifications@github.com wrote:

@AllenDowney https://github.com/AllenDowney good summary. pull-request?

I had meant that df.a = 'foo' could translate to df['a'] = 'foo', but
then you would have to have a method df.set_attribute('a2','foo') that
could then set an actual attribute. But on 2nd thought this is really a big
change. So let's go with your summary.

I think an option for the warning is the best as it will happend EVERY
time (unless you turn of the option entirely, and can be worked around by
using pd.option_context to set the context


Reply to this email directly or view it on GitHub
#7175 (comment).

Contributor

jtratner commented Sep 22, 2014

+1 for warning. Also if you make the warning message generic (i.e., don't
include the attribute name), you'll only see the warning once per script,
because I believe the default is to suppress warnings that are generated
multiple times.

On Mon, Sep 22, 2014 at 7:28 AM, jreback notifications@github.com wrote:

@AllenDowney https://github.com/AllenDowney good summary. pull-request?

I had meant that df.a = 'foo' could translate to df['a'] = 'foo', but
then you would have to have a method df.set_attribute('a2','foo') that
could then set an actual attribute. But on 2nd thought this is really a big
change. So let's go with your summary.

I think an option for the warning is the best as it will happend EVERY
time (unless you turn of the option entirely, and can be worked around by
using pd.option_context to set the context


Reply to this email directly or view it on GitHub
#7175 (comment).

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Sep 22, 2014

Contributor

ahh, but I think we DO want this warning to keep happening!

Contributor

jreback commented Sep 22, 2014

ahh, but I think we DO want this warning to keep happening!

@Zenadix

This comment has been minimized.

Show comment
Hide comment
@Zenadix

Zenadix Dec 11, 2014

Yes. This should trigger a warning.

Zenadix commented Dec 11, 2014

Yes. This should trigger a warning.

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 11, 2016

+1 for a warning (pandas user here which just run into this issue).

ghost commented Mar 11, 2016

+1 for a warning (pandas user here which just run into this issue).

@jreback jreback added this to the 0.18.1 milestone Mar 11, 2016

@jreback jreback removed this from the Next Major Release milestone Mar 11, 2016

@jreback jreback modified the milestones: 0.18.2, 0.18.1 Apr 25, 2016

@jreback jreback modified the milestones: 0.18.2, Next Major Release Jul 6, 2016

@paulwasit

This comment has been minimized.

Show comment
Hide comment
@paulwasit

paulwasit Nov 1, 2016

Has the warning been implemented ? The milestone says 0.18.2, but on 0.19.0 it doesn't seem to appear.

paulwasit commented Nov 1, 2016

Has the warning been implemented ? The milestone says 0.18.2, but on 0.19.0 it doesn't seem to appear.

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 1, 2016

Contributor

no, it says Next Major Release, and this issue is still open. pull-requests to implement are welcome.

Contributor

jreback commented Nov 1, 2016

no, it says Next Major Release, and this issue is still open. pull-requests to implement are welcome.

@paulwasit

This comment has been minimized.

Show comment
Hide comment
@paulwasit

paulwasit Nov 1, 2016

Ah ok. I would gladly give it a try, but I am not sure where to start looking :/

paulwasit commented Nov 1, 2016

Ah ok. I would gladly give it a try, but I am not sure where to start looking :/

@jreback

This comment has been minimized.

Show comment
Hide comment
@jreback

jreback Nov 1, 2016

Contributor

right here

easy enough to see if one is setting using a pandas object but the column is not defined, then could show a warning

Contributor

jreback commented Nov 1, 2016

right here

easy enough to see if one is setting using a pandas object but the column is not defined, then could show a warning

@paulwasit

This comment has been minimized.

Show comment
Hide comment
@paulwasit

paulwasit Nov 1, 2016

Thanks @jreback ! I will have a look and see what I can do.

paulwasit commented Nov 1, 2016

Thanks @jreback ! I will have a look and see what I can do.

deniederhut added a commit to deniederhut/pandas that referenced this issue Jul 15, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

deniederhut added a commit to deniederhut/pandas that referenced this issue Jul 19, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

@jreback jreback modified the milestones: 0.21.0, Next Major Release Jul 19, 2017

deniederhut added a commit to deniederhut/pandas that referenced this issue Jul 19, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

deniederhut added a commit to deniederhut/pandas that referenced this issue Jul 27, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

deniederhut added a commit to deniederhut/pandas that referenced this issue Jul 29, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

deniederhut added a commit to deniederhut/pandas that referenced this issue Aug 4, 2017

ENH: adds warning when setting list-like into attribute
Adds warning in generic setattr logical branch for when attribute does
not exist and user is supplying a list-like object. Warning states that
Series cannot be assigned into nonexistent columns, and includes a link
to stable documentation. Closes pandas-dev#7175.

jreback added a commit that referenced this issue Aug 7, 2017

rs2 added a commit to rs2/pandas that referenced this issue Aug 30, 2017

Rebase from master (#1)
* consolidated the duplicate definitions of NA values (in parsers & IO) (#16589)

* GH15943 Fixed defaults for compression in HDF5 (#16355)

* DOC: add header=None to read_excel docstring (#16689)

* TST: Test against python-dateutil master (#16648)

* BUG: .iloc[:] and .loc[:] return a copy of the original object #13873 (#16443)

closes #13873

* TST: Add test of building frame from named Series and columns (#9232) (#16700)

* DOC: fix wrongly placed versionadded (#16702)

* DOC: pin sphinx to version 1.5 (#16704)

* CI: restore np 113 in ci builds (#16656)

* Revert "BLD: fix numpy on 3.6 build as 1.13 was released but no deps are built for it (#16633)"

This reverts commit dfebd8a.

closes #16634

* BUG: Fix regression for RGB(A) color arguments (#16701)

* Add test

* Pass tuples that are RGB or RGBA like in list

* Update what's new

* change whatsnew to reflect regression fix

* Add test for RGBA as well

* CI: pin jemalloc=4.4.0 (#16727)

* MAINT: Drop Categorical.order & sort (#16728)

Deprecated back in 0.18.1

xref gh-12882

* Fix reading Series with read_hdf (#16610)

* Added test to reproduce issue #16583

* Fix #16583 by adding an explicit `mode` argument to `read_hdf`

kwargs which are meant for the opening of the HDFStore should be filtered out
before passing the remaining kwargs to the `select` function to load the data.

* Noted fix for #16583 in WhatsNew

* DOC: typo (#16733)

* whatsnew v0.21.0.txt typos (#16742)

* whatsnew v0.20.3 edits (#16743)

* BUG: do not raise UnsortedIndexError if sorting is not required

closes #16734

Author: Pietro Battiston <me@pietrobattiston.it>

This patch had conflicts when merged, resolved by
Committer: Jeff Reback <jeff.reback@twosigma.com>

Closes #16736 from toobaz/index_what_you_can and squashes the following commits:

f77e2b3 [Pietro Battiston] BUG: do not raise UnsortedIndexError if sorting is not required

* DOC: whatsnew typos

* Test for #16726. unittest that ensures datetime is understood (#16744)

* Test for #16726. unittest that ensures datetime is understood

* Corrected the test as suggested by @TomAugspurger

* Fixed flake8 errors and warnings

* DOC: some rst fixes (#16763)

* DOC: Update Sphinx Deprecated Directive (#16512)

* MAINT: Drop Index.sym_diff (#16760)

Deprecated in 0.18.1

xref gh-12591, gh-12594

* MAINT: Drop pd.options.display.mpl_style (#16761)

Deprecated in 0.18.0

xref gh-12190

* DOC: remove section on Panel4D support in HDF io (#16783)

* DOC: add section on data validation and library engarde (#16758)

* TST: register slow marker (#16797)

* TST: register slow marker

* Update setup.cfg

* BUG: Load data from a CategoricalIndex for dtype comparison, closes #… (#16738)

* BUG: Load data from a CategoricalIndex for dtype comparison, closes #16627

* Enable is_dtype_equal on CategoricalIndex, fixed some doc typos, added ordered CategoricalIndex test

* Flake8 windows suggestion

* Fixed some documentation/formatting issues, clarified the purpose of the test case.

* Bug in pd.merge() when merge/join with multiple categorical columns (#16786)

closes #16767

* BUG: Fix read of py3 PeriodIndex DataFrame HDF made in py2 (#16781) (#16790)

In Python3, reading a DataFrame with a PeriodIndex from an HDF file
created in Python2 would incorrectly return a DataFrame with an
Int64Index.

* BUG: Fix Series doesn't work in pd.astype(). Now treat Series as dict. (#16725)

* FIX: Allow aggregate to return dictionaries again #16741 (#16752)

* BUG: fix to_latex bold_rows option (#16708)

* Revert "CI: pin jemalloc=4.4.0 (#16727)" (#16731)

This reverts commit 09d8c22.

* CI: use dist/trusty rather than os/linux (#16806)

closes #16730

* TST: Verify columns entirely below chop_threshold still print (#6839) (#16809)

* BUG: clip dataframe column-wise #15390 (#16504)

* TST: Verify that positional shifting works with duplicate columns (#9092) (#16810)

* BUG: render dataframe as html do not produce duplicate element id's (#16780) (#16801)

* BUG: when rendering dataframe as html do not produce duplicate element id's #16780

* CLN: removing spaces in code causes pylint check to fail

* DOC: moved whatsnew comment to 0.20.3 release from 0.21.0

* fix BUG: ValueError when performing rolling covariance on multi indexed DataFrame (#16814)

* fix multi index names

* fix line length to pep8

* added what's new entry and reference issue number in test

* Update test_multi.py

* Update v0.20.3.txt

* BUG: rolling.cov with multi-index columns should presever the MI (#16825)

xref #16814

* use network decorator on additional tests (#16824)

* BUG: TimedeltaIndex raising ValueError when slice indexing (#16637) (#16638)

* Bug issue 16819 Index.get_indexer_not_unique inconsistent return types vs get_indexer (#16826)

* TST: Verify that float columns stay float after pivot (#7142) (#16815)

* BUG/MAINT: Change default of inplace to False in pd.eval (#16732)

* BUG: kind parameter on categorical argsort (#16834)

* DOC: Updated cookbook to show usage of Grouper instead of TimeGrouper… (#16794)

* BUG: allow empty multiindex (fixes .isin regression, GH16777) (#16782)

* BUG: fix missing sort keyword for PeriodIndex.join (#16586)

* COMPAT: 32-bit compat for testing of indexers (#16849)

xref #16826

* BUG: fix infer frequency for business daily (#16683)

* DOC: Whatsnew updates (#16853)

[ci skip]

* TST/PKG: Move test HDF5 file to legacy (#16856)

It wasn't being picked up in our package data otherwise

* COMPAT: moar 32-bit compat for testing of indexers (#16861)

xref #16826

* MAINT: Drop the get_offset_name method (#16863)

Deprecated since 0.18.0

xref gh-11834

* DOC: Fix missing parentheses in documentation (#16862)

* BUG: rolling.quantile does not return an interpolated result (#16247)

* ENH - Modify Dataframe.select_dtypes to accept scalar values (#16860)

* COMPAT: moar 32-bit compat for testing of indexers (#16869)

xref #16826

* Confirm that select was *not* clearer in 0.12 (#16878)

* Added tests for _get_dtype (#16845)

* BUG: Series.isin fails or categoricals (#16858)

* COMPAT with dateutil 2.6.1, fixed ambiguous tz dst behavior (#16880)

* fix wrongly named method (#16881)

* TST/PKG: Removed pandas.util.testing.slow definition (#16852)

* MAINT: Remove unused mock import (#16908)

We import it, set it as an attribute, and then don't use it.

* Let _get_dtype accept Categoricals and CategoricalIndex  (#16887)

* Fixes for #16896(TimedeltaIndex indexing regression for strings) (#16907)

* Fix for #16909(DeltatimeIndex.get_loc is not working on np.deltatime64 data type) (#16912)

* DOC: Recommend sphinx 1.5 for now (#16929)

For the SciPy sprint tomorrow, until the cause of the doc-building slowdown is fully identified.

* BUG: Allow value labels to be read with iterator (#16926)

All value labels to be read before the iterator has been used
Fix issue where categorical data was incorrectly reformatted when
write_index was False

closes #16923

* DOC: Update flake8 command instructions (#16919)

* TST: Don't assert that a bug exists in numpy (#16940)

Better to ignore the warning from the bug, rather than assert the bug is still there

After this change, numpy/numpy#9412 _could_ be backported to fix the bug

* CI: add .pep8speakes.yml

* CLN16668: remove OrderedDefaultDict (#16939)

* Change "pls" to "please" in error message (#16947)

* BUG: MultiIndex sort with ascending as list (#16937)

* DOC: Improving docstring of pop method (#16416) (#16520)

* PEP8

* WARN: add stacklevel to to_dict() UserWarning (#16927) (#16936)

* ERR: add stacklevel to to_dict() UserWarning (#16927)

* TST: Add warning testing to to_dict()

* Fix warning assertion on to_dict() test

* Add github issue to documentation on to_dict() warning test

* CI: fix pep8speaks .yml file

* DOC: whatsnew 0.21.0 edits

* CI: disable codecov reporting

* MAINT: Move series.remove_na to core.dtypes.missing.remove_na_arraylike

Closes gh-16935

* Support non unique period indexes on join and merge operations (#16949)

* Support non unique period indexes on join and merge operations

* Add frame assertion on tests and release notes

* Explicitly use dtype int64 on arange

* BUG: Set secondary axis font size for `secondary_y` during plotting

The parameter was not being respected for `secondary_y`.

Closes gh-12565

* DOC: more whatsnew fixes

* DOC: Reset index examples

closes #16416

Author: aernlund <awe220@nyumc.org>

Closes #16967 from aernlund/reset_index_docs and squashes the following commits:

3c6a4b6 [aernlund] DOC: added examples to reset_index
4838155 [aernlund] DOC: added examples to reset_index
2a51e2b [aernlund] DOC: added examples to reset_index

* channel from pandas to conda-forge (#16966)

* BUG: coercing of bools in groupby transform (#16895)

* DOC: misspelling in DatetimeIndex.indexer_between_time [CI skip] (#16963)

* CLN: some residual code removed, xref to #16761 (#16974)

* ENH: Create a 'Y' alias for date_range yearly frequency

Closes gh-9313

* Revert "ENH: Create a 'Y' alias for date_range yearly frequency" (#16976)

This reverts commit 9c096d2, as it was prematurely made.

* DOC: behavior when slicing with missing bounds (#16932)

closes #16917

* TST: Add test for sub-char in read_csv (#16977)

Closes gh-16893.

* DEPR: deprecate html.border option (#16970)

* DOC: document convention argument for resample() (#16965)

* DOC: document convention argument for resample()

* DOC: Clarify 'it' in aggregate doc (#16989)

Closes gh-16988.

* CLN/COMPAT: for various py2/py3 in doc/bench scripts (#16984)

* PERF: SparseDataFrame._init_dict uses intermediary dict, not DataFrame (#16883)

Closes gh-16773.

* MAINT: Drop line_width and height from options (#16993)

Deprecated since 0.11 and 0.12 respectively.

* COMPAT: Add back remove_na for seaborn (#16992)

Closes gh-16971.

* COMPAT: np.full not available in all versions, xref #16773 (#17000)

* DOC, TST: Clarify whitespace behavior in read_fwf documentation (#16950)

Closes gh-16772

* API: add infer_objects for soft conversions (#16915)

* API: add infer_objects for soft conversions

* doc fixups

* fixups

* doc

* BUG: np.inf now causes Index to upcast from int to float (#16996)

Closes gh-16957.

* DOC: Make highlight functions match documentation (#16999)

Closes gh-16998.

* BUG: Large object array isin

closes #16012

Author: Morgan Stuart <morgansstuart243@gmail.com>

Closes #16969 from Morgan243/large_array_isin and squashes the following commits:

31cb4b3 [Morgan Stuart] Removed unneeded details from whatsnew description
4b59745 [Morgan Stuart] Linting errors; additional test clarification
186607b [Morgan Stuart] BUG #16012 - fix isin for large object arrays

* BUG: reindex would throw when a categorical index was empty #16770

closes #16770

Author: ri938 <r_irv938@hotmail.com>
Author: Jeff Reback <jeff@reback.net>
Author: Tuan <tuan.d.tran@hotmail.com>
Author: Forbidden Donut <forbdonut@gmail.com>

This patch had conflicts when merged, resolved by
Committer: Jeff Reback <jeff@reback.net>

Closes #16820 from ri938/bug_issue16770 and squashes the following commits:

0e2d315 [ri938] Merge branch 'master' into bug_issue16770
9802288 [ri938] Update v0.20.3.txt
1f2865e [ri938] Update v0.20.3.txt
83fd749 [ri938] Update v0.20.3.txt
eab3192 [ri938] Merge branch 'master' into bug_issue16770
7acc09f [ri938] Minor correction to previous submit
6e8f1b3 [ri938] Minor corrections to previous submit (#16820)
9ed80f0 [ri938] Bring documentation into line with master branch.
26e1a60 [ri938] Move documentation of change to the next major release 0.21.0
59b17cd [Jeff Reback] BUG: rolling.cov with multi-index columns should presever the MI (#16825)
5362447 [Tuan] fix BUG: ValueError when performing rolling covariance on multi indexed DataFrame (#16814)
800b40d [ri938] BUG: render dataframe as html do not produce duplicate element id's (#16780) (#16801)
a725fbf [Forbidden Donut] BUG: Fix read of py3 PeriodIndex DataFrame HDF made in py2 (#16781) (#16790)
8f8e3d6 [ri938] TST: register slow marker (#16797)
0645868 [ri938] Add backticks in documentation
0a20024 [ri938] Minor correction to previous submit
69454ec [ri938] Minor corrections to previous submit (#16820)
3092bbc [ri938] BUG: reindex would throw when a categorical index was empty #16770

* BUG: Don't with empty Series for .isin (#17006)

Empty Series initializes to float64, even when the data type is object for .isin,
leading to an error with membership.

Closes gh-16991.

* ENH: Use 'Y' as an alias for end of year (#16978)

Closes gh-9313
Redo of gh-16958

* DOC: infer_objects doc fixup (#17018)

* Fixes SparseSeries initiated with dictionary raising AttributeError (#16960)

* DOC: Improving docstring of reset_index method (#16416) (#16975)

* DOC: add warning to append about inefficiency (#17017)

* DOC : Remove redundant backtick (#17025)

* DOC: Document business frequency aliases (#17028)

Follow-up to gh-16978.

* DOC: Fix double back-tick in 'Reshaping by Melt' section (#17030)

See current stable docs for the issue: https://pandas.pydata.org/pandas-docs/stable/reshaping.html#reshaping-by-melt

The double ` is causing the entire paragraph to be fixed width until the next double `. This commit removes the extra "`"

* Define DataFrame plot methods in DataFrame (#17020)

* CLN: move safe_sort from core.algorithms to core.sorting (#17034)

COMPAT: safe_sort will only coerce list-likes to object, not a numpy string type

xref: pandas-dev#17003 (comment)

* DOC: Fixed Minor Typo (#17043)

Cocumentation to Documentation

* BUG: do not cast ints to floats if inputs o crosstab are not aligned (#17011)

closes #17005

* BUG in merging categorical dates

closes #16900

Author: Dave Willmer <dave.willmer@gmail.com>

This patch had conflicts when merged, resolved by
Committer: Jeff Reback <jeff@reback.net>

Closes #16986 from dwillmer/cat_fix and squashes the following commits:

1ea1977 [Dave Willmer] Minor tweaks + comment
21a35a0 [Dave Willmer] Merge branch 'cat_fix' of https://github.com/dwillmer/pandas into cat_fix
04d5404 [Dave Willmer] Update tests
3cc5c24 [Dave Willmer] Merge branch 'master' into cat_fix
5e8e23b [Dave Willmer] Add whatsnew item
b82d117 [Dave Willmer] Lint fixes
a81933d [Dave Willmer] Remove unused import
218da66 [Dave Willmer] Generic solution to categorical problem
48e7163 [Dave Willmer] Test inner join
8843c10 [Dave Willmer] Fix TypeError when merging categorical dates

* BUG: __setitem__ with a tuple induces NaN with a tz-aware DatetimeIndex (#16889) (#16897)

* Added test for _get_dtype_type. (#16899)

* BUG/API: dtype inconsistencies in .where / .setitem / .putmask / .fillna (#16821)

* CLN/BUG: fix ndarray assignment may cause unexpected cast

supersedes #14145
closes #14001

* API: This fixes a number of inconsistencies and API issues
w.r.t. dtype conversions.

This is a reprise of #14145 & #16408.

This removes some code from the core structures & pushes it to internals,
where the primitives are made more consistent.

This should all us to be a bit more consistent for pandas2 type things.

closes #16402
supersedes #14145
closes #14001

CLN: remove uneeded code in internals; use split_and_operate when possible

* BUG: Improved thread safety for read_html() GH16928 (#16930)

* Fixed 'add_methods' when the 'select' argument is specified. (#17045)

* TST: Fix error message check in np.argsort comparision (#17051)

Closes gh-17046.

* TST: Move some Series ctor tests to SharedWithSparse (#17050)

* BUG: Made SparseDataFrame.fillna() fill all NaNs

A continuation of pandas-dev#16178
closes #16112
closes #16178

Author: Kernc <kerncece@gmail.com>
Author: keitakurita <kris337jbn@yahoo.co.jp>

This patch had conflicts when merged, resolved by
Committer: Jeff Reback <jeff@reback.net>

Closes #16892 from kernc/sparse-fillna and squashes the following commits:

c1cd33e [Kernc] fixup! BUG: Made SparseDataFrame.fillna() fill all NaNs
2974232 [Kernc] fixup! BUG: Made SparseDataFrame.fillna() fill all NaNs
4bc01a1 [keitakurita] BUG: Made SparseDataFrame.fillna() fill all NaNs

* BUG: Use size_t to avoid array index overflow; add missing malloc of error_msg

Fix a few locations where a parser's `error_msg` buffer is written to
without having been previously allocated. This manifested as a double
free during exception handling code making use of the `error_msg`.
Additionally, use `size_t/ssize_t` where array indices or lengths will
be stored. Previously, int32_t was used and would overflow on columns
with very large amounts of data (i.e. greater than INTMAX bytes).

xref #14696
closes #16798

Author: Jeff Knupp <jeff.knupp@enigma.com>
Author: Jeff Knupp <jeff@jeffknupp.com>

Closes #17040 from jeffknupp/16790-core-on-large-csv and squashes the following commits:

6a1ba23 [Jeff Knupp] Clear up prose
a5d5677 [Jeff Knupp] Fix linting issues
4380c53 [Jeff Knupp] Fix linting issues
7b1cd8d [Jeff Knupp] Fix linting issues
e3cb9c1 [Jeff Knupp] Add unit test plus '--high-memory' option, *off by default*.
2ab4971 [Jeff Knupp] Remove debugging code
2930eaa [Jeff Knupp] Fix line length to conform to linter rules
e4dfd19 [Jeff Knupp] Revert printf format strings; fix more comment alignment
3171674 [Jeff Knupp] Fix some leftover size_t references
0985cf3 [Jeff Knupp] Remove debugging code; fix type cast
669d99b [Jeff Knupp] Fix linting errors re: line length
1f24847 [Jeff Knupp] Fix comment alignment; add whatsnew entry
e04d12a [Jeff Knupp] Switch to use int64_t rather than size_t due to portability concerns.
d5c75e8 [Jeff Knupp] BUG: Use size_t to avoid array index overflow; add missing malloc of error_msg

* TST: remove some test warnings in parser tests (#17057)

TST: move highmemory test to proper location in c_parser_only

xref #16798

* DOC: Add more examples for reset_index (#17055)

* MAINT: Add dash in high memory message

Follow-up to gh-17057.

* MAINT: kwards --> kwargs in parsers.pyx

* CLN: Cleanup comments in before_install_travis.sh

envars.sh doesn't exist anymore.  In fact, it's been gone for awhile.

* MAINT: Remove duplicate Series sort_index check

Duplicate boolean validation check for sort_index in series/test_validate.py

* BLD: Pin pyarrow=0.4.1 (#17065)

Addresses gh-17064.

Also add some additional build information when calling `pd.show_versions`

* ENH: provide "inplace" argument to set_axis()

closes #14636

Author: Pietro Battiston <me@pietrobattiston.it>

Closes #16994 from toobaz/set_axis_inplace and squashes the following commits:

8fb9d0f [Pietro Battiston] REF: adapt NDFrame.set_axis() calls to new signature
409f502 [Pietro Battiston] ENH: provide "inplace" argument to set_axis(), change signature

* BUG: Fix parser field type compatability on 32-bit systems. (#17071)

Closes gh-17063

* COMPAT: rename isnull -> isna, notnull -> notna (#16972)

closes #15001

* BUG: Thoroughly dedup columns in read_csv (#17060)

* ENH: Add skipna parameter to infer_dtype (#17066)

Currently defaults to False for backwards compatibility.  Will default to True in the future.

Closes gh-17059.

* MAINT: Remove unused variable in test_scalar.py

The "expected" variable is unused at the end of a test in indexing/test_scalar.py

* TST: Add tests/indexing/ and reshape/ to setup.py (#17076)

Looks like we just forgot about them.  Oops.

* CI: partially revert #17065, un-pin pyarrow on some builds

* DOC: whatsnew typos

* TST: Check more error messages in tests (#17075)

* BUG: Respect dtype when calling pivot_table with margins=True

closes #17013

This fix actually exposed an occurrence of #17035 in an existing test
(as well as in one I added).

Author: Pietro Battiston <me@pietrobattiston.it>

Closes #17062 from toobaz/pivot_margin_int and squashes the following commits:

2737600 [Pietro Battiston] Removed now obsolete workaround
956c4f9 [Pietro Battiston] BUG: respect dtype when calling pivot_table with margins=True

* MAINT: Add missing space in parsers.pyx

"2< heuristic" --> "2 < heuristic"

* MAINT: Add missing paren around print statement

Stray verbose print statement in parsers.pyx was bare without any parentheses.

* DOC: fix typos in missing.rst

xref #16972

* DOC: further clean-up null/na changes (#17113)

* BUG: Allow pd.unique to accept tuple of strings (#17108)

* BUG: Allow Series with same name with crosstab (#16028)

Closes gh-13279

* COMPAT: make sure use_inf_as_null is deprecated (#17126)

closes #17115

* CI: bump version of xlsxwriter to 0.5.2 (#17142)

* DOC: Clean up instructions in ISSUE_TEMPLATE (#17146)

* Add missing space to the NotImplementedError's message for compound dtypes (#17140)

* DOC: (de)type the return value of concat (#17079) (#17119)

* BUG: Thoroughly dedup column names in read_csv (#17095)

* DOC: Additions/updates to documentation (#17150)

* ENH: add to/from_parquet with pyarrow & fastparquet (#15838)

* DOC: doc typos, xref #15838

* TST: test for categorical index monotonicity (#17152)

* correctly determine bottleneck version

* tests for categorical index monotonicity

* fix Index.is_monotonic to point to Index.is_monotonic_increasing directly

* MAINT: Remove non-standard and inconsistently-used imports (#17085)

* DOC: typos in whatsnew

* DOC: whatsnew 0.21.0 fixes

* BUG: Fix CSV parsing of singleton list header (#17090)

Closes gh-7757.

* ENH: Support strings containing '%' in add_prefix/add_suffix (#17151) (#17162)

* REF: repr - allow block to override values that get formatted (#17143)

* MAINT: Drop unnecessary newlines in issue template

* remove direct import of nan

Author: Brock Mendel <jbrockmendel@gmail.com>

Closes #17185 from jbrockmendel/dont_import_nan and squashes the following commits:

ee260b8 [Brock Mendel] remove direct import of nan

* use == to test String equality (#17171)

* ENH: Add warning when setting into nonexistent attribute (#16951)

 closes #7175
 closes #5904

* DOC: added string processing comparison with SAS  (#16497)

* CLN: remove unused get methods in internals (#17169)

* Remove unused get methods that would raise AttributeError if called

* Remove unnecessary import

* TST: Partial Boolean DataFrame Indexing (#17186)

Closes gh-17170

* CLN: Reformat docstring for IPython fixture

* Define Series.plot and Series.hist in class definition (#17199)

* BUG: support pandas objects in iloc with old numpy versions (#17194)

closes #17193

* Implement _make_accessor classmethod for PandasDelegate (#17166)

* Create ABCDateOffset (#17165)

* BUG: resample and apply modify the index type for empty Series (#17149)

* DOC: Updated NDFrame.astype docs (#17203)

* MAINT: Minor touch-ups to GitHub PULL_REQUEST_TEMPLATE (#17207)

Remove leading space from task-list so that tasks aren't nested.

* CLN: replace %s syntax with .format in core.computation (#17209)

* Bugfix for multilevel columns with empty strings in Python 2 (#17099)

* CLN/ASV clean-up frame stat ops benchmarks (#17205)

* BUG: Rolling apply on DataFrame with Datetime index returns NaN (#17156)

* CLN: Remove import exception handling (#17218)

Imports should succeed on all versions of Python that pandas supports.

* MAINT: Remove extra the's in deprecation messages (#17222)

* DOC: Patch docs in _decorators.py

* CLN: replace %s syntax with .format in pandas.util (#17224)

* Add 'See also' sections (#17223)

* move pivot_table doc-string to DataFrame (#17174)

* Remove import of pandas as pd in core.window (#17233)

* TST: Move more frame tests to SharedWithSparse (#17227)

* REF: _get_objs_combined_axis (#17217)

* ENH/PERF: Remove frequency inference from .dt accessor (#17210)

* ENH/PERF: Remove frequency inference from .dt accessor

* BENCH: Add DatetimeAccessor benchmark

* DOC: Whatsnew

* Fix apparent typo in tests (#17247)

* COMPAT: avoid calling getsizeof() on PyPy

closes #17228

Author: mattip <matti.picus@gmail.com>

Closes #17229 from mattip/getsizeof-unavailable and squashes the following commits:

d2623e4 [mattip] COMPAT: avoid calling getsizeof() on PyPy

* CLN: replace %s syntax with .format in pandas.core.reshape (#17252)

Replaced %s syntax with .format in pandas.core.reshape.  Additionally, made some of the existing positional .format code more explicit.

* ENH: Infer compression from non-string paths (#17206)

* Fix bugs in IntervalIndex.is_non_overlapping_monotonic (#17238)

* BUG: Fix behavior of argmax and argmin with inf (#16449) (#16449)

Closes #13595

* CLN: Remove have_pytz (#17266)

Closes gh-17251

* CLN: replace %s syntax with .format in core.dtypes and core.sparse (#17270)

* Replace imports of * with explicit imports (#17269)

xref #17234

* TST: pytest deprecation warnings GH17197 (#17253)

Test parameters with marks are updated according to the updated API of
Pytest.
https://docs.pytest.org/en/latest/changelog.html#pytest-3-2-0-2017-07-30
https://docs.pytest.org/en/latest/parametrize.html

* Handle more date/datetime/time formats (#15871)

* DOC: add example on json_normalize (#16438)

* BUG: Have object dtype for empty Categorical.categories (#17249)

* BUG: Have object dtype for empty Categorical ctor

Previously we had a `Float64Index`, which is inconsistent with, e.g., the
regular Index constructor.

* TST: Update tests in multi for new return

Previously these relied worked around the return type by wrapping list-likes
in `np.array` and relying on that to cast to float. These workarounds are no
longer nescessary.

* TST: Update union_categorical tests

This relied on `NaN` being a float and empty being a float. Not a necessary
test anymore.

* TST: set object dtype

* CLN: replace %s syntax with .format in pandas.tseries (#17290)

* TST: parameterize consistency tests for rolling/expanding windows (#17292)

* FIX: define `DataFrame.items` for all versions of python (#17214)

* PERF: Update ASV publish config (#17293)

Stricter cutoffs for considering regressions

[ci skip]

* DOC: Expand docstrings for head / tail methods (#16941)

* MAINT: Use set literal for unsupported + depr args

Initializes unsupported and deprecated argument sets with set literals instead of the set constructor in pandas/io/parsers.py, as the former is slightly faster than the latter.

* DOC: Add proper docstring to maybe_convert_indices

Patches several spelling errors and expands current doc to a proper doc-string.

* DOC: Improving docstring of take method (#16948)

* BUG: Fixed regex in asv.conf.json (#17300)

In pandas-dev#17293 I messed up the syntax. I
used a glob instead of a regex. According to the docs at
http://asv.readthedocs.io/en/latest/asv.conf.json.html#regressions-thresholds we
want to use a regex. I've actually manually tested this change and verified that
it works.

[ci skip]

* Remove unnecessary usage of _TSObject (#17297)

* BUG: clip should handle null values

closes #17276

Author: Michael Gasvoda <mgasvoda@mercatus.gmu.edu>
Author: mgasvoda <mgasvoda01@gmail.com>

Closes #17288 from mgasvoda/master and squashes the following commits:

a1dbdf2 [mgasvoda] Merge branch 'master' into master
9333952 [Michael Gasvoda] Checking output of tests
4e0464e [Michael Gasvoda] fixing whatsnew text
c442040 [Michael Gasvoda] formatting fixes
7e23678 [Michael Gasvoda] formatting updates
781ea72 [Michael Gasvoda] whatsnew entry
d9627fe [Michael Gasvoda] adding clip tests
9aa0159 [Michael Gasvoda] Treating na values as none for clips

* BUG: fillna returns frame when inplace=True if value is a dict (#16156) (#17279)

* CLN: Index.append() refactoring (#16236)

* DEPS: set min versions (#17002)

closes #15206, numpy >= 1.9
closes #15543, matplotlib >= 1.4.3
scipy >= 0.14.0

* CLN: replace %s syntax with .format in core.tools, algorithms.py, base.py (#17305)

* BUG: Fix strange behaviour of Series.iloc on MultiIndex Series (#17148) (#17291)

* DOC: Add module doc-string to tseries/api.py

* MAINT: Clean up docs in pandas/errors/__init__.py

* CLN: replace %s syntax with .format in missing.py, nanops.py, ops.py (#17322)

Replaced %s syntax with .format in missing.py, nanops.py, ops.py. Additionally, made some of the existing positional .format code more explicit.

* Make pd.Period immutable (#17239)

* Bug: groupby multiindex levels equals rows (#16859)

closes #16843

* BUG: Cannot use tz-aware origin in to_datetime (#16842)

closes #16842

Author: step4me <prosikeffect@gmail.com>

Closes #17244 from step4me/step4me-feature and squashes the following commits:

09d051d [step4me] BUG: Cannot use tz-aware origin in to_datetime (#16842)

* Replace usage of total_seconds compat func with timedelta method (#17289)

* CLN: replace %s syntax with .format in core/indexing.py (#17357)

Progress toward issue #16130. Converted old string formatting to new string formatting in core/indexing.py.

* DOC: Point to dev-docs in issue template (#17353)

[ci skip]

* CLN: remove total_seconds compat from json (#17341)

* CLN: Move test_intersect_str_dates (#17366)

Moves test_intersect_str_dates from tests/indexes/test_range.py to tests/indexes/test_base.py.

* BUG: Respect dups in reindexing CategoricalIndex (#17355)

When the indexer is identical to the elements.
We should still return duplicates when the indexer
contains duplicates.

Closes gh-17323.

* Unify Index._dir_* with Series implementation (#17117)

* BUG: make order of index from pd.concat deterministic (#17364)

closes #17344

* Fix typo that causes several NaT methods to have incorrect docstrings (#17327)

* CLN: replace %s syntax with .format in io/formats/format.py (#17358)

Progress toward issue #16130. Converted old string formatting to new string formatting in io/formats/format.py.

* PKG: Added pyproject.toml for PEP 518 (#16745)

Declaring build-time requirements: https://www.python.org/dev/peps/pep-0518/

* DOC: Update Overview page in documentation (#17368)

* Update Overview page in documentation

* DOC Revise Overview page

* DOC Make further revisions in Overview webpage

* Update overview.rst

Remove references to Panel

* API: Have MultiIndex consturctors always return a MI (#17236)

* API: Have MultiIndex constructors return MI

This removes the special case for MultiIndex constructors returning
an Index if all the levels are length-1. Now this will return a
MultiIndex with a single level.

This is a backwards incompatabile change, with no clear method for
deprecation, so we're making a clean break.

Closes #17178

* fixup! API: Have MultiIndex constructors return MI

* Update for comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment