ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

feefladder · 2021-06-25T14:00:08Z

closes ENH: #40231 does not allow writing multiple dataframes to a single sheet #42221, related to ENH: Add if_sheet_exists parameter to ExcelWriter #40231
tests added / passed
Ensure all linting tests pass, see here for how to run them
whatsnew entry - not applicable

I mainly wanted this functionality myself, and am very open to input :)

github-actions · 2021-08-12T00:02:51Z

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

feefladder · 2021-08-14T18:06:14Z

This is still under discussion

rhshadrach · 2021-08-17T20:59:10Z

(Most of the) Original discussion: #40231 (comment)
Summary of why this is a regression: #42221 (comment)

As back in #40231, I'm +1 on having this as a feature, and even more so since it was identified that PR actually removed the feature in question.

feefladder · 2021-08-18T08:48:31Z

The original opposition to the 'overwrite'/'write_to': #40231 (comment)
Basically that it is a mutating method that is not provided somewhere else.
But append mode is also only provided here (AFAIK, could be wrong). So you're already mutating the workbook, why would mutating sheets then be a problem.
On top of that, it was possible previously #42221

pandas/io/excel/_base.py

jreback

pls add a release note in 1.4.0 in the enhancements section

pandas/io/excel/_base.py

jreback · 2021-08-31T23:09:35Z

can you rebase

pep8speaks · 2021-09-07T06:20:55Z

Hello @joeperdefloep! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-11-15 15:26:58 UTC

feefladder · 2021-09-07T06:48:19Z

I messed up a bit while rebasing and had to force-push. apologies for that :/

feefladder · 2021-09-07T20:07:39Z

I really do not understand why the test fails. It says it fails doctests in _base.py, without mentioning a reason...

pandas/io/excel/_base.py

rhshadrach · 2021-09-19T22:46:16Z

@joeperdefloep - still some docstring issues on the CI. ping if you're not sure how to fix

/home/runner/work/pandas/pandas/pandas/io/excel/_base.py:docstring of pandas.io.excel._base.ExcelWriter:52: WARNING: Explicit markup ends without a blank line; unexpected unindent.
looking for now-outdated files... none found
/home/runner/work/pandas/pandas/pandas/io/excel/_base.py:docstring of pandas.io.excel._base.ExcelWriter:53: WARNING: Bullet list ends without a blank line; unexpected unindent.

feefladder · 2021-09-20T15:38:34Z

@rhshadrach shuold be good. Small question:
How did you get that output? I tried $ python scripts/validate_docstrings.py pandas.ExcelWriter
but got NameError due to ExcelWriter. Also in #43445.

these errors you see are definitely because of these lines. I wanted to have only the overlay option as a versionadded 1.4.0 But was not sure how to do that. I just made a versionchanged 1.4.0

rhshadrach · 2021-09-23T02:43:53Z

If you click on Details unde the "Some checks were not successful", you can dig into the CI output.

jreback · 2021-10-03T22:52:35Z

@joeperdefloep can you merge master and address test failures

pandas/io/excel/_base.py

pandas/io/excel/_openpyxl.py

pandas/tests/io/excel/test_openpyxl.py

feefladder · 2021-11-05T17:32:21Z

Sorry it took so long.... Done now :)

lithomas1

Can you add a whatsnew note(this is changing API so should be needed)?

doc/source/whatsnew/v1.4.0.rst

pandas/tests/io/excel/test_openpyxl.py

jreback · 2021-11-06T22:50:22Z

pandas/io/excel/_base.py

+    ...     if_sheet_exists="overlay",
+    ... ) as writer:
+    ...     df1.to_excel(writer, sheet_name="Sheet1")
+    ...     df2.to_excel(writer, sheet_name="Sheet1", startcol=3)


does overlay require startrow to be not 1? (e.g should we error if this is not specified)

Without startrow (e.g. startrow=0) it will overwrite the headers. This could be desired behaviour so I think we should not error.

This is what I would expect - unspecified starts at the very top left corner of the sheet.

rhshadrach

lgtm

rhshadrach · 2021-11-13T22:30:26Z

pandas/io/excel/_base.py

+    ...     if_sheet_exists="overlay",
+    ... ) as writer:
+    ...     df1.to_excel(writer, sheet_name="Sheet1")
+    ...     df2.to_excel(writer, sheet_name="Sheet1", startcol=3)


This is what I would expect - unspecified starts at the very top left corner of the sheet.

rhshadrach · 2021-11-13T22:44:07Z

@joeperdefloep - Can you resolve the conflicts.

rhshadrach · 2021-11-16T21:48:29Z

@jreback - friendly ping.

jreback · 2021-11-17T02:02:23Z

thanks @joeperdefloep very nice!

feefladder mentioned this pull request Jun 26, 2021

ENH: #40231 does not allow writing multiple dataframes to a single sheet #42221

Closed

feefladder pushed a commit to feefladder/pandas that referenced this pull request Jun 26, 2021

DOC: added docs to pandas-dev#42222

9e20d68

feefladder pushed a commit to feefladder/pandas that referenced this pull request Jun 26, 2021

DOC: added docs to pandas-dev#42222

c5ff4b1

feefladder force-pushed the write-to branch from 9e20d68 to c5ff4b1 Compare June 26, 2021 12:30

feefladder pushed a commit to feefladder/pandas that referenced this pull request Jun 26, 2021

moved if_sheet_exists docs to pandas-dev#42222

d226ae5

github-actions bot added the Stale label Aug 12, 2021

lithomas1 added Enhancement IO Excel read_excel, to_excel Needs Review Waiting for review/response from a maintainer. and removed Stale labels Aug 16, 2021

lithomas1 requested review from rhshadrach and jreback August 16, 2021 22:19

jreback requested changes Aug 18, 2021

View reviewed changes

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

jreback added this to the 1.4 milestone Aug 21, 2021

jreback requested changes Aug 21, 2021

View reviewed changes

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

rhshadrach requested changes Aug 22, 2021

View reviewed changes

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

rhshadrach mentioned this pull request Aug 31, 2021

ENH: Rename ExcelWriter to make clear attributes are not public #43088

Closed

Joeperdefloep added 4 commits September 7, 2021 08:22

ENH: add if_sheet_exists='write_to' to ExcelWriter

bc6534a

DOC: added docs to pandas-dev#42222

66fcd15

resolved comments

cb59157

versionadded, fix tests

b9ed327

feefladder force-pushed the write-to branch from 9205e98 to b9ed327 Compare September 7, 2021 06:23

renamed overwrite_cells to overlay

211683d

rhshadrach requested changes Sep 7, 2021

View reviewed changes

pandas/io/excel/_base.py Show resolved Hide resolved

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

ENH: GH42222 fixed comments

087274a

jreback requested changes Sep 9, 2021

View reviewed changes

pandas/io/excel/_base.py Show resolved Hide resolved

lithomas1 removed the Needs Review Waiting for review/response from a maintainer. label Sep 10, 2021

jreback changed the title ~~ENH: add if_sheet_exists='write_to' to ExcelWriter~~ ENH: add if_sheet_exists='overlay' to ExcelWriter Sep 10, 2021

ENH: GH42222 versionchanged

9f79b0f

Joeperdefloep and others added 2 commits November 4, 2021 16:24

ENH: GH42222 fixed indent

e7362eb

Merge branch 'pandas-dev:master' into write-to

d92f414

twoertwein requested changes Nov 5, 2021

View reviewed changes

pandas/io/excel/_base.py Show resolved Hide resolved

pandas/io/excel/_base.py Outdated Show resolved Hide resolved

pandas/io/excel/_openpyxl.py Show resolved Hide resolved

pandas/tests/io/excel/test_openpyxl.py Show resolved Hide resolved

ENH: GH 42222 @twoertwein's comments

d0a4565

twoertwein approved these changes Nov 5, 2021

View reviewed changes

feefladder requested review from jreback and rhshadrach November 5, 2021 17:31

lithomas1 requested changes Nov 6, 2021

View reviewed changes

ENH: GH 42222 What's new entry

54e274d

jreback requested changes Nov 6, 2021

View reviewed changes

ENH: GH42222 whatsnew issue number, improved test

aa9afbc

rhshadrach approved these changes Nov 13, 2021

View reviewed changes

Merge branch 'master' into write-to

ba38c31

jreback approved these changes Nov 17, 2021

View reviewed changes

jreback merged commit 80372bb into pandas-dev:master Nov 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

feefladder commented Jun 25, 2021 •

edited

github-actions bot commented Aug 12, 2021

feefladder commented Aug 14, 2021

rhshadrach commented Aug 17, 2021

feefladder commented Aug 18, 2021 •

edited

jreback left a comment

jreback commented Aug 31, 2021

pep8speaks commented Sep 7, 2021 •

edited

feefladder commented Sep 7, 2021

feefladder commented Sep 7, 2021

rhshadrach commented Sep 19, 2021

feefladder commented Sep 20, 2021 •

edited

rhshadrach commented Sep 23, 2021

jreback commented Oct 3, 2021

feefladder commented Nov 5, 2021

lithomas1 left a comment

jreback Nov 6, 2021

feefladder Nov 7, 2021

rhshadrach Nov 13, 2021

rhshadrach left a comment

rhshadrach Nov 13, 2021

rhshadrach commented Nov 13, 2021

rhshadrach commented Nov 16, 2021

jreback commented Nov 17, 2021

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

Conversation

feefladder commented Jun 25, 2021 • edited

github-actions bot commented Aug 12, 2021

feefladder commented Aug 14, 2021

rhshadrach commented Aug 17, 2021

feefladder commented Aug 18, 2021 • edited

jreback left a comment

Choose a reason for hiding this comment

jreback commented Aug 31, 2021

pep8speaks commented Sep 7, 2021 • edited

Comment last updated at 2021-11-15 15:26:58 UTC

feefladder commented Sep 7, 2021

feefladder commented Sep 7, 2021

rhshadrach commented Sep 19, 2021

feefladder commented Sep 20, 2021 • edited

rhshadrach commented Sep 23, 2021

jreback commented Oct 3, 2021

feefladder commented Nov 5, 2021

lithomas1 left a comment

Choose a reason for hiding this comment

jreback Nov 6, 2021

Choose a reason for hiding this comment

feefladder Nov 7, 2021

Choose a reason for hiding this comment

rhshadrach Nov 13, 2021

Choose a reason for hiding this comment

rhshadrach left a comment

Choose a reason for hiding this comment

rhshadrach Nov 13, 2021

Choose a reason for hiding this comment

rhshadrach commented Nov 13, 2021

rhshadrach commented Nov 16, 2021

jreback commented Nov 17, 2021

feefladder commented Jun 25, 2021 •

edited

feefladder commented Aug 18, 2021 •

edited

pep8speaks commented Sep 7, 2021 •

edited

feefladder commented Sep 20, 2021 •

edited