Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add if_sheet_exists='overlay' to ExcelWriter #42222

Merged
merged 14 commits into from Nov 17, 2021
32 changes: 28 additions & 4 deletions pandas/io/excel/_base.py
Expand Up @@ -689,14 +689,16 @@ class ExcelWriter(metaclass=abc.ABCMeta):
host, port, username, password, etc., if using a URL that will
be parsed by ``fsspec``, e.g., starting "s3://", "gcs://".

.. versionadded:: 1.2.0
if_sheet_exists : {'error', 'new', 'replace'}, default 'error'
.. versionchanged:: 1.4.0
if_sheet_exists : {'error', 'new', 'replace', 'overlay'}, default 'error'
twoertwein marked this conversation as resolved.
Show resolved Hide resolved
How to behave when trying to write to a sheet that already
exists (append mode only).

* error: raise a ValueError.
* new: Create a new sheet, with a name determined by the engine.
* replace: Delete the contents of the sheet before writing to it.
* overlay: Write contents to the existing sheet without removing the old
contents.

.. versionadded:: 1.3.0
engine_kwargs : dict, optional
Expand Down Expand Up @@ -764,6 +766,28 @@ class ExcelWriter(metaclass=abc.ABCMeta):
>>> with pd.ExcelWriter("path_to_file.xlsx", mode="a", engine="openpyxl") as writer:
... df.to_excel(writer, sheet_name="Sheet3")

Here, the `if_sheet_exists` parameter can be set to replace a sheet if it
feefladder marked this conversation as resolved.
Show resolved Hide resolved
already exists:
feefladder marked this conversation as resolved.
Show resolved Hide resolved

>>> with ExcelWriter(
... "path_to_file.xlsx",
... mode="a",
... engine="openpyxl",
... if_sheet_exists="replace",
... ) as writer:
... df.to_excel(writer, sheet_name="Sheet1")

You can also write multiple DataFrames to a single sheet. Note that the
``if_sheet_exists`` parameter needs to be set to ``overlay``:

>>> with ExcelWriter("path_to_file.xlsx",
... mode="a",
... engine="openpyxl",
... if_sheet_exists="overlay",
... ) as writer:
... df1.to_excel(writer, sheet_name="Sheet1")
... df2.to_excel(writer, sheet_name="Sheet1", startcol=3)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does overlay require startrow to be not 1? (e.g should we error if this is not specified)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without startrow (e.g. startrow=0) it will overwrite the headers. This could be desired behaviour so I think we should not error.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I would expect - unspecified starts at the very top left corner of the sheet.


You can store Excel file in RAM:

>>> import io
Expand Down Expand Up @@ -949,10 +973,10 @@ def __init__(

self.mode = mode

if if_sheet_exists not in [None, "error", "new", "replace"]:
if if_sheet_exists not in [None, "error", "new", "replace", "overlay"]:
twoertwein marked this conversation as resolved.
Show resolved Hide resolved
raise ValueError(
f"'{if_sheet_exists}' is not valid for if_sheet_exists. "
"Valid options are 'error', 'new' and 'replace'."
"Valid options are 'error', 'new', 'replace' and 'overlay'."
)
if if_sheet_exists and "r+" not in mode:
raise ValueError("if_sheet_exists is only valid in append mode (mode='a')")
Expand Down
4 changes: 3 additions & 1 deletion pandas/io/excel/_openpyxl.py
Expand Up @@ -437,10 +437,12 @@ def write_cells(
f"Sheet '{sheet_name}' already exists and "
f"if_sheet_exists is set to 'error'."
)
elif self.if_sheet_exists == "overlay":
wks = self.sheets[sheet_name]
twoertwein marked this conversation as resolved.
Show resolved Hide resolved
else:
raise ValueError(
f"'{self.if_sheet_exists}' is not valid for if_sheet_exists. "
"Valid options are 'error', 'new' and 'replace'."
"Valid options are 'error', 'new', 'replace' and 'overlay'."
)
else:
wks = self.sheets[sheet_name]
Expand Down
3 changes: 2 additions & 1 deletion pandas/tests/io/excel/test_openpyxl.py
Expand Up @@ -139,6 +139,7 @@ def test_write_append_mode(ext, mode, expected):
[
("new", 2, ["apple", "banana"]),
("replace", 1, ["pear"]),
("overlay", 1, ["pear", "banana"]),
feefladder marked this conversation as resolved.
Show resolved Hide resolved
],
)
def test_if_sheet_exists_append_modes(ext, if_sheet_exists, num_sheets, expected):
Expand Down Expand Up @@ -170,7 +171,7 @@ def test_if_sheet_exists_append_modes(ext, if_sheet_exists, num_sheets, expected
(
"invalid",
"'invalid' is not valid for if_sheet_exists. Valid options "
"are 'error', 'new' and 'replace'.",
"are 'error', 'new', 'replace' and 'overlay'.",
),
(
"error",
Expand Down