Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: support Styler in ExcelFormatter #15530

Closed
wants to merge 53 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
ada5101
ENH: support Styler in ExcelFormatter
jnothman Feb 28, 2017
f1cde08
FIX column offset incorrect in refactor
jnothman Feb 28, 2017
96680f9
Largely complete CSSToExcelConverter and Styler.to_excel()
jnothman Apr 5, 2017
8780076
Merge branch 'master' into excel_style
jnothman Apr 5, 2017
0ce72f9
Use inherited font size for em_pt
jnothman Apr 5, 2017
cb5cf02
Fix bug where inherited not being passed; avoid classmethods
jnothman Apr 5, 2017
c589c35
Fix some lint errors (yes, the code needs testing)
jnothman Apr 5, 2017
176e51c
Fix NameError
jnothman Apr 5, 2017
d103f61
Refactoring and initial tests for CSS to Excel
jnothman Apr 5, 2017
7db59c0
Test inherited styles in converter
jnothman Apr 5, 2017
dc953d4
Font size and border width
jnothman Apr 5, 2017
f62f02d
File restructure
jnothman Apr 5, 2017
3b26087
Make get_level_lengths non-private
jnothman Apr 6, 2017
eb02cc1
Fix testing ImportError
jnothman Apr 6, 2017
1984cab
Fix making get_level_lengths non-private
jnothman Apr 6, 2017
9a5b791
Fix testing ImportError
jnothman Apr 6, 2017
1a8818f
Lint
jnothman Apr 6, 2017
f17a0f4
Some border style tests
jnothman Apr 6, 2017
efce9b6
More CSS to Excel testing; define ExcelFormatter.write
jnothman Apr 7, 2017
350eab5
remove spurious blank line
jnothman Apr 7, 2017
306eebe
Module-level docstring
jnothman Apr 8, 2017
a1127f6
Merge branch 'master' into excel_style
jnothman Apr 8, 2017
a43d6b7
Cleaner imports
jnothman Apr 8, 2017
c1fc232
Remove debugging print statements
jnothman Apr 8, 2017
8e9a567
Fixes from integration testing
jnothman Apr 8, 2017
7c54a69
Fix test failures; avoid hair border which renders strangely
jnothman Apr 8, 2017
9a62699
Fix tests and add TODOs to tests
jnothman Apr 8, 2017
433be03
Documentation
jnothman Apr 8, 2017
096f26c
Merge remote-tracking branch 'upstream/master' into excel_style
jnothman Apr 9, 2017
b1d774b
What's new heading
jnothman Apr 9, 2017
2c3d015
Fix JSON syntax in IPynb
jnothman Apr 9, 2017
c4f59c6
Doc tweaks
jnothman Apr 9, 2017
79eae41
Documentation tweaks
jnothman Apr 9, 2017
6d3ffc6
Lint
jnothman Apr 9, 2017
6ff8a46
Fix loose character; sorry
jnothman Apr 9, 2017
61fdc69
Complete testing basic CSS -> Excel conversions
jnothman Apr 9, 2017
d144fdf
Font name strings
jnothman Apr 9, 2017
4e72993
Fix what's new heading
jnothman Apr 9, 2017
60d6a3b
add doc/source/styled.xlsx to the gitignore
jnothman Apr 9, 2017
0256fc6
Return after unhandled font size warning
jnothman Apr 9, 2017
d5db0ac
Remove obsolete TODO
jnothman Apr 9, 2017
e2cfa77
Test Styler.to_excel
jnothman Apr 19, 2017
ceb9171
reasons for xfails
jnothman Apr 19, 2017
3071bac
Complete tests
jnothman Apr 19, 2017
14035c5
Merge branch 'master' into excel_style
jnothman Apr 19, 2017
9669d7d
Require jinja in test with df.style
jnothman Apr 19, 2017
6168765
Recommended changes to what's new
jnothman Apr 19, 2017
6465913
More pytest-like test_styler_to_excel; enhancements to xlwt
jnothman Apr 19, 2017
934df06
Display df, not styled
jnothman Apr 19, 2017
a5d51f9
Merge branch 'master' into excel_style
jnothman Apr 19, 2017
de53808
Remove debug code
jnothman Apr 19, 2017
836f39e
Revert changes to xlwt
jnothman Apr 19, 2017
c7a51ca
Test currently fails on openpyxl1 due to version incompatibilities
jnothman Apr 20, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -103,4 +103,5 @@ doc/source/index.rst
doc/build/html/index.html
# Windows specific leftover:
doc/tmp.sv
doc/source/styled.xlsx
doc/source/templates/
Binary file added doc/source/_static/style-excel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
77 changes: 56 additions & 21 deletions doc/source/style.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# HTML Styling\n",
"# Styling\n",
"\n",
"*New in version 0.17.1*\n",
"\n",
"<p style=\"color: red\">*Provisional: This is a new feature and still under development. We'll be adding features and possibly making breaking changes in future releases. We'd love to hear your feedback.*<p style=\"color: red\">\n",
"<span style=\"color: red\">*Provisional: This is a new feature and still under development. We'll be adding features and possibly making breaking changes in future releases. We'd love to hear your feedback.*</span>\n",
"\n",
"This document is written as a Jupyter Notebook, and can be viewed or downloaded [here](http://nbviewer.ipython.org/github/pandas-dev/pandas/blob/master/doc/source/html-styling.ipynb).\n",
"\n",
Expand Down Expand Up @@ -49,7 +49,6 @@
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
Expand All @@ -62,9 +61,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about these. I can try to fix up later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a big deal, I think I have another PR in the works that does the same thing.

"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
Expand Down Expand Up @@ -130,9 +127,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"def color_negative_red(val):\n",
Expand Down Expand Up @@ -186,9 +181,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"def highlight_max(s):\n",
Expand Down Expand Up @@ -240,7 +233,7 @@
"source": [
"Above we used `Styler.apply` to pass in each column one at a time.\n",
"\n",
"<p style=\"background-color: #DEDEBE\">*Debugging Tip*: If you're having trouble writing your style function, try just passing it into <code style=\"background-color: #DEDEBE\">DataFrame.apply</code>. Internally, <code style=\"background-color: #DEDEBE\">Styler.apply</code> uses <code style=\"background-color: #DEDEBE\">DataFrame.apply</code> so the result should be the same.</p>\n",
"<span style=\"background-color: #DEDEBE\">*Debugging Tip*: If you're having trouble writing your style function, try just passing it into <code style=\"background-color: #DEDEBE\">DataFrame.apply</code>. Internally, <code style=\"background-color: #DEDEBE\">Styler.apply</code> uses <code style=\"background-color: #DEDEBE\">DataFrame.apply</code> so the result should be the same.</span>\n",
"\n",
"What if you wanted to highlight just the maximum value in the entire table?\n",
"Use `.apply(function, axis=None)` to indicate that your function wants the entire table, not one column or row at a time. Let's try that next.\n",
Expand All @@ -251,9 +244,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"def highlight_max(data, color='yellow'):\n",
Expand Down Expand Up @@ -819,9 +810,7 @@
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"metadata": {},
"outputs": [],
"source": [
"def magnify():\n",
Expand Down Expand Up @@ -854,6 +843,53 @@
" .set_table_styles(magnify())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Export to Excel\n",
"\n",
"*New in version 0.20.0*\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might want to add an Experimental statement somehwere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one right below, isn't there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one right below, isn't there?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh i c, ok then.

"\n",
"<span style=\"color: red\">*Experimental: This is a new feature and still under development. We'll be adding features and possibly making breaking changes in future releases. We'd love to hear your feedback.*</span>\n",
"\n",
"Some support is available for exporting styled `DataFrames` to Excel worksheets using the `OpenPyXL` engine. CSS2.2 properties handled include:\n",
"\n",
"- `background-color`\n",
"- `border-style`, `border-width`, `border-color` and their {`top`, `right`, `bottom`, `left` variants}\n",
"- `color`\n",
"- `font-family`\n",
"- `font-style`\n",
"- `font-weight`\n",
"- `text-align`\n",
"- `text-decoration`\n",
"- `vertical-align`\n",
"- `white-space: nowrap`\n",
"\n",
"Only CSS2 named colors and hex colors of the form `#rgb` or `#rrggbb` are currently supported."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.style.\\\n",
" applymap(color_negative_red).\\\n",
" apply(highlight_max).\\\n",
" to_excel('styled.xlsx', engine='openpyxl')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A screenshot of the output:\n",
"\n",
"![Excel spreadsheet with styled DataFrame](_static/style-excel.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -1039,8 +1075,7 @@
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
Expand Down
34 changes: 34 additions & 0 deletions doc/source/whatsnew/v0.20.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Highlights include:
- Improved user API when accessing levels in ``.groupby()``, see :ref:`here <whatsnew_0200.enhancements.groupby_access>`
- Improved support for ``UInt64`` dtypes, see :ref:`here <whatsnew_0200.enhancements.uint64_support>`
- A new orient for JSON serialization, ``orient='table'``, that uses the :ref:`Table Schema spec <whatsnew_0200.enhancements.table_schema>`
- Experimental support for exporting ``DataFrame.style`` formats to Excel , see :ref:`here <whatsnew_0200.enhancements.style_excel>`
- Window Binary Corr/Cov operations now return a MultiIndexed ``DataFrame`` rather than a ``Panel``, as ``Panel`` is now deprecated, see :ref:`here <whatsnew_0200.api_breaking.rolling_pairwise>`
- Support for S3 handling now uses ``s3fs``, see :ref:`here <whatsnew_0200.api_breaking.s3>`
- Google BigQuery support now uses the ``pandas-gbq`` library, see :ref:`here <whatsnew_0200.api_breaking.gbq>`
Expand Down Expand Up @@ -398,6 +399,39 @@ To convert a ``SparseDataFrame`` back to sparse SciPy matrix in COO format, you

sdf.to_coo()

.. _whatsnew_0200.enhancements.style_excel:

Excel output for styled DataFrames
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Experimental support has been added to export ``DataFrame.style`` formats to Excel using the ``openpyxl`` engine. (:issue:`15530`)

For example, after running the following, ``styled.xlsx`` renders as below:

.. ipython:: python

np.random.seed(24)
df = pd.DataFrame({'A': np.linspace(1, 10, 10)})
df = pd.concat([df, pd.DataFrame(np.random.RandomState(24).randn(10, 4),
columns=list('BCDE'))],
axis=1)
df.iloc[0, 2] = np.nan
df
styled = df.style.\
applymap(lambda val: 'color: %s' % 'red' if val < 0 else 'black').\
apply(lambda s: ['background-color: yellow' if v else ''
for v in s == s.max()])
styled.to_excel('styled.xlsx', engine='openpyxl')

.. image:: _static/style-excel.png

.. ipython:: python
:suppress:
import os
os.remove('styled.xlsx')

See the :ref:`Style documentation <style>` for more detail.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ref isn't working. @TomAugspurger, I assume this is related to the nbsphinx changes. There is another ref to style in 0.17.1


.. _whatsnew_0200.enhancements.intervalindex:

IntervalIndex
Expand Down
33 changes: 11 additions & 22 deletions pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -1419,28 +1419,17 @@ def to_excel(self, excel_writer, sheet_name='Sheet1', na_rep='',
index_label=None, startrow=0, startcol=0, engine=None,
merge_cells=True, encoding=None, inf_rep='inf', verbose=True,
freeze_panes=None):
from pandas.io.excel import ExcelWriter
need_save = False
if encoding is None:
encoding = 'ascii'

if isinstance(excel_writer, compat.string_types):
excel_writer = ExcelWriter(excel_writer, engine=engine)
need_save = True

formatter = fmt.ExcelFormatter(self, na_rep=na_rep, cols=columns,
header=header,
float_format=float_format, index=index,
index_label=index_label,
merge_cells=merge_cells,
inf_rep=inf_rep)

formatted_cells = formatter.get_formatted_cells()
excel_writer.write_cells(formatted_cells, sheet_name,
startrow=startrow, startcol=startcol,
freeze_panes=freeze_panes)
if need_save:
excel_writer.save()

from pandas.io.formats.excel import ExcelFormatter
formatter = ExcelFormatter(self, na_rep=na_rep, cols=columns,
header=header,
float_format=float_format, index=index,
index_label=index_label,
merge_cells=merge_cells,
inf_rep=inf_rep)
formatter.write(excel_writer, sheet_name=sheet_name, startrow=startrow,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this is nice

startcol=startcol, freeze_panes=freeze_panes,
engine=engine)

def to_stata(self, fname, convert_dates=None, write_index=True,
encoding="latin-1", byteorder=None, time_stamp=None,
Expand Down
44 changes: 44 additions & 0 deletions pandas/io/formats/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# -*- coding: utf-8 -*-
"""
Common helper methods used in different submodules of pandas.io.formats
"""


def get_level_lengths(levels, sentinel=''):
"""For each index in each level the function returns lengths of indexes.

Parameters
----------
levels : list of lists
List of values on for level.
sentinel : string, optional
Value which states that no new index starts on there.

Returns
----------
Returns list of maps. For each level returns map of indexes (key is index
in row and value is length of index).
"""
if len(levels) == 0:
return []

control = [True for x in levels[0]]

result = []
for level in levels:
last_index = 0

lengths = {}
for i, key in enumerate(level):
if control[i] and key == sentinel:
pass
else:
control[i] = False
lengths[last_index] = i - last_index
last_index = i

lengths[last_index] = len(level) - last_index

result.append(lengths)

return result
Loading