-
-
Notifications
You must be signed in to change notification settings - Fork 18k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: support Styler in ExcelFormatter #15530
Conversation
pandas/formats/format.py
Outdated
@@ -1902,11 +1907,26 @@ def _format_hierarchical_rows(self): | |||
indexcolval, header_style) | |||
gcolidx += 1 | |||
|
|||
for cell in self._generate_body(coloffset=gcolidx): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And if uninterested in merging the whole patch, could you consider merging just this refactoring, so that it's easier to extend ExcelFormatter
?
is this intended to address #1663? |
I suppose so. And it seems I hacked a bit quickly and need to fix some test failures. |
I couldn't tell whether #1663 had anything to do with |
also have a look at comments #7565 |
I think I've got the gist. Obviously this builds on that support together with |
Codecov Report
@@ Coverage Diff @@
## master #15530 +/- ##
==========================================
+ Coverage 90.79% 90.82% +0.03%
==========================================
Files 156 159 +3
Lines 50534 50794 +260
==========================================
+ Hits 45883 46135 +252
- Misses 4651 4659 +8
Continue to review full report at Codecov.
|
can you show a picture of what this does? e.g. the excel picture. |
While I like the idea of using I would prefer it if a default |
okay. I was hoping to pose this as an intermediate step, seeing as we can
surely only support a subset of CSS, ideally corresponding to what is
currently generated through Styler's declarative API. (There are many ways
to specify border styles, colours, etc in CSS.) I am not sure of the time
frame in which I can make a solid attempt at that. I see that providing a
default converter has the advantage of not exposing the internal excel
styling language to users.
Yes, @jreback there seem to be problems with rendering. Will get back to
you shortly.
…On 1 Mar 2017 8:07 am, "chris-b1" ***@***.***> wrote:
While I like the idea of using Styler to define Excel styles, I'm not
sure about this api. I do understand that this would be intended to
advanced / experimental.
I would prefer it if a default style_converter was provided, and
extensible in some way. Handle simple things like background color, text
color, etc by default.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#15530 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz63qaoL5UHJpxwZYSryYveorRv4zHks5rhIyWgaJpZM4MORU3>
.
|
Sorted. I needed a |
I have added writing to the test in the PR description above... with paths in /tmp/ so not actually to be used in a test. Output is as follows: @chris-b1, in the PR description I have also drafted a |
Note that openpyxl cannot support conversion of the |
I do have some concern that exporting to HTML and using Excel's import may be appropriate and much simpler in many cases. |
@jnothman would not be a problem to have an optional dep like that |
@jnothman - thanks - a couple thoughts. Your And to be clear, I personally would be fine initially supporting a tiny subset of CSS (even just font color and backround). Could take the dependency if needed, although I'm not sure how much it would really add in this case? |
we de-facto have you might want to define this new style sheet as |
Thanks @jreback for the feedback on API.
I considered this. I mutate in order to take advantage of
I'm not yet certain about the need for a CSS parser, but as is the way of these things, you can implement something that works for the limited cases you've thought of, but then someone will break it, even just by specifying "red" or "hsl(0, 100%, 50%)" or "rgb(255, 0, 0)" or "rgba(255, 0, 0, 100)" instead of "#ff0000", let alone by inserting a comment or unexpected use of whitespace, etc.
Font and background colour seem a good place to start, followed by borders. Ideally we'd support the bar charting provided by |
I've attempted to implement a few handlers. Tricky parsing cases include Tricky not in terms of parsing but in terms of conversion are elements that require state to be stored that is not naturally represented in the Excel style dicts. One (not very useful) example is that |
I have wasted far too much time on this, but I think the principled way to go about CSS to Excel conversion for a list of CSS declarations:
A full CSS interpreter would also deal with matching selectors to contexts and ordering declarations by specificity.
|
status on this @chris-b1 |
@jnothman - thoughts? I'm not opposed to exposing something small / experimental, like your top comment. |
I'll see if i can give this a few minutes today, but I'm generally quite
time poor atm
…On 4 Apr 2017 3:28 am, "chris-b1" ***@***.***> wrote:
@jnothman <https://github.com/jnothman> - thoughts? I'm not opposed to
exposing something small / experimental, like your top comment.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#15530 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz63fcI5va_oL5GpFZwJ5B_srjetU5ks5rsSxMgaJpZM4MORU3>
.
|
@@ -858,6 +847,53 @@ | |||
"cell_type": "markdown", | |||
"metadata": {}, | |||
"source": [ | |||
"## Export to Excel\n", | |||
"\n", | |||
"*New in version 0.20.0*\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you might want to add an Experimental
statement somehwere
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's one right below, isn't there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's one right below, isn't there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh i c, ok then.
@jnothman ok if you'd make those small changes. ping on green. |
There were larger changes that came out of smoke testing xlwt with
test_styler_to_excel. See commit 6465913 (`and False` there is removed later in de53808). Let me know if you'd rather leave xlwt alone.
…On 19 April 2017 at 23:54, Jeff Reback ***@***.***> wrote:
@jnothman <https://github.com/jnothman> ok if you'd make those small
changes. ping on green.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#15530 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz61PxdGhFf61clcdCmSbWlKu048wUks5rxhIEgaJpZM4MORU3>
.
|
I had misunderstood the intention of the reviewer.
thanks, very nice patch!. watch the dev docs: http://pandas-docs.github.io/pandas-docs-travis/ to see what the built looks like; if needed please issue a followup. (this could take an hour or so). |
Thanks @jnothman! |
Oh I hope so! I kept wondering if maybe we should just export HTML and ask Google to convert it to Excel... But I'd much rather this interface, personally! |
Thanks a lot for the reviews and approvals. |
import os | ||
os.remove('styled.xlsx') | ||
|
||
See the :ref:`Style documentation <style>` for more detail. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This ref isn't working. @TomAugspurger, I assume this is related to the nbsphinx changes. There is another ref to style
in 0.17.1
closes pandas-dev#1663 Author: Joel Nothman <joel.nothman@gmail.com> Closes pandas-dev#15530 from jnothman/excel_style and squashes the following commits: c7a51ca [Joel Nothman] Test currently fails on openpyxl1 due to version incompatibilities 836f39e [Joel Nothman] Revert changes to xlwt de53808 [Joel Nothman] Remove debug code a5d51f9 [Joel Nothman] Merge branch 'master' into excel_style 934df06 [Joel Nothman] Display df, not styled 6465913 [Joel Nothman] More pytest-like test_styler_to_excel; enhancements to xlwt 6168765 [Joel Nothman] Recommended changes to what's new 9669d7d [Joel Nothman] Require jinja in test with df.style 14035c5 [Joel Nothman] Merge branch 'master' into excel_style 3071bac [Joel Nothman] Complete tests ceb9171 [Joel Nothman] reasons for xfails e2cfa77 [Joel Nothman] Test Styler.to_excel d5db0ac [Joel Nothman] Remove obsolete TODO 0256fc6 [Joel Nothman] Return after unhandled font size warning 60d6a3b [Joel Nothman] add doc/source/styled.xlsx to the gitignore 4e72993 [Joel Nothman] Fix what's new heading d144fdf [Joel Nothman] Font name strings 61fdc69 [Joel Nothman] Complete testing basic CSS -> Excel conversions 6ff8a46 [Joel Nothman] Fix loose character; sorry 6d3ffc6 [Joel Nothman] Lint 79eae41 [Joel Nothman] Documentation tweaks c4f59c6 [Joel Nothman] Doc tweaks 2c3d015 [Joel Nothman] Fix JSON syntax in IPynb b1d774b [Joel Nothman] What's new heading 096f26c [Joel Nothman] Merge remote-tracking branch 'upstream/master' into excel_style 433be03 [Joel Nothman] Documentation 9a62699 [Joel Nothman] Fix tests and add TODOs to tests 7c54a69 [Joel Nothman] Fix test failures; avoid hair border which renders strangely 8e9a567 [Joel Nothman] Fixes from integration testing c1fc232 [Joel Nothman] Remove debugging print statements a43d6b7 [Joel Nothman] Cleaner imports a1127f6 [Joel Nothman] Merge branch 'master' into excel_style 306eebe [Joel Nothman] Module-level docstring 350eab5 [Joel Nothman] remove spurious blank line efce9b6 [Joel Nothman] More CSS to Excel testing; define ExcelFormatter.write f17a0f4 [Joel Nothman] Some border style tests 1a8818f [Joel Nothman] Lint 9a5b791 [Joel Nothman] Fix testing ImportError 1984cab [Joel Nothman] Fix making get_level_lengths non-private eb02cc1 [Joel Nothman] Fix testing ImportError 3b26087 [Joel Nothman] Make get_level_lengths non-private f62f02d [Joel Nothman] File restructure dc953d4 [Joel Nothman] Font size and border width 7db59c0 [Joel Nothman] Test inherited styles in converter d103f61 [Joel Nothman] Refactoring and initial tests for CSS to Excel 176e51c [Joel Nothman] Fix NameError c589c35 [Joel Nothman] Fix some lint errors (yes, the code needs testing) cb5cf02 [Joel Nothman] Fix bug where inherited not being passed; avoid classmethods 0ce72f9 [Joel Nothman] Use inherited font size for em_pt 8780076 [Joel Nothman] Merge branch 'master' into excel_style 96680f9 [Joel Nothman] Largely complete CSSToExcelConverter and Styler.to_excel() f1cde08 [Joel Nothman] FIX column offset incorrect in refactor ada5101 [Joel Nothman] ENH: support Styler in ExcelFormatter
I'm interested in making a separate package of the CSS handling here, in part so that it can be extended without small changes to Pandas, and in part so that it is available to be used elsewhere. Is there a nice way to avoid synchrony issues? |
css.py and test_css.py are now living (under different names) at https://github.com/jnothman/cssdecl |
what you would do is release a version of this package then in pandas we would do an import of this package and use its api this would have to be for 0.21.0 (which is starting soon actually) |
I don't want to call the package stable enough to import without pinned
versions. Could we just keep a parallel copy but recommend against changes
in Pandas?
…On 26 April 2017 at 23:21, Jeff Reback ***@***.***> wrote:
what you would do is release a version of this package
then in pandas we would do an import of this package and use its api
this would have to be for 0.21.0 (which is starting soon actually)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#15530 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz62KTYiKnjBoyhqi84ePHFefhgOsYks5rz0TogaJpZM4MORU3>
.
|
no that's exactly what u so |
Okay. There's version 0.0.0 released...
https://pypi.org/project/cssdecl/#history
If you'd rather 0.1 I can do that.
…On 27 April 2017 at 02:45, Jeff Reback ***@***.***> wrote:
no that's exactly what u so
release 0.1 and we can pin in pandas for now
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#15530 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAEz6xwKItsO7rG740OMI6tNhnf2uArQks5rz3S2gaJpZM4MORU3>
.
|
I assume you don't want me to include a hard dependency on pypi's cssdecl, but make it an optional dependency like openpyxl is? If so it seems hard to require a particular version when I'm expecting backwards incompatible changes. |
yes normally u start versions > 0 iow 0.1 |
Fyi, I'm having trouble with conda build: https://groups.google.com/a/continuum.io/forum/m/#!topic/conda/yKDGIMZZl2o |
closes pandas-dev#1663 Author: Joel Nothman <joel.nothman@gmail.com> Closes pandas-dev#15530 from jnothman/excel_style and squashes the following commits: c7a51ca [Joel Nothman] Test currently fails on openpyxl1 due to version incompatibilities 836f39e [Joel Nothman] Revert changes to xlwt de53808 [Joel Nothman] Remove debug code a5d51f9 [Joel Nothman] Merge branch 'master' into excel_style 934df06 [Joel Nothman] Display df, not styled 6465913 [Joel Nothman] More pytest-like test_styler_to_excel; enhancements to xlwt 6168765 [Joel Nothman] Recommended changes to what's new 9669d7d [Joel Nothman] Require jinja in test with df.style 14035c5 [Joel Nothman] Merge branch 'master' into excel_style 3071bac [Joel Nothman] Complete tests ceb9171 [Joel Nothman] reasons for xfails e2cfa77 [Joel Nothman] Test Styler.to_excel d5db0ac [Joel Nothman] Remove obsolete TODO 0256fc6 [Joel Nothman] Return after unhandled font size warning 60d6a3b [Joel Nothman] add doc/source/styled.xlsx to the gitignore 4e72993 [Joel Nothman] Fix what's new heading d144fdf [Joel Nothman] Font name strings 61fdc69 [Joel Nothman] Complete testing basic CSS -> Excel conversions 6ff8a46 [Joel Nothman] Fix loose character; sorry 6d3ffc6 [Joel Nothman] Lint 79eae41 [Joel Nothman] Documentation tweaks c4f59c6 [Joel Nothman] Doc tweaks 2c3d015 [Joel Nothman] Fix JSON syntax in IPynb b1d774b [Joel Nothman] What's new heading 096f26c [Joel Nothman] Merge remote-tracking branch 'upstream/master' into excel_style 433be03 [Joel Nothman] Documentation 9a62699 [Joel Nothman] Fix tests and add TODOs to tests 7c54a69 [Joel Nothman] Fix test failures; avoid hair border which renders strangely 8e9a567 [Joel Nothman] Fixes from integration testing c1fc232 [Joel Nothman] Remove debugging print statements a43d6b7 [Joel Nothman] Cleaner imports a1127f6 [Joel Nothman] Merge branch 'master' into excel_style 306eebe [Joel Nothman] Module-level docstring 350eab5 [Joel Nothman] remove spurious blank line efce9b6 [Joel Nothman] More CSS to Excel testing; define ExcelFormatter.write f17a0f4 [Joel Nothman] Some border style tests 1a8818f [Joel Nothman] Lint 9a5b791 [Joel Nothman] Fix testing ImportError 1984cab [Joel Nothman] Fix making get_level_lengths non-private eb02cc1 [Joel Nothman] Fix testing ImportError 3b26087 [Joel Nothman] Make get_level_lengths non-private f62f02d [Joel Nothman] File restructure dc953d4 [Joel Nothman] Font size and border width 7db59c0 [Joel Nothman] Test inherited styles in converter d103f61 [Joel Nothman] Refactoring and initial tests for CSS to Excel 176e51c [Joel Nothman] Fix NameError c589c35 [Joel Nothman] Fix some lint errors (yes, the code needs testing) cb5cf02 [Joel Nothman] Fix bug where inherited not being passed; avoid classmethods 0ce72f9 [Joel Nothman] Use inherited font size for em_pt 8780076 [Joel Nothman] Merge branch 'master' into excel_style 96680f9 [Joel Nothman] Largely complete CSSToExcelConverter and Styler.to_excel() f1cde08 [Joel Nothman] FIX column offset incorrect in refactor ada5101 [Joel Nothman] ENH: support Styler in ExcelFormatter
Pandas is a nice way to export data from Python to Excel. It even has an internal representation of Excel cell styling. It also has a data-driven framework for styling Pandas dataframes with CSS.
This PR connects those features together in a rudimentary and experimental way, for advanced users only. It allows
ExcelFormatter
to accept aStyler
in place of aDataFrame
and to provide astyle_converter
callback for conversion from CSS styles to the styles understood byExcelWriter
.While I acknowledge that I am extending a hidden feature (to make it more usable) and that high-coverage conversion of CSS to Excel styles is no small feat so ?all prospective usages will be a hack, I would enjoy using this feature and think others would too.
I wasn't sure how to test this as no tests directly instantiate
ExcelFormatter
. (I could of course alter theto_excel
parameters to provide the new feature, but I thought it might best remain hidden. WDYT?)A test follows:
git diff upstream/master | flake8 --diff