-
-
Notifications
You must be signed in to change notification settings - Fork 30.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modernize pydoc to use better HTML and separate CSS #54925
Comments
This is a straight-forward project. Pydoc currently generated 1990's style html which mixes content and presentation, making it very difficult for users to customize the appearance of the output. It is full of html like: <table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">
def section(self, title, fgcol, bgcol, contents, width=6,
prelude='', marginalia=None, gap=' '):
"""Format a section with a heading."""
if marginalia is None:
marginalia = '<tt>' + ' ' * width + '</tt>'
result = '''<p>
<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="section">
<tr bgcolor="%s">
<td colspan=3 valign=bottom> <br>
<font color="%s" face="helvetica, arial">%s</font></td></tr>
''' % (bgcol, fgcol, title)
if prelude:
result = result + '''
<tr bgcolor="%s"><td rowspan=2>%s</td>
<td colspan=2>%s</td></tr>
<tr><td>%s</td>''' % (bgcol, marginalia, prelude, gap) Please convert it to simple, validated HTML with the formatting moved to a simple, validated default style sheet. Liberally apply div/span elements with class/id attributes as necessary. |
I believe Ron has done something along these lines already. See bpo-2001. Committed css file is empty, (see r86971) but I am adding Ron to "nosy" in case he has something in the pipeline. |
See also the calendar module for an example of a "modern" html. |
I uploaded the css file I used in an experimental version of pydoc. It may give some useful starting values. Before this is done, the old server code should be removed (also for 3.3). (another issue?) There are two files in the tools/scripts directory that may need attention as well. pydocgui.pyw and pydoc3. |
Those two scripts are just three-liners pydoc launchers. I disagree about the CSS reset. Default values and user-set values have a reason for being there, and I share the viewpoint that they should not be reset. |
I'm looking for a deeper fix, all the in-line styling replaced by a stylesheet. Can you guys work together on bring this to fruition? |
Eric, most of what's in that file is what I figured out by trial and error in order to get it to work on the different browsers at that time. (about 3 years ago.) You are probably more experienced with css than I am, so you are more than welcome to update and change anything in there. :-) What do you think about starting with a set of static html pages to get the css and html to work nice, and then only after that is done, edit pydoc to generate those pages. That should get us mostly there, and simplify what needs to be done in pydoc. |
A good procedure is to start afresh with an empty, embedded style sheet and replace the html styling attributes one at a time, while keeping the overall look and feel the same. At the end of the process the embedded style sheet can be converted to external. |
I think that's how I ended up with the style sheet I uploaded. It works, but it can be a slow process. Another factor is the pydoc server will reread an external style sheet on browser refreshes. So you can see the results of style sheet changes without restarting pydoc. If it's embedded, you need to edit the program and restart everything each time. I was thinking that I may still be able to make a set of static html files that go along with that style sheet. Then we can adjust the css class names and make other changes, then use that as a guide for replacing the html. Eric, what do you think? |
The uploaded stylestyle is *much* bigger than I expected. There should be no more than a one-to-one correspondence. With inheritance, perhaps even fewer entries. |
Ok, I just looked at them again, I didn't remember how different it was. They probably won't be much help other than maybe seeing how some things could be done. Here's a zip file of some saved pages, so you can take a look. |
The CSS also has a validation error, some warnings and could be written better IMHO (specifically I don't like the placement of the {}). I also don't believe that extensive CSS resets are useful -- they usually just lead to lot of redefining. I usually limit myself to * { margin: 0; padding: 0; } and possibly specify specific values when necessary. Minor differences with different browsers can be perfectly fine in lot of cases. |
[Raymond]
I wonder how desirable it is to preserve the look and feel of the pages. We all agree on externalizing the CSS and add a way for the users to add their own stylesheet; why not take the opportunity to also improve the style? Huge blocks of colors are not that attractive to me :) Regarding workflow, I’d find easier to start from bare HTML that works nice (yes, I test with w3m) and then add style. Technically, I’d like to maintain the HTML as a small set of files (in pydoc_data) containing fragments of HTML with placeholders ({} or $): That’s easy to edit, to validate (a very simple script can put the fragments together) and to use. I agree that the CSS file should be as short as possible (in content), but not too short (in file size). For example, trailing commas in properties and brackets on their own line will allow nice diffs, just like Python. (I won’t have much time for Python in December, but we have a lot of time before 3.3b1 :) |
There's no rush on this one.
If the current look cannot be recreated, it should be taken as a sign that the content/presentation split wasn't done well. |
Here is a tentative start on this. (css_v1.diff) The css file is much better. It's shorter, simpler and validated. The header and navbar panel use it in the new server. Added a markup call to the topic page contents. (The same markup call is already used for the doc strings.) Extra items in the css are what I come up with from testing different ways of doing things. |
The HtmlDoc class has methods that take colors. Can this be changed or does it need to be depreciated first? def heading(self, title, fgcol, bgcol, extras=''):
"""Format a page heading."""
return '''
<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">
<tr bgcolor="%s">
<td valign=bottom> <br>
<font color="%s" face="helvetica, arial"> <br>%s</font></td
><td align=right valign=bottom
><font color="%s" face="helvetica, arial">%s</font></td></tr></table>
''' % (bgcol, fgcol, title, fgcol, extras or ' ') For the interactive server, I can override these methods with no problem, but the generated docs won't benefit from this until the HtmlDoc class is replaced. Any suggestions? |
Well, you could reuse these arguments to mean CSS classes, and have styles like ".red { color: red }" :) |
It may be useful to change those to 'id=' and 'class=' if possible. It isn't clear to me how much of pydoc is still part of the public api in python 3.x. pydoc.__all__ is set only to ['help']. Entering help(pydoc) just gives the basic help and command line arguments along with. DATA There is nothing in the official (online) docs on importing pydoc or any of it's classes or methods. But dir(pydoc) shows all the methods, and the HTMLDoc class is still importable even though they aren't listed in __all__. |
Here is a new diff which updates all the new pydoc pages to use the css file. The css file is simpler and cleaner. I also made a few adjustments to the url handler error handling, and changed the titles in the head sections so they say "Pydoc" instead of "Python" as they are PyDoc pages about Python. None of these changes effect any of the old pydoc code yet. This is about as far as we can go without removing the old tk panel and server. Time for some feed back. And how close do we really need it to be to the original? :-) |
A few comments about css_v2.diff:
Regarding the HTML:
There are also some minor incontinences in the indentantion, e.g.: |
We may have to do dome depreciating when it comes to the old HTMLDoc class methods. Would it be possible to depreciate the whole class instead of the separate methods? If so, it would be good if we can squeeze that into 3.2. <hint hint> Or else we may not be able to finish this until Python 3.4. The empty css elements will be used later. Here is the css_v3.diff for review. I'd like to hear any thoughts about the general html structure and class/id names. Beside validating them, I test with firefox and chromium. I don't have easy access to ms-explorer or the current mac browser. |
These arguments should not really be of concern. If we indeed deem them public API, they can stay but be ignored. |
To go forward I can create a new private api instead of changing HTMLDoc, that would be preferable. Should the -w option also use the new html pages? Or do we need a new option for that? |
A reminder: Check for instances where html.escape is not called on data inserted into the html pages. I'll update the patch as the non-css (error handling) parts made it into python 3.2. :-) |
I added a few comments for ccs_v3.diff on rietveld. |
Thanks for the update, Berker. I'll keep myself in the nosy list as I'm interested to learn about this anyway :) |
When rhettinger created this issue, the goal was "Pydoc currently generated 1990's style html which mixes content and presentation, making it very difficult for users to customize the appearance of the output. ... Please convert it to simple, validated HTML with the formatting moved to a simple, validated default style sheet." Apparently, the statement of this issue is no longer correct. However, in order to do so, one apparently needs to have access to a private list mentioned in a comment (https://mail.python.org/mailman/private/core-mentorship/2016-August/003622.html) One thing that motivated to submit my tentative "solution" to this issue here almost two years ago (solution which, I thought, met the stated goal) was a discussion at Pycon as to how one could encourage new contributors. Is this the purpose of the core-mentorship list? Does one need to join the list if one wishes to contribute? |
Hi Andre, You can subscribe to the core mentorship mailing list here: https://mail.python.org/mailman/listinfo/core-mentorship Once you subscribed, you'll get access to the said discussion :) It is not required, but it's a great place to ask questions or get help from existing core developers. HTH |
I've removed the easy tag. (Aside for those who can do the operation: to remove all tags from an issue one must select '-no selection-'; just ctrl-clicking the last tag to remove its highlight doesn't work...or at least that's what I remember, I didn't actually test it this time.) |
I see that pydoc still useful, and have 6 points I'd like to work on with pydoc. Is this too much for one issue? Should I break bits out to another issue(s)? Although it would make sense to do it all in one go. ----------------------------------------------- pydoc changes
Currently: user@server:/mnt/c/course$ pydoc -w ./helloFunc.py TODO: Update the help strings.
This changed from Python2: TODO: Remove the last two lines of the help output from Py3 pydoc when run from the command line.
This can add confusion. It would be more useful to display a specific error, and refer users to more help, like ls does:
|
You sound like you think we want to get rid of pydoc, which is certainly not true :). As I understand it it is only the *html* output of pydoc that we are considering deprecating. Yes, you should open separate issues for your proposed changes. 1 and 2 should be separate issues, 3 and 4 should be combined into a proposal to convert pydoc to using argparse. 5 you can skip, and 6 I don't think will be accepted, although you are welcome to propose it along with some motivating use cases. |
I disagree with deprecating HTML output. Pydoc has builtin HTTP server, it provides the documentation with hyperlinks. It would be nice if IDLE use this feature for displaying interactive help. I think we should enhance HTML output not just by making it looking better, but by adding more interactivity. For example add the ability to collapse classes, methods, etc, add more hyperlinks. |
That sounds like a reasonable argument to me. To clarify something in my last post: 5 should be skipped in terms of opening new issues because it is already covered by this issue. If Terry would like to see pydoc enhanced to support idle, then I think that would decide the issue. |
Since David added me as nosy, I will address the IDLE issues raised above. But first, a question. Does 'deprecating HTML output' merely mean deprecating the old output in favor of new and better HTML output, or deprecating having any html output, which would also mean deprecating the html doc server? I am strongly against the latter. Currently, IDLE, in both pyshell and run.py, imports pydoc and sets "pydoc.pager = pydoc.plainpager". Bypassing pydoc.getpager this way is not documented, but seems perhaps necessary. Other than this, I believe 'help(xyz)' is treated by IDLE just like any other user code. Serhiy, msg278126 > "It would be nice if IDLE use this feature" There are actually two features: hmtl output, and the html server, which uses the default browser to display the output. Since a year ago, IDLE help displays the html IDLE doc produced by sphinx in a subclass of tkinter Text that feeds the page to a subclass of HTMLParser. The code is in help.py. Sphinx's html seems to follow Frédéric's guidelines in msg246917. I suspect the CSS files are ignored. We might want to do something similar with pydocs html output. Raymond: is sphinx's pydoctheme anything like the css you are looking for? Could it be used as a starting point? There have been requests that 'long' help output (for modules and classes) be displayed separately from Shell itself so that it does not obscure history and be easier to refer to. If we do this, using html instead of plain text would be nice, especially if new features were added. Serhiy, cont. > "I think we should enhance HTML output not just by making it looking better, but by adding more interactivity. For example add the ability to collapse classes, methods, etc, add more hyperlinks." I mainly use pydoc server for tkinter. I would *love* have the repetitive 'inherited methods' section for each class collapsed by default. Ditto for when I use help interactively. To me, this is the single worst feature of pydoc output. David msg278127 > "If Terry would like to see pydoc enhanced to support idle, then I think that would decide the issue." I am not sure what you mean by 'the issue' and I can't currently think of anything IDLE-specific that pydoc should do, other than be consistent. However, producing modern, decent looking html output would make html use possible. Incorporating the current output, as displayed in the server, might be a net negative PR move. Side note: thinking about how to make a clickable link in a Text widget, I realized that that IDLE already does some syntax tagging for colors (keywords, builtins, def and class identifiers). The same tags could be bound to double-click or right-click to display help popups, such as if_stmt ::= "if" expression ":" suite It selects exactly one of the suites by evaluating the expressions one Related help topics: TRUTHVALUE |
My memory of the thread (I haven't gone back and checked) was that the thought was to drop html output altogether because the sphinx docs were more useful for anyone wanting to use html, and network connectivity was so universal in this day and age that having a local html server for offline uses was of minimal utility. So your vote that it be kept because it is useful to idle and can be made more useful by improving the html seems to me to decide the issue in favor of making the improvements (given someone willing to do the patch review and any additional needed coding). |
I've found the HTML to be useful (-w mode, not running a server) for generating quick documentation (much lighter weight commitment than using sphinx). I show this in my intro classes and the engineers are usually impressed with it. This is really easy and requires neither configuration files or reST markup: $ python -m pydoc -w statistics
wrote statistics.html So, my vote it to keep it. I would still like modern HTML 5 with CSS however. |
On Sat, Mar 23, 2019 at 6:27 PM Raymond Hettinger <report@bugs.python.org>
I submitted code to do this in January 2015 on the bug tracker but, except André Roberge
|
I stumbled upon this issue because I think this is a potential improvement. One of the reasons I don't use it is due to the outdated UI but since it also generates HTML doc for installed packages in a virtual environment I find this to be a useful feature like rdoc (ruby), rustdoc (rust) and are actively maintained helping documentation of third party packages too for local reference. I have less knowledge about CSS but I tried updating css_v3.diff with master and added boostrap.css from CDN to pydoc which adds spacing along with improving layout and font. Attached is a screenshot that is looks better with the CSS patch and bootstrap. Including bootstrap is not lightweight option in the Python distribution but can be seen as a proof of improvement. |
Thanks @aroberge for the patch and your efforts on improving it https://aroberge.blogspot.com/2015/01/scratching-itch-improving-pydoc.html |
Hey all, I'd like to note my interest in this issue getting addressed. I have a use case where I'd like to generate documentation for a module and using non-core modules to do that requires some approval that I would like to avoid. I understand there are other more important issues but this issue has had a patch waiting for review for ten years. Can we move this forward somehow? |
Someone would need to take the patch, apply it to current main branch, fix issues and open a PR. |
On second thought, this issue is not purely programming, but also has a UI component. Better HTML and separated CSS would be good, but before that I think we’d need someone with design chops to mock up a better page layout and style (better than big coloured rectangles!), then I think have a discussion on python-dev (not to discuss all details in committee, but to validate the overall change). |
Tried myself at it. I'm not a front-end guy, fasten your seatbelts. I tried to keep it minimal, but after some work the change is not that small, I focused on the HTML modernization only, not the design (to try to land at least a first step toward modernization). |
Does someone want to review #28651? |
@JulienPalard The issue seems to have been solved. I think we can issue the problem. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: