New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intregration with Dash #662

Closed
Kapeli opened this Issue Feb 11, 2014 · 27 comments

Comments

Projects
None yet
@Kapeli
Copy link

Kapeli commented Feb 11, 2014

I'll list here all the issues I have found while trying to integrate ReadTheDocs with Dash. It would be great if this list could be worked through so that Dash could someday integrate with RTD.

Searching issues:

  • I mainly want to add RTD integration to cover Python packages, and the issue is that there doesn't seem to be a correlation between Python packages listed at pypi.python.org and RTD project names. For example, beautifulsoup.readthedocs.org has docs for beautifulsoup in Chinese, while beautiful-soup-4.readthedocs.org contains the actual docs for beautifulsoup. A Dash user will most likely search for "beautifulsoup" and install the Chinese docs.
  • Searching for something like "zend" doesn't give you the "zf2" project. Ideally, a search for both "zf" and "zend" should return the "zf2" project.
    • Possible workaround: one idea would be for RTD to search the title of the main page of the generated docs.

Organization issues:

  • As far as I can tell, anyone can claim any project name and have docs generated for it without any kind of verification/review. That's kind of bad, as you get issues like the one I mentioned regarding beautifulsoup. I've also seen some other Chinese docs that are listed as English.

Versioning issues:

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Feb 11, 2014

Another one in Chinese: redis.readthedocs.org.

@ericholscher

This comment has been minimized.

Copy link
Member

ericholscher commented Feb 12, 2014

Yea. We have no reliable way of mapping these things. Since RTD is not just
for Python projects, I don't want to introduce more Python-specific
functionality.

Any reason you only want to use RTD for Python projects?

On Wed, Feb 12, 2014 at 6:20 AM, Bogdan Popescu notifications@github.comwrote:

Another one in Chinese: redis.readthedocs.org.

Reply to this email directly or view it on GitHubhttps://github.com//issues/662#issuecomment-34815442
.

Eric Holscher
Maker of the internet residing in Portland, Or
http://ericholscher.com

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Feb 12, 2014

The plan is to use RTD for everything, but with a slight focus on Python.

However, Python users would most likely search for Python packages, using the same names from PyPI and they'd get the Chinese docs or not find anything at all.

It sucks even for non-Python searches. For example, searching for redis you get redis.readthedocs.org.

For a search of mongodb the API returns this, where the first result (mongodb) is a 404 and the second result (mongodb-documentation) is in Chinese.

I guess the main problem is that RTD has no kind of moderation and/or organisation going on...

@ericholscher

This comment has been minimized.

Copy link
Member

ericholscher commented Feb 14, 2014

Sure. The only canonical resource that we have for projects is their source code URL. Pypi doesn't index that data, so there is no good way to link them together. I don't know what kind of moderation you want/expect us to do?

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Feb 14, 2014

I don't know what kind of moderation you want/expect us to do?

In terms of moderation, I'd want that there would be no Chinese (or anything but English) docs on the English side of ReadTheDocs, or have the other language docs clearly marked as such.

Going to http://redis.readthedocs.org/en/latest/ gives you Chinese docs for Redis, despite of the /en/ in the URL. As far as I understand, RTD supports language separation, but it's not enforced in any way.

@zxaos

This comment has been minimized.

Copy link

zxaos commented Mar 3, 2014

@Kapeli Doesn't PyPi recommend official documentation at pythonhosted.org anyway instead of readthedocs? Would searching that be a workable alternative?

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Mar 3, 2014

@zxaos it would be an alternative, but I see no way of searching it and I also couldn't find any packages that have docs there. Do you know any?

@zxaos

This comment has been minimized.

Copy link

zxaos commented Mar 4, 2014

@Kapeli I don't think there's a search per-se. It looks like each library has its own subdirectory, based on the library name.

Many, but not all of them, seem to use Sphinx docs if it makes parsing easier.

Some examples:

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Mar 4, 2014

Very few of the packages seem to have docs there. Also, there doesn't seem to be a way to download the docs, so I can't use it, sorry.

@odeckmyn

This comment has been minimized.

Copy link

odeckmyn commented Mar 8, 2014

Why not let the Dash2 user enter the RTFD url he wants as a 1st step. This solves every issues seen in this thread while Python community solves the “catalog of docs” issue ?

User would enter any Sphinx generated URL to integrate into (as far as I understand correctly your work)

Let me give http://beautiful-soup-4.readthedocs.org URL to Dash2 in order to integrate BeautifulSoup documentation. Or http://scikit-learn.org/stable/documentation.html for SciKit documentation (on his own URL, neither RTFD nor pythonhosted platform, but still Sphinx doc).

What do you think ?

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Mar 8, 2014

This is what I don't like about that:

  1. I have no way to download the docs for offline use, RTD provides that, other websites might as well, but I would have no way of guessing the URL. I'd have to write an offline website cloner, which is not feasible because a clone might take a very long time and I can't show any kind of progress for it.
  2. Versioning. How would Dash check for updates and know when to regenerate the docs?

These 2 issues would be fixed if I'd only support RTD (i.e. only allow RTD URLs), but I'm not that ok with that since I'm basically telling users to go find the docs on RTD and come back, which would be a source of complaints and would boil down to the same initial problem: you can't really find projects on RTD.

@odeckmyn

This comment has been minimized.

Copy link

odeckmyn commented Mar 8, 2014

I understand all that. Just for your information, I never search for a python documentation from a documentation site (neither RTD nor python-hosted), but 99% of times from the github file (=README most of times). So, from my point of view, asking user to enter the URL to documentation is not an issue. But I’m not the one in charge of user support :)

BTW, thanx for your work.

@chris-erickson

This comment has been minimized.

Copy link

chris-erickson commented Mar 11, 2014

I also usually just find the docs on the github page as opposed to searching for them. I would be completely happy if I had to manually link in the packages I want docs for. I don't typically go through that many, so this would scale fine for me - even if I had to make sure I have the most up-to-date versions. I do understand the bigger problems this might create, however.

@pavarnos

This comment has been minimized.

Copy link

pavarnos commented Apr 27, 2014

+1 for downloadable docs. I travel a lot, and its very handy to have downloadable versions of package docs so I can work on a plane or in an airport where there is no free wifi. For example, I found this issue after hunting for downloadable docs for the phptools/phpword and phptools/phpexcel toolset before boarding a 12 hour flight. These are large libraries with non-trivial multi page manuals that are so much easier to use of you can look up the manuals.

@GaretJax

This comment has been minimized.

Copy link
Contributor

GaretJax commented Aug 23, 2014

Another possible (ad interim) solution is to build Dash docsets (*.docset) during the compilation process on RTD and then offering a download link at the same place where the download links for PDF etc. are found.

@ghuntley ghuntley referenced this issue Feb 9, 2015

Closed

Documentation #771

6 of 20 tasks complete
@ghuntley

This comment has been minimized.

Copy link

ghuntley commented Feb 9, 2015

Project @reactiveui would love to see this functionality added. Ref: reactiveui/ReactiveUI#771

We are a .NET / C# / Windows / Mono / Xamarin project.

@jmagnusson

This comment has been minimized.

Copy link

jmagnusson commented Jun 13, 2015

I agree with @chris-erickson that being able to manually add the docs that you want would be perfectly good. Together with @GaretJax's idea on having RTD generate .docset files I think we're on to something.

What do you think @Kapeli?

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Jun 13, 2015

I'm very reluctant to add half-baked features to Dash, because they cause a lot of confusion and complaints from users. I might do it sometime in the future though, but I'd much rather work on features that I can add properly.

@jmagnusson

This comment has been minimized.

Copy link

jmagnusson commented Jun 14, 2015

@Kapeli I understand your point! Sucks to be left out though because you're a Python junkie 😢

@ericholscher

This comment has been minimized.

Copy link
Member

ericholscher commented Sep 18, 2015

Closing this, as I think it doesn't make sense to support Dash as things are currently architected. If the Python world ever ends up with a specific API doc tool, it would make sense to integrate that with Dash, but since Dash isn't used for Prose documentation, Read the Docs will likely never support it fully.

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Sep 18, 2015

Dash works fine with Prose docs. It's a lot harder to support than API docs, yes, but there are a lot of docsets that are not API-related or API-only. That's my job though and you're not supposed to do anything about it. I'm not expecting RTD to generate docsets.

My issues with supporting RTD have nothing to do with the docs themselves. I don't think I mentioned the docs anywhere in this issue, I might be wrong though. I admit I have lots of issues with Sphinx, but I can work around them.

My issues are with the services RTD provides (searching, versioning and organisation). If RTD would fix those issues, or at least some of them, I'd be able to integrate it.

If this is something you're not willing to do, that's fine and I understand and respect your decision. Don't blame it on Dash and the Prose docs though. The main issue is RTD's API and organisation. Everything else I can work around.

@ericholscher ericholscher reopened this Sep 18, 2015

@ericholscher

This comment has been minimized.

Copy link
Member

ericholscher commented Sep 18, 2015

Good points. The main issue on our side is that most package repositories (PyPI specifically) don't allow any kind of oauth claiming, and we don't really have a sane way to namespace and claim packages. Since we aren't a language specific tool, there are a large number of things that could be named "sphinx" for example.

I guess the proper solution would be to allow a project to specify a link to an existing package in a repository somewhere (eg. 'beautifulsoup' on RTD -> bs4 on PyPI), but again without platform support, it's almost impossible for us to verify a claim of ownership.

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Sep 18, 2015

Project naming is an issue, but I think I can work around it a bit on my end by warning users about the issue, sorting the results and possibly allowing users to search using the full RTD URL of a project they want.

Things have changed with Dash and RTD since this issue was opened, so maybe it's best if we go through it step by step.

How do I search for projects using the API? The only thing I could find was https://read-the-docs.readthedocs.org/en/latest/api.html#file-search, but it doesn't seem to work (see http://readthedocs.org/api/v1/file/search/?format=json&q=virtualenvwrapper). File search is not really what I want though, just project search. For my needs, I think I'd want an API that searches projects using the RTD name (i.e. pip), the RTD URL (i.e. http://pip.readthedocs.org/) and the project URL (i.e. https://pip.pypa.io/) and nothing else. I can work with any kind of project search API you have though.

@stantond

This comment has been minimized.

Copy link

stantond commented Aug 5, 2016

@Kapeli If this is still a thing, that search problem could sit in a new issue to be solved in isolation

(I'd love to see RTD integration in Dash!)

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Aug 5, 2016

@stantond Good idea. I've done that. One thing I've noticed is that there's an API to get the complete list of projects on RTD. I can definitely use that to just grab the list of all projects and implement my own search over it.

I'll start working on it and experimenting.

I'm closing this issue as it's become too old/outdated.

@Kapeli Kapeli closed this Aug 5, 2016

@jonashaag

This comment has been minimized.

Copy link

jonashaag commented Sep 7, 2018

@Kapeli what's the state of Dash and RTD integration? I'd be interested in helping with this.

@Kapeli

This comment has been minimized.

Copy link

Kapeli commented Sep 7, 2018

I've given up on it for now. Even if I come up with a way to get what I need from RTD (search, versioning and so on), there's still the issue with Sphinx, which doesn't have a strict/common format between docs and is hard to index right. Using the intersphinx index is the best approach I have so far, but it often misses things and is incomplete, even for the main Python docs.

It's one of those features that I'm afraid to add because then I'll be stuck maintaining it forever and constantly add fixes/workarounds for. Docs for Node.js packages are in the same boat.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment