Documentation for Hadoop jobs ahead of where release on PyPI is

[ryaanwells]

Hello,

The "top artists" documentation is ahead of the released package on PyPI. See here, the luigi.contrib.hadoop.JobTask doesn't exist in version 1.1.2 (it's still at luigi.hadoop.JobTask) and seems to be a change in 1.1.3 which is yet to be released on PyPI.

This isn't a huge problem, but it may be worth noting somewhere that the documentation is for the bleeding edge?

What are your thoughts?

[erikbern]

Thanks – we should probably just release a 1.1.3 next week to resolve this

[Tarrasch]

@erikbern Can you release early next week? Or even now, doesn't matter for me. :)

[ryaanwells]

A release will solve this problem now but since the documentation site updates automatically whenever the repo does (saw that my PR went straight to RTD!) then we'll still be in this situation.

Few suggestions: state that the documentation is for the state of the repo at the HEAD, or manually update RTD every time you do a release to PyPI.

It's up to you how you wish this to be managed, but for the sake of less confusion it may be worth considering :)

[erikbern]

yes I haven't thought about it. Its' a good point. I'm open to anything

[Tarrasch]

There's latest page and a stable page:

http://luigi.readthedocs.org/en/latest/

http://luigi.readthedocs.org/en/stable/

Not sure what stable really means though and where it comes from...

[ryaanwells]

It's the way the RTD does it's versioning

https://read-the-docs.readthedocs.org/en/latest/versions.html

Looks like if the releases are tagged then they can be accessed somehow? Either way, the "stable" could be setup to be the latest release, and "latest" to be the HEAD?

[Tarrasch]

@ryaanwells, do you have any idea why stable did not update? I both released to pypi and added a github tag today.

[hellais]

@Tarrasch the name of the tag is not properly named. It needs to follow semantic versioning (http://docs.readthedocs.org/en/latest/versions.html#how-we-envision-versions-working), in particular the two latest tags need to be renamed from "v1.2.1" and "v1.2.0" to "1.2.1" and "1.2.0" (no v).

What is the release procedure that this project is following? Is a specification of it available somewhere?

[Tarrasch]

Aha, I see. Thanks @hellais! I prepended "v" since that is what github suggested. Also, ffs, that's what the semver page does themselves in the top of their page.

About release procedure, we don't have any good procedure but we would be happy to adopt one! You can discuss it in this email thread I think: https://groups.google.com/forum/#!topic/luigi-user/o98fJZ65oK0

[Tarrasch]

I added 1.2.1 now (without v). It still hasn't updated but I guess it can take a while.

[hellais]

It's actually weird that it takes a while. If you have git hooks setup for rtd it should be immediate. You probably want to look at how the hook is setup on github and consult the settings page on rtd.

[Tarrasch]

Anyway, it's up to date now. Thanks @hellais!

Anyway @ryaanwells, I think it makes sense that people who want to use the pypi package use the stable documentation page. Can this ticket be closed?

[ryaanwells]

Almost.. :)

I've just added a MR here that links the readme to the bleeding edge docs, this will replicate over to RTD as well.

However when you go to http://luigi.readthedocs.org you end up on the "latest" docs and not the "stable" ones. Can we update this so it points to the "stable" ones? https://read-the-docs.readthedocs.org/en/latest/versions.html#redirects-on-root-urls

Other than that I'm happy to close!

[Tarrasch]

I'm totally confused. Where do I set the "Default version" that the root url will redrict to? Is it this? I've no idea where to set that.

Also, I'm not sure if rtd supports it, but it would actually be nice to have the docs up there for every tag on github. In case somebody is stuck on an old version of luigi. Anyone knows if that's possible?

Ping @steenzout who seem to have been playing around with those parts of the code :)

[ryaanwells]

Yeah, the documentation is actually pretty naff, ironically.

So I think I've found it. If you go to the admin panel for the Luigi docs, then the Versions tab, you can set the default branch there! It'll update throughout the rest of the settings.

In the bottom left corner of the docs there's a version selector, it's not that obvious but you can swap between versions. Looking to see if there's a nicer way we can represent that.

[Tarrasch]

Maybe @erikbern has permissions to fix this?

[erikbern]

I changed the default version to stable

I don't know how RTD knows what version is stable though

[erikbern]

Actually sounds like it will work automatically

http://docs.readthedocs.org/en/latest/versions.html

"We also create a stable version, if your project has any tagged releases. stable will be automatically kept up to date to point at your highest version."

[Tarrasch]

Sweet. Let's close this then?

[ryaanwells]

Yeah, that change has worked, happy to close this off. Thanks all 👍

[ryaanwells]

Only thing is the link to bleeding edge docs is missing from the main readme on RTD. Probably not that big an issue and will be fixed with the next release.

[Tarrasch]

Oh yea right. Yea it'll be fixed with next release :)