Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Website building process needs improvement #180

Closed
jlstevens opened this Issue Jun 24, 2015 · 9 comments

Comments

Projects
None yet
3 participants
@jlstevens
Copy link
Contributor

jlstevens commented Jun 24, 2015

At the minimum, the dev/Tutorials should be automatically updated.

@jlstevens jlstevens added the docs label Jun 24, 2015

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 24, 2015

I know why everything is configured this way (we don't want to bloat the code repositories) but the process is fairly miserable even just for updating the content in /dev/Tutorials:

  1. Stash whatever you are doing.
  2. Checkout gh-pages.
  3. Make sure to pull the latest version of gh-pages.
  4. Run git submodule update --init (take a long time and downloads around 1.8GB!)
  5. Checkout holoviews-tutorials then pull the submodule you want to update (typically dev/Tutorials)
  6. Make the commit(s) needed to update the submodule reference(s).
  7. Push to gh-pages.
  8. Trigger a documentation build on buildbot.
  9. Switch back to master and delete the Tutorials and dev/Tutorials directories.
  10. Pop the stash.

Note that some of these steps are not required if you keep a copy of holoviews somewhere on your hard drive just for this purpose!

Unfortunately there are a lot of steps but I want them all automated (at least for dev/Tutorials).

Edit 1: Philipp tells me that fewer steps are required if you work on doozy directly. He'll explain more shortly...

Edit 2: Now I find out doozy won't push so all these steps are still insufficient. You apparently have to work via doozy.

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 24, 2015

I think we should consider a new approach - our website is huge due to the notebooks and the gh-pages branch is bloating our code repository. I would consider removing the gh-pages branch and keeping the website in an entirely separate repository.

As we have enough repositories, maybe we could have a holoviews-website subfolder on the master branch of the ioam.github.com repository?

@philippjfr

This comment has been minimized.

Copy link
Contributor

philippjfr commented Jun 24, 2015

I totally agree that now we a dev section of the homepage this process should be automated. The list you gave there isn't the currently the way it works however.

  1. Make your changes
  2. Tell buildbot to build the docs
  3. Go to doozy then:
cd /var/lib/buildbot/slaves/holoviews_docs/build/doc/reference_data
git checkout holoviews-tutorials
git push
cd ../..
git checkout gh-pages
git submodule update --init
cd (dev/)Tutorials
git checkout holoviews-tutorials
git pull
cd ..
git add Tutorials
git commit -m "Updated (dev/)Tutorials submodule reference"
git push

This final step is what could be automated now that we have the dev folder on gh-pages.

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 24, 2015

Ok thanks! At least we now have an 'official' set of instructions for the current procedure...

What do you think about the idea of using a subfolder of ioam.github.com? I'm sure we could arrange the redirects to make the website links work the same way they do now...

Edit: For reference, this page on setting the CNAME might help...

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 24, 2015

Some notes to add to Philipp's instructions:

  • You need to log in as user topo-buildbot on doozy.
  • I had to git reset HEAD --hard to switch to the gh-pages branch for some reason.
  • The submodule update step is still very slow!
  • I had to use git remote set-url origin https://github.com/ioam/holoviews.git before the final git push.

As for moving the gh-pages branch somewhere else, this is the most useful page I've found so far

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 24, 2015

Ok. The tutorial links are finally fixed to point to the archive for 1.2.0!

This was much too much effort for something that should happen automatically nightly (again, for the dev version).

@jlstevens jlstevens added the testing label Jun 24, 2015

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Jun 27, 2015

The website has been updated for 1.3.0 - once again, it has been a pretty painful process.

In addition to the automation, we should also think about keeping snapshots of the website (including the tutorials!) for each version. We could do this with tags, but you would need to tag both gh-pages in the holoviews repository as well as the holoviews-tutorials branch of the ioam.github.com repository (and some other branches as well).

Adding these git tags is easy but making a snapshot of the whole website is a bit more effort...

Edit: A single separate repository just for the website would make this much easier...

@philippjfr philippjfr added this to the v1.4.0 milestone Sep 11, 2015

@philippjfr philippjfr self-assigned this Sep 11, 2015

@philippjfr philippjfr referenced this issue Nov 10, 2015

Merged

Data API #284

42 of 42 tasks complete

@philippjfr philippjfr modified the milestones: Post v1.4.0, v1.4.0 Dec 1, 2015

@philippjfr philippjfr modified the milestones: v1.4.2, Post v1.4.0 Jan 26, 2016

@philippjfr philippjfr assigned jlstevens and unassigned philippjfr Feb 3, 2016

@jlstevens

This comment has been minimized.

Copy link
Contributor Author

jlstevens commented Feb 3, 2016

This issue has finally been addressed in PR #429 together with many other improvements including moving the websiteto S3, the travis.holoviews.org page and the buildbot.holoviews.org page.

Other than some polish, this work is now complete and the process of updating the website and test data is now much easier. For an example of one of the first PR's that was updated using the new system, see PR #436.

Closing the issue.

@jlstevens jlstevens closed this Feb 3, 2016

@jbednar

This comment has been minimized.

Copy link
Contributor

jbednar commented Feb 3, 2016

Hooray!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.