ENH: Support pushing to *.github.io from other repo. #190

danielballan · 2017-05-22T21:40:29Z

Doctr already supports publishing to the master branch of an org/org.github.io repo. I believe it does not support pushing from org/some_repo (and perhaps multiple other repos) into org/org.github.io. (Why do this? Different projects in an org can publish into subdirectories of one org-wide gh-pages repo and get a nice URL structure: org.github.io/repo_a, org.github.io/repo_b, etc. You could do this with separate gh-pages branches, but it's nice to keep the large build projects off in their own org-wide repo and keep the repos' branches light. The projects matplotlib, cmocean, and trackpy all do this.)

The issue arises from deploy_branch beingmaster. Local master refers to the history of org/some_repo, while doctr_remote/master refers to the history of org/org.github.io and doctr mixes them up.

I have resolved this by doing the local work on a special branch with an unambiguous name that tracks doctr_remote/{deploy_branch} and is explicitly pushed to it.

I think this modification does not break existing use cases but someone more familiar with the codebase should take a careful look. This Travis build shows it operating successfully for my use case of off this PR.

asmeurer · 2017-05-22T22:37:54Z

Tests fail (pyflakes).

I'll push the branch up to origin so that it tests the deploy (by the way, this is the best branch name I've ever seen).

asmeurer · 2017-05-22T22:41:08Z

doctr/travis.py

-    if new_deploy_branch or local_deploy_branch_exists:
-        run(['git', 'checkout', deploy_branch])
-    else:
-        run(['git', 'checkout', '-b', deploy_branch, '--track', 'doctr_remote/{}'.format(deploy_branch)])


Does this still automatically create gh-pages on the remote if it doesn't exist?

Yes, if I understand correctly that part happens in create_deploy_branch which is still called, just above the first line I touched. To be clear I have not tested that; I just don't think the behavior had changed.

Actually I was wrong about that -- it did not successfully create a master branch if one did not exist -- but that is fixed in 8007280, which I think will pass once it runs in context where it's allowed to push.

asmeurer · 2017-05-22T22:44:45Z

Seems like this would have needed to have been done in the first place for .github.io support to be working.

asmeurer · 2017-05-22T22:46:19Z

I'd like to see a deploy added to .travis.yml to deploy to drdoctr.github.io (it seems that wasn't done in #168).

danielballan · 2017-05-23T03:04:19Z

Seems like this would have needed to have been done in the first place for .github.io support to be working.

Seemed that way to me too but I figured someone would have caught it by now? I'll try to add that test you suggest.

danielballan · 2017-05-23T12:41:06Z

I amended my original commit to fix the pyflakes failure and added a deployment to drdoctr.github.io.

danielballan · 2017-05-23T15:21:29Z

I think this will pass if it runs on the upstream fork.

danielballan · 2017-05-23T15:22:19Z

(The failure is: we can't create a branch that tracks the remote doctr_remote/master if it doesn't exist, and we can't create that remote if canpush is False.)

danielballan · 2017-08-19T17:05:08Z

Please let me know when you have the bandwidth to review/merge and I will happily rebase.

asmeurer · 2017-08-19T19:11:53Z

I can review, but I think this PR was not working (at least it was failing the tests).

If we are deploying *to* a github.io repo we must deploy to 'master' (GH does not even let you configure this -- it's always 'master') regardless of whether we are publishing content *from* a *.github.io repo or some other repo. Thus, deploy_repo, not deploy_dir.

Use the working branch approach in branch creation as well. Former output (from Travis failure): ``` Creating master branch git checkout --orphan master fatal: A branch named 'master' already exists. ```

danielballan · 2017-08-21T14:03:58Z

@asmeurer The Travis failure will go away when this is merged into master, where canpush is True, and doctr_remote/master can be created. In this PR branch, canpush is False, so we we can't create doctr_remote/master and therefore we cannot create a local branch that tracks it. (This error happens here in the build log.)

There is a working example you can see: I have been deploying NSLS-II docs using the version of doctr in this branch. See a working build here, which published this commit. This particular build ran after my rebase, verifying that the rebase didn't break it.

asmeurer · 2017-08-22T18:21:46Z

Well we've got it set up to not make PRs mergeable unless they pass, so I've gone ahead and created the master branch.

asmeurer · 2017-08-22T18:24:35Z

Now it failed:

git fetch doctr_remote
Checking out doctr working branch tracking doctr_remote/master
git checkout -b __doctr_working_branch --track doctr_remote/master
fatal: Cannot update paths and switch to branch '__doctr_working_branch' at the same time.
Did you intend to checkout 'doctr_remote/master' which can not be resolved as commit?

By the way, we should put an except block in our try/finally that prints an exception message to make it easier to find the actual error in the build logs.

danielballan · 2017-08-22T18:34:48Z

Thanks. The ball is in my court then. I'll see if I can get the tests to pass.

danielballan · 2017-08-22T18:56:51Z

@asmeurer It looks like the failing builds (the ones in travis-ci/pr) haven't run since yesterday, before the master branch was created. Would you re-run them? I don't think I have the necessary permissions to do that.

danielballan · 2017-08-22T18:57:50Z

Ah, pushing a new (empty) commit made Travis spin. Tests pass. Now let me remove that empty commit....

asmeurer · 2017-08-22T19:00:28Z

I did a merge on master after merging another PR, which should make it re-run. I'll merge this is Travis passes.

danielballan · 2017-08-22T19:01:42Z

What sort of error message did you have in mind for that try/finally block? Do you want something with caps or whitespace to stand out in the logs? Or is there something slicker we can do?

asmeurer · 2017-08-22T19:02:42Z

I'm working on the message at #239. If you have a better suggestion let me know.

asmeurer · 2017-08-22T19:08:48Z

OK, let's see if https://github.com/drdoctr/drdoctr.github.io gets updated.

asmeurer · 2017-08-22T19:20:31Z

Oh I need to run doctr configure and actually set up the deploy key. But why didn't the build fail? https://travis-ci.org/drdoctr/doctr/jobs/267309777

asmeurer · 2017-08-22T19:21:51Z

I think maybe it's because we already have doctr_remote from the other doctr commands. It too should be cleaned up in the finally block.

asmeurer · 2017-08-22T19:23:20Z

Oh nevermind that's already taken care of. The docs aren't pushed because it thinks they haven't changed.

asmeurer · 2017-08-22T19:24:16Z

OK, I think you need to clear the __doctr_working_branch in the finally block. I'll open a PR.

asmeurer · 2017-08-22T19:25:24Z

No, you're already doing that too. I'm not seeing why it's failing.

asmeurer · 2017-08-22T19:26:51Z

Oh, I see. It's because we (probably stupidly) have it set up in our Travis that you have to have --sync for the docs to actually push.

asmeurer · 2017-08-22T19:29:21Z

Which actually leads to a second question, which is how we can actually test this, since doctr is currently set up to use only one deploy key, but we would need two.

I guess I can generate a new deploy key manually and use it for both repos (hopefully GitHub allows that).

asmeurer · 2017-08-22T19:32:11Z

No, GitHub doesn't allow it.

danielballan · 2017-08-22T19:40:04Z

Right. Is it worth adding something to the test that makes a tiny random change to the docs each time?

asmeurer · 2017-08-22T19:42:02Z

The docs do get updated each time (sphinx adds timestamps). So that isn't necessary. I added --sync here, but it won't work until we can get a deploy key set up, which we need to figure out how to do (#242).

danielballan · 2017-08-22T19:50:04Z

Ah, got it.

danielballan force-pushed the other-master branch from 30e8e71 to 8618a15 Compare May 22, 2017 21:45

asmeurer reviewed May 22, 2017

View reviewed changes

danielballan force-pushed the other-master branch from 8618a15 to 8b73763 Compare May 23, 2017 12:30

danielballan force-pushed the other-master branch 8 times, most recently from a3cfb58 to 8007280 Compare May 23, 2017 14:49

stuartcampbell mentioned this pull request May 25, 2017

Revert to using standard doctr NSLS-II/docs#66

Open

danielballan added 4 commits August 20, 2017 16:03

ENH: Support pushing to *.github.io from other repo.

fd722b5

TST: Add a test deploying to drdoctr.github.io repo.

3369538

FIX: Properly create new master branch on deploy_remote if needed.

7408e1e

Use the working branch approach in branch creation as well. Former output (from Travis failure): ``` Creating master branch git checkout --orphan master fatal: A branch named 'master' already exists. ```

danielballan force-pushed the other-master branch from 8007280 to 7408e1e Compare August 21, 2017 12:55

danielballan force-pushed the other-master branch from dc82213 to 7408e1e Compare August 22, 2017 18:58

Merge branch 'master' into other-master

4b05a97

asmeurer merged commit c9f25d4 into drdoctr:master Aug 22, 2017

danielballan deleted the other-master branch August 22, 2017 19:23

asmeurer mentioned this pull request Aug 22, 2017

Multiple deploy keys #242

Closed

This was referenced Sep 13, 2017

FIX: Use NSLSII doctr for now ... slaclab/skywalker#11

Merged

FIX: Use NSLSII doctr now ... pcdshub/happi#10

Merged

danielballan mentioned this pull request Jul 18, 2018

Attempt to have docs autogenerate and deploy to webpage NSLS-II-SIX/sixtools#7

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Support pushing to *.github.io from other repo. #190

ENH: Support pushing to *.github.io from other repo. #190

danielballan commented May 22, 2017

asmeurer commented May 22, 2017

asmeurer May 22, 2017

danielballan May 23, 2017

danielballan May 23, 2017

asmeurer commented May 22, 2017

asmeurer commented May 22, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017 •

edited

Loading

danielballan commented Aug 19, 2017

asmeurer commented Aug 19, 2017

danielballan commented Aug 21, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

danielballan commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

ENH: Support pushing to *.github.io from other repo. #190

ENH: Support pushing to *.github.io from other repo. #190

Conversation

danielballan commented May 22, 2017

asmeurer commented May 22, 2017

asmeurer May 22, 2017

Choose a reason for hiding this comment

danielballan May 23, 2017

Choose a reason for hiding this comment

danielballan May 23, 2017

Choose a reason for hiding this comment

asmeurer commented May 22, 2017

asmeurer commented May 22, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017

danielballan commented May 23, 2017 • edited Loading

danielballan commented Aug 19, 2017

asmeurer commented Aug 19, 2017

danielballan commented Aug 21, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

danielballan commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

asmeurer commented Aug 22, 2017

danielballan commented Aug 22, 2017

danielballan commented May 23, 2017 •

edited

Loading