Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs deployment updates #749

Merged
merged 7 commits into from Feb 3, 2018
@@ -45,13 +45,20 @@ conda list -e

(cd docs && python premake.py && make html && cd -)
ls -lt docs/_build
ls -lt docs/_build/html
pwd
echo travis_fold:end:build.docs

echo travis_fold:start:upload.docs
if [[ "$TRAVIS_BRANCH" != "master" ]]; then

if [[ "$TRAVIS_BRANCH" == "master" ]]; then
python devtools/ci/push-docs-to-s3.py
elif [[ "$TRAVIS_BRANCH" == "docs_deploy" ]]; then
# change the behavior for the docs testing branch (docs_deploy branch in

This comment has been minimized.

Copy link
@jhprinz

jhprinz Feb 3, 2018

Contributor

interesting idea to have a branch for docs...

This comment has been minimized.

Copy link
@dwhswenson

dwhswenson Feb 3, 2018

Author Member

yeah, that brach has actually been around for a while to test deployment behavior, since PRs from my own fork don't get the AWS keys.

This comment has been minimized.

Copy link
@jhprinz

jhprinz Feb 3, 2018

Contributor

but you disabled this now, correct?

This comment has been minimized.

Copy link
@dwhswenson

dwhswenson Feb 3, 2018

Author Member

oh, right now it behaves the same as a fork PR. But the trick is that to make that deploy (if testing why normal deployment fails), all you have to do is switch out which line is commented there. It specifically has the (commented out, but ready to use) --clobber option which will overwrite (instead of sync diffs). I figure that's probably what you want if you're in that situation ;-)

# the openpathsampling/openpathsampling GitHub repo) in this block
echo "No docs deploy on branch $TRAVIS_BRANCH"
# python devtools/ci/push-docs-to-s3.py --clobber
else
python devtools/ci/push-docs-to-s3.py
echo "No docs deploy on branch $TRAVIS_BRANCH"
fi
echo travis_fold:end:upload.docs
@@ -1,18 +1,45 @@
from __future__ import print_function
import os
import pip
import tempfile
import subprocess
import openpathsampling.version

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--clobber', action='store_true',
help='completely overwrite S3 bucket (default false)')
parser.add_argument('--dry', action='store_true')
opts = parser.parse_args()

CLOBBER = opts.clobber

BUCKET_NAME = 'openpathsampling.org'
if not openpathsampling.version.release:
PREFIX = 'latest'
else:
PREFIX = openpathsampling.version.short_version

if not any(d.project_name == 's3cmd' for d in pip.get_installed_distributions()):
raise ImportError('The s3cmd package is required. try $ pip install s3cmd')
def is_s3cmd_installed():
dists = pip.get_installed_distributions()
if not any(d.project_name == 's3cmd' for d in dists):
raise ImportError('The s3cmd package is required. '
'try $ pip install s3cmd')

def run_cmd(template, config_filename, dry=False):
cmd = template.format(
config=config_filename,
bucket=BUCKET_NAME,
prefix=PREFIX)
if dry:
return_val = cmd
else:
is_s3cmd_installed()
return_val = subprocess.call(cmd.split())
return return_val


# The secret key is available as a secure environment variable
# on travis-ci to push the build documentation to Amazon S3.
with tempfile.NamedTemporaryFile('w') as f:
@@ -22,19 +49,27 @@
'''.format(**os.environ))
f.flush()

template = ('s3cmd --config {config} '
'sync docs/_build/html/ s3://{bucket}/{prefix}/')
cmd = template.format(
config=f.name,
bucket=BUCKET_NAME,
prefix=PREFIX)
return_val = subprocess.call(cmd.split())

# Sync index file.
template = ('s3cmd --config {config} '
'sync devtools/ci/index.html s3://{bucket}/')
cmd = template.format(
config=f.name,
bucket=BUCKET_NAME)
return_val = subprocess.call(cmd.split())
s3cmd_core = "s3cmd --config {config} --no-mime-magic --guess-mime-type"

if CLOBBER:
s3cmd = s3cmd_core + " put --recursive "
else:
s3cmd = s3cmd_core + " sync "

# get the relevant docs onto S3
template = s3cmd + "docs/_build/html/ s3://{bucket}/{prefix}/"
print(run_cmd(template, f.name, dry=opts.dry))
#cmd = template.format(
#config=f.name,
#bucket=BUCKET_NAME,
#prefix=PREFIX)
#return_val = subprocess.call(cmd.split())

# get the index file (main redirect)
template = s3cmd + 'devtools/ci/index.html s3://{bucket}/'
print(run_cmd(template, f.name, dry=opts.dry))
#cmd = template.format(
#config=f.name,
#bucket=BUCKET_NAME)
#return_val = subprocess.call(cmd.split())

ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.