Permalink
Browse files

webpage utility: upgrade preexisting timestamps

Merges #150

Attempt to upgrade preexisting timestamps during continuous
deployment, such that manually upgrading incomplete timestamps
should no longer be necessary.

Add two mutually exclusive arguments to webpage.py:
--ots-cache and --no-ots-cache for configuring the OTS client
caching.

Disable timestamp caching for deploy.sh (i.e. on Travis CI)
  • Loading branch information...
dhimmel committed Dec 11, 2018
1 parent 1614fa3 commit aa664b4fa8974ef2f57f73593d781173b24243ea
Showing with 67 additions and 5 deletions.
  1. +60 −3 build/webpage.py
  2. +1 −0 ci/deploy.sh
  3. +6 −2 webpage/README.md
@@ -18,7 +18,21 @@ def parse_arguments():
parser.add_argument(
'--version',
default=os.environ.get('TRAVIS_COMMIT', 'local'),
help="Used to create webpage/v/{version} directory. Generally a commit hash, tag, or 'local'",
help="Used to create webpage/v/{version} directory. "
"Generally a commit hash, tag, or 'local'. "
"(default: '%(default)s')"
)
cache_group = parser.add_mutually_exclusive_group()
cache_group.add_argument(
'--no-ots-cache',
action='store_true',
help="disable the timestamp cache."
)
cache_group.add_argument(
'--ots-cache',
default=pathlib.Path('ci/cache/ots'),
type=pathlib.Path,
help="location for the timestamp cache (default: %(default)s)."
)
args = parser.parse_args()
return args
@@ -138,7 +152,7 @@ def create_version(args):
args.freeze_directory.joinpath('index.html').write_text(redirect_html)


def get_versions():
def get_versions(args):
"""
Extract versions from the webpage/v directory, which should each contain
a manuscript.
@@ -149,10 +163,53 @@ def get_versions():
return versions


def ots_upgrade(args):
"""
Upgrade OpenTimestamps .ots files in versioned commit directory trees.
Upgrades each .ots file with a separate ots upgrade subprocess call due to
https://github.com/opentimestamps/opentimestamps-client/issues/71
"""
ots_paths = list()
for version in get_versions(args):
ots_paths.extend(args.versions_directory.joinpath(version).glob('**/*.ots'))
ots_paths.sort()
for ots_path in ots_paths:
process_args = ['ots']
if args.no_ots_cache:
process_args.append('--no-cache')
else:
process_args.extend(['--cache', str(args.ots_cache)])
process_args.extend([
'upgrade',
str(ots_path),
])
process = subprocess.run(
process_args,
stderr=subprocess.PIPE,
universal_newlines=True,
)
if process.returncode != 0:
print(f"OpenTimestamp upgrade command returned nonzero code ({process.returncode}).")
if not process.stderr.strip() == 'Success! Timestamp complete':
print(
f">>> {' '.join(map(str, process.args))}\n"
f"{process.stderr}"
)
backup_path = ots_path.with_suffix('.ots.bak')
if backup_path.exists():
if process.returncode == 0:
backup_path.unlink()
else:
# Restore original timestamp if failure
backup_path.rename(ots_path)


if __name__ == '__main__':
args = parse_arguments()
configure_directories(args)
print(args)
create_version(args)
versions = get_versions()
versions = get_versions(args)
print(versions)
ots_upgrade(args)
@@ -31,6 +31,7 @@ git fetch origin gh-pages:gh-pages output:output

# Configure versioned webpage
python build/webpage.py \
--no-ots-cache \
--checkout=gh-pages \
--version=$TRAVIS_COMMIT

@@ -20,15 +20,19 @@ In general, a version is identified by the commit hash of the source content tha
The `*.ots` files in version directories are OpenTimestamps which can be used to verify manuscript existence at or before a given time.
[OpenTimestamps](https://opentimestamps.org/) uses the Bitcoin blockchain to attest to file hash existence.
The `deploy.sh` script run during continuous deployment creates the `.ots` files.
However, these files are initially dependent on calendar services and must be upgraded at a later time by running the following in the `gh-pages` branch:
There is a delay before timestamps get confirmed by a Bitcoin block.
Therefore, `.ots` files are initially incomplete and should be upgraded at a later time, so that they no longer rely on the availability of a calendar server to verify.
`webpage.py`, which is run during continuous deployment, identifies files matched by `webpage/v/**/*.ots` and attempts to upgrade them.
You can also manually upgrade timestamps, by running the following in the `gh-pages` branch:

```sh
# Requires a local bitcoin node with JSON-RPC configured
ots upgrade v/*/*.ots
rm v/*/*.ots.bak
git add v/*/*.ots
```

Verifying timestamps with the `ots verify` command requires running a local bitcoin node with JSON-RPC configured, at this time.

## Source

The manuscripts in this directory were built from

0 comments on commit aa664b4

Please sign in to comment.