Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
The Internet, peer reviewed.

Use SHA512 and perfom HKDF extract step

Nick Stenning originally had the ephemeral key set to 64 octets in
length. I lowered it to 32 when I introduced HKDF because I was
defaulting to SHA256 because it didn't need to be any longer than
the digest size according to the spec and SHA256 has a digest size
of 256 (= 32 * 8) bits.

However, Pyramid and WebOb default to using SHA512 in most cases
so it makes sense to use an input key that matches the digest size.
Some references seem to suggest using a key that's the internal
block size (which would be 1024 = 8 * 128), such as the comments
in WebOb ond Pyramid, whereas others (such as the IPSec RFC4868)
suggest or require that key lengths equal the digest size.

In any case, it seems convenient to default to SHA512 for our key
derivation and to use the digest length as the output length.

Additionally, I switched to using the HKDF class, instead of the
HKDFExpand class, in order to cover the fact that the input secret
supplied by the operator may not be cryptographically strong or may
not have well-distributed entropy.

Finally, I added a fixed, non-secret salt of 'h.security' which is
probably unnecessary with HKDF but means that if the operator is
using HKDF in other systems with the same input keying material
that the extract step will create a different master key, further
secured by the various info arguments to the derivations thereof.

I think this is probably pretty sound at this point.
latest commit 255b729c13
Randall Leeds tilgovi authored
Failed to load latest commit information.
docs
h Use SHA512 and perfom HKDF extract step
tests/js Use a singular time expression one year
.buildpacks Add Gemfiles and ruby buildpack
.gitattributes add versioneer
.gitignore Remove the leaning tower of python
.prospector.yaml Skip a broken pylint checker
.travis.yml Use key derivation to provide secret keys
AUTHORS Add myself to AUTHORS
CHANGES.txt Merge tag 'v0.3.2'
CHECKS Zero-downtime deploys on dokku
CONTRIBUTING.rst Update CONTRIBUTING.rst
Gemfile Add Gemfiles and ruby buildpack
Gemfile.lock Add Gemfiles and ruby buildpack
INSTALL.ArchLinux.rst Simplification & cleanup of INSTALL.* & README
INSTALL.Debian.rst Simplification & cleanup of INSTALL.* & README
INSTALL.Fedora.rst Simplification & cleanup of INSTALL.* & README
INSTALL.OSX.rst Simplification & cleanup of INSTALL.* & README
INSTALL.Ubuntu.rst whitespace
INSTALL.Vagrant.rst Simplification & cleanup of INSTALL.* & README
INSTALL.rst Simplification & cleanup of INSTALL.* & README
LICENSE Add KaTex to License and Notice
MANIFEST.in Include bootstrap and destroy scripts in MANIFEST
Makefile Restore "ic" removed by b5b3241
NOTICE Add KaTex to License and Notice
Procfile Add notification worker to Procfile
README.rst Use key derivation to provide secret keys
SCALE Add notification to SCALE file
Vagrantfile Simplified IP assignment
alembic.ini fix alembic logging
bootstrap Newest pip manages its own cache
development.ini Use key derivation to provide secret keys
gunicorn.conf.py Gunicorn understands WEB_CONCURRENCY itself
host.pem good news everyone! paste supports https easily
karma.config.js Small refactorings + tests for anchoring
package.json More view, sort, and threading refactoring
production.ini Use key derivation to provide secret keys
requirements.txt Move raven requirement to setup.py
run Upgrade to Gunicorn 19.2
runtime.txt Add runtime.txt to pin Python version
setup.cfg Add pep257 to dev dependencies; explain in README
setup.py Merge pull request #1981 from hypothesis/use-kdf-for-keys
test.ini Add test stub for mappings
versioneer.py Bump versioneer to v0.11

README.rst

Hypothesis

Build Status

About

Hypothesis brings community peer review to The Internet. It is a web application which enables rich annotation of web content. The project acts as a community aggregator for annotations and identity provider [*] for annotators. It also serves embed code for an annotation agent designed with modern web browsers in mind.

Installation

See INSTALL.rst for installation instructions. Platform specific instructions can be found in the install documents as well.

TL;DR

$ # install dependencies $ ./bootstrap $ ./run

Running

To run the server in development mode, simply use the run command:

$ ./run

This will start the server on port 5000 (http://localhost:5000), reload the application whenever changes are made to the source code, and restart it should it crash for some reason.

Note

Using the bookmarklet or otherwise embedding the application may not be possible on sites accessed via HTTPS due to browser policy restricting the inclusion of non-SSL content. If you wish to test the development server with these sites uncomment the line beginning with "certfile" in the development server configuration file, "development.ini". This will cause the server to use a self-signed certificate included for testing.

Development

The project builds heavily on components developed for the Annotator Project and is structured to encourage upstream contribution where possible. However, work is required to make Annotator more extensible in order to facilitate contribution.

Additionally, work is underway to support the data model described by the Open Annotation Core specification document.

Join us in #hypothes.is on freenode for discussion.

If you'd like to contribute to the project should also subscribe to the development mailing list and read about contributing. Then consider getting started on one of the issues that are ready for work. Issues tagged with the label 'New Contributor Friendly' are ideal for those just getting started.

Debugging

The pyramid_debugtoolbar package is loaded by default in the development environment. This will provide stack traces for exceptions and allow basic debugging. A more advanced profiler can also be accessed at the /_debug_toolbar path.

http://localhost:5000/_debug_toolbar/

Check out the pyramid_debugtoolbar documentation for information on how to use and configure it.

Code Quality

We run a variety of analysis tools on the python codebase using the prospector package. This is run by the CI on each push but can also be run manually via the lint make command:

$ make lint

Our linting is set to low by default. However, if you'd like to help dust it off a but more, you can run the prospector command directly and add the -s medium to get more nagging results from pep8, pylint, etc.

Additionally, we'd love help spiffing up our docstrings! You can give those a detailed look (and scrub!) by running:

$ ./bin/pep257 h

Or...for your own sanity, you can pass it the name of a specific file, instead of h (ex: h/api.py).

Testing

There are test suites for both the front- and back-end code.

To run the Python suite, invoke the tests in the standard fashion:

$ ./bin/python setup.py test

To run the JavaScript suite, ensure the assets are built and then invoke the karma test runner:

$ hypothesis assets development.ini
$ $(npm bin)/karma start karma.config.js --single-run

As a convenience, there is a make target which will do all of the above:

$ make test

It's also possible to run a subset of the tests using karma run:

$ $(npm bin)/karma start karma.config.js & # Start the server in the bg.
$ hypothesis assets development.ini && \
  $(npm bin)/karma run karma.config.js -- --grep={FILTER_STRING}

This will run generally be much faster than running karma start --single-run each time. The frontend tests can also be debugged by visiting http://localhost:9876/debug.html and opening the browser console.

Browser Extensions

Run the following command at the prompt to build the Chrome extension:

$ ./bin/hypothesis extension development.ini chrome http://localhost:5000

Or, to load the assets from within the extension:

$ ./bin/hypothesis extension development.ini chrome http://localhost:5000 \
chrome-extension://extensionid/public

To build an extension with a feature flag enabled use the environment variable:

FEATURE_NOTIFICATION=true \
hypothesis extension production.ini chrome \
https://hypothes.is chrome-extension://extensionid/public

To build the Firefox extension, run the following:

$ ./bin/hypothesis extension development.ini firefox \
http://localhost:5000 resource://firefox-at-hypothes-dot-is/hypothesis/data

If you are managing your virtual environment yourself, the script may not be located in the bin directory, but should be available in your path when the virtual environment is activated.

The fourth argument is the base URL for the application. An optional (for the Chrome extension), fifth argument may be passed to override the URL prefix used for static assets.

At this point, a working extension should exist in either ./build/chrome or ./build/firefox but with the development configuration the static assets are still loaded from the server. Start the application and ensure that the assets are built by visiting the home page or by running ./bin/hypothesis assets.

Note: Bundling the assets in the extension only works at the moment when the webassets.debug setting is falsy.

Deployment

Heroku

The project is set up to run out of the box on Heroku using these add-ons:

  • Heroku PostgreSQL
  • Mailgun, Mandrill, or SendGrid for sending e-mail
  • RedisToGo for session storage

Docker

The following docker link names are supported for automatic configuration of services:

  • elasticsearch
  • mail
  • nsqd
  • redis
  • statsd

Manual

The following shell environment variables are supported:

  • CLIENT_ID a unique API key for authentication
  • CLIENT_SECRET a unique API secret for signing authentication requests
  • DATABASE_URL in the format used by Heroku
  • ELASTICSEARCH_INDEX the Elasticsearch index for annotation storage
  • MAIL_DEFAULT_SENDER a sender address for outbound mail
  • SECRET_KEY a unique string secret

Customized embedding

By default, Hypothesis instantiates the Annotator.Host class defined in the injected code loaded by embed.js. It is possible to change this by assigning an alternate constructor to window.hypothesisRole. To customize the plugins that are loaded, define a function window.hypothesisConfig which returns an options object. This is then passed to the constructor as the second argument:

window.hypothesisConfig = function () {
  return {
    app: 'https://example.com/custom_sidebar_iframe',
    Toolbar: {container: '.toolbar-wrapper'}
  };
};

With the exception of app, the properties for the options object are the names of Annotator plugins and their values are the options passed to the individual plugin constructors.

The app property should be a url pointing to the HTML document that will be embedded in the page.

The full range of possibilities here is still in need of documentation and we would appreciate any help to improve that.

Documentation

To build the documentation, ensure that Sphinx is installed and issue the `make html` command from the docs directory:

$ cd docs/
$ make html

License

Hypothesis is released under the 2-Clause BSD License, sometimes referred to as the "Simplified BSD License" or the "FreeBSD License". Some third-party components are included. They are subject to their own licenses. All of the license information can be found in the included LICENSE file.

[*] Community and identity features are not finished. Get involved and help!
Something went wrong with that request. Please try again.