Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make letsencrypt-auto release-suitable, self-upgrading, and self-contained. #1665

Merged
merged 112 commits into from Jan 19, 2016
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
8ba831a
WIP. Here's a letsencrypt-auto script that downloads a new copy of it…
erikrose Nov 30, 2015
ec415b2
Add a sig to test against.
erikrose Nov 30, 2015
1a8f40e
This works now, to the point where it calls the downloaded version of…
erikrose Nov 30, 2015
a75c743
Compute latest stable version of letsencrypt properly.
erikrose Nov 30, 2015
602e977
Stop catching exception types that are no longer thrown by get().
erikrose Nov 30, 2015
7fb9295
Find a better semantic for HumanException.
erikrose Nov 30, 2015
2c36f59
Return a temp dir, not the file within.
erikrose Dec 1, 2015
86203c8
Add peep and sample requirements file.
erikrose Dec 1, 2015
fe77da2
Unquote heredoc terminators.
erikrose Dec 1, 2015
e3ace6f
Split large independent scripts off from the main body of the proof-o…
erikrose Dec 2, 2015
4abe7ab
Merge master in to get up to date.
erikrose Dec 2, 2015
ec9a498
Move OS-package bootstrappers to a private folder.
erikrose Dec 2, 2015
cdd855c
Add build script for letsencrypt-auto.
erikrose Dec 2, 2015
66436c5
le-auto now doesn't trigger sh syntax errors when run.
erikrose Dec 2, 2015
f9d1de6
Remove test signature, which I shouldn't have committed.
erikrose Dec 2, 2015
9d6cbea
Fix some errors.
erikrose Dec 2, 2015
3f0bcb5
Add real requirements, suitable as of ab9051ff09ef69a6cdf272deaa6e7df…
erikrose Dec 2, 2015
a1b2626
Print the final letsencrypt invocation before doing it.
erikrose Dec 2, 2015
346ec58
Add visual separators between language changes.
erikrose Dec 2, 2015
4a69584
Standardize semicolon use.
erikrose Dec 2, 2015
fc52608
Rewrap some comments.
erikrose Dec 2, 2015
5bae8e0
Install not only LE's dependencies but LE itself.
erikrose Dec 2, 2015
be6c34d
Make --no-self-upgrade public.
erikrose Dec 2, 2015
02255fa
Upgrade peep to 2.5, for compatibility with pip 7.x.
erikrose Dec 3, 2015
46779da
In Phase 2, recreate a venv and reinstall Python packages only if nec…
erikrose Dec 3, 2015
5cc69d9
In Phase 1, download a new letsencrypt-auto script only if necessary.
erikrose Dec 3, 2015
55a52d1
"none" is clearer than "0.0.0" as a sentinel value.
erikrose Dec 4, 2015
4bcd594
Put off rm-ing the venv for as long as possible, since it triggers a …
erikrose Dec 4, 2015
4a44c46
Add a header for peep errors...
erikrose Dec 4, 2015
6db54e2
Correct length of dividers.
erikrose Dec 4, 2015
1da5e47
Put quotes around variables that might contain spaces.
erikrose Dec 4, 2015
8b2c5cb
Update LE package pins to 0.1.0, the public beta.
erikrose Dec 4, 2015
0c4a7bb
Make le-auto pull the requisite things from env vars so we can run ag…
erikrose Dec 9, 2015
d9cde2b
Get the first end-to-end test of letsencrypt-auto passing.
erikrose Dec 18, 2015
7e04f52
Add built letsencrypt-auto.
erikrose Jan 5, 2016
e6cece5
Document le-auto env vars.
erikrose Jan 5, 2016
cad4e98
Merge in master to get up to date.
erikrose Jan 5, 2016
fa30625
Update the built version of letsencrypt-auto.
erikrose Jan 5, 2016
5f694e3
Create a fake release in a PyPI-style json file
pde Jan 5, 2016
7d182c2
Substitute test-only values for the env vars.
erikrose Jan 5, 2016
726376f
A real release signture key
pde Jan 5, 2016
14d3d4a
Merge branch 'letsencrypt-auto-release-testing' of ssh://github.com/l…
pde Jan 5, 2016
d0bbe44
Switch to a throwaway testing key
pde Jan 5, 2016
b2a0142
First attempt at signing with a throwaway key
pde Jan 5, 2016
9181271
Change version of le-auto script to the one published in the pypi.json.
erikrose Jan 5, 2016
0f78753
Swap _ for - so the phase-1 upgrade doesn't 404.
erikrose Jan 5, 2016
1ad21f9
Update signature!
pde Jan 5, 2016
56bda20
Survive unsuccessful apt-get update...
pde Jan 6, 2016
f4011cc
Move sudo to the top
pde Jan 6, 2016
8bb0631
Updated versions
bmw Jan 6, 2016
88c4260
Merge branch 'letsencrypt-auto-release-testing' of ssh://github.com/l…
pde Jan 6, 2016
b008770
Resign
pde Jan 6, 2016
0a122cb
Try baking in this 0.1.22 thing
pde Jan 6, 2016
404de84
Update le-auto requirements file to fake LE 0.1.22 release.
erikrose Jan 6, 2016
484b032
Add hashes for new cffi 1.3.1 packages.
erikrose Jan 6, 2016
d83dda8
Rebuild and re-sign le-auto.
erikrose Jan 6, 2016
275d3b4
Swap _ for - so the phase-1 upgrade doesn't 404.
erikrose Jan 5, 2016
5aa9fe9
Survive unsuccessful apt-get update...
pde Jan 6, 2016
8a3bbf9
Move sudo to the top
pde Jan 6, 2016
4940ee2
Add hashes for new cffi 1.3.1 packages.
erikrose Jan 6, 2016
ba6bf45
Update pinning of LE packages to 0.1.1.
erikrose Jan 6, 2016
4fd9d39
Mark changes in letsencrypt-auto-release-testing as having been incor…
erikrose Jan 6, 2016
762709a
Remove needless message about reusing venv. Rebuild le-auto.
erikrose Jan 6, 2016
4b075df
Cut down mock PyPI dir listing HTML.
erikrose Jan 7, 2016
98b3c41
Add le-auto tests for "no upgrade needed" and "only a phase-2 upgrade…
erikrose Jan 7, 2016
e5e5c2d
Don't stomp on the in-tree le-auto during tests.
erikrose Jan 7, 2016
134b7ab
Add a test for when openssl signature verification fails during phase…
erikrose Jan 7, 2016
bb31d71
Add a test for failed hash verification during phase-2 upgrade.
erikrose Jan 8, 2016
1d719bd
Teach le-auto about dependencies that are conditional on the Python v…
erikrose Jan 8, 2016
cd43e90
Rename letsencrypt_auto dir to match other dirs.
erikrose Jan 8, 2016
f2586fb
Merge master in so Travis will test this PR.
erikrose Jan 8, 2016
5512838
Get le-auto tests running on Travis.
erikrose Jan 8, 2016
2eb3e09
Check correct signature presence for release
pde Jan 11, 2016
7cfb10b
These signatures should be in git
pde Jan 11, 2016
7a8507d
helpful documentation
pde Jan 11, 2016
bbd53d6
Ensure we have an leauto signature before releasing
pde Jan 11, 2016
0c09eaf
Switch to real release key
pde Jan 11, 2016
d915f63
Add tool for requesting & handling offline signatures
pde Jan 11, 2016
e17bb27
Remove test key
pde Jan 11, 2016
2f3425a
Switch to real key, and add signing to release script. Close #1573.
erikrose Jan 11, 2016
be653e8
Use SHA256 openssl signatures
pde Jan 11, 2016
916f891
Clearer notes about when / how to edit the script
pde Jan 11, 2016
1b3c8e8
Better processing & documentation of leauto flags
pde Jan 11, 2016
66ca744
Take le-auto tests out of Travis until we figure out why sudo:require…
erikrose Jan 11, 2016
6c05197
Remove mock as an install requirement.
erikrose Jan 11, 2016
ed56264
Master master into letsencrypt-auto-release so Travis will build it.
erikrose Jan 12, 2016
7ee23b7
Get all tests, even le_auto, working on Travis.
erikrose Jan 12, 2016
a3288a9
Disable too-many-instance-attributes for the acme linter.
erikrose Jan 12, 2016
cb5beb8
Fix Fedora 23 crasher.
erikrose Jan 12, 2016
3abf028
Merge branch 'letsencrypt-auto-release' of https://github.com/erikros…
pde Jan 12, 2016
a7ae436
Bring built le-auto script up to date.
erikrose Jan 12, 2016
2d4c21a
Bring built le-auto script up to date.
erikrose Jan 12, 2016
435dfc0
Undelete the old letsencrypt-auto for now
pde Jan 12, 2016
e192cce
Fix fake letsencrypt
bmw Jan 13, 2016
7945db7
Rebuild sdist
bmw Jan 13, 2016
ab07620
Fixed fake letsencrypt hash
bmw Jan 13, 2016
86266f5
Remove backported Python 2.7 assertion helpers.
erikrose Jan 13, 2016
d813097
Upgrade half-sign to sha256. Bring back old le-auto temporarily. Impr…
erikrose Jan 13, 2016
587e2e7
Revert "Get all tests, even le_auto, working on Travis."
bmw Jan 13, 2016
a1f6678
Revert changes to Dockerfile
bmw Jan 13, 2016
a287b50
Fix Vagrantfile path
bmw Jan 13, 2016
bccb212
Fix paths in contributing.rst
bmw Jan 13, 2016
c3ea4bd
Roll back change to acme's pylintrc, which was needed to get lint to …
erikrose Jan 13, 2016
25e428c
Bring built le-auto up to date again.
erikrose Jan 13, 2016
2771249
Merge master in before computing a known-good set for 0.2.0.
erikrose Jan 13, 2016
ecbe2a5
Merge master in to get the unconditionalization of dependencies.
erikrose Jan 15, 2016
1706619
Update known-good-set, and make deps unconditional.
erikrose Jan 15, 2016
e1bd164
Revert moving mock to test_requires.
erikrose Jan 15, 2016
e923901
Add mock==1.0.1, the Python 2.6 compatible version, to le-auto reqs.
erikrose Jan 15, 2016
aefd5b2
Revert switch to `python setup.py test` in tox.ini.
erikrose Jan 19, 2016
b20eab6
Remove errant DS_Store. Ick.
erikrose Jan 19, 2016
3c1ba61
Remove nosetests -v option from setup.cfg, and add trailing newline.
erikrose Jan 19, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions Dockerfile
Expand Up @@ -22,8 +22,8 @@ WORKDIR /opt/letsencrypt
# directories in its path.


COPY bootstrap/ubuntu.sh /opt/letsencrypt/src/ubuntu.sh
RUN /opt/letsencrypt/src/ubuntu.sh && \
COPY letsencrypt_auto/letsencrypt-auto /opt/letsencrypt/src/letsencrypt-auto
RUN /opt/letsencrypt/src/letsencrypt-auto --os-packages-only && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* \
/tmp/* \
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile-dev
Expand Up @@ -22,8 +22,8 @@ WORKDIR /opt/letsencrypt

# TODO: Install non-default Python versions for tox.
# TODO: Install Apache/Nginx for plugin development.
COPY bootstrap/ubuntu.sh /opt/letsencrypt/src/ubuntu.sh
RUN /opt/letsencrypt/src/ubuntu.sh && \
COPY letsencrypt_auto/letsencrypt-auto /opt/letsencrypt/src/letsencrypt-auto
RUN /opt/letsencrypt/src/letsencrypt-auto --os-packages-only && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* \
/tmp/* \
Expand Down
2 changes: 1 addition & 1 deletion Vagrantfile
Expand Up @@ -7,7 +7,7 @@ VAGRANTFILE_API_VERSION = "2"
# Setup instructions from docs/contributing.rst
$ubuntu_setup_script = <<SETUP_SCRIPT
cd /vagrant
./bootstrap/install-deps.sh
./letsencrypt-auto/letsencrypt-auto --os-packages-only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bad path

./bootstrap/dev/venv.sh
SETUP_SCRIPT

Expand Down
27 changes: 0 additions & 27 deletions bootstrap/_arch_common.sh

This file was deleted.

50 changes: 0 additions & 50 deletions bootstrap/_deb_common.sh

This file was deleted.

23 changes: 0 additions & 23 deletions bootstrap/_gentoo_common.sh

This file was deleted.

49 changes: 0 additions & 49 deletions bootstrap/_rpm_common.sh

This file was deleted.

14 changes: 0 additions & 14 deletions bootstrap/_suse_common.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/archlinux.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/centos.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/debian.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/fedora.sh

This file was deleted.

8 changes: 0 additions & 8 deletions bootstrap/freebsd.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/gentoo.sh

This file was deleted.

46 changes: 0 additions & 46 deletions bootstrap/install-deps.sh

This file was deleted.

18 changes: 0 additions & 18 deletions bootstrap/mac.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/manjaro.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/suse.sh

This file was deleted.

1 change: 0 additions & 1 deletion bootstrap/ubuntu.sh

This file was deleted.

58 changes: 10 additions & 48 deletions docs/contributing.rst
Expand Up @@ -22,7 +22,7 @@ once:

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./bootstrap/install-deps.sh
./letsencrypt-auto/letsencrypt-auto --os-packages-only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect path.

./bootstrap/dev/venv.sh

Then in each shell where you're working on the client, do:
Expand Down Expand Up @@ -359,75 +359,37 @@ Now run tests inside the Docker image:
Notes on OS dependencies
========================

OS level dependencies are managed by scripts in ``bootstrap``. Some notes
are provided here mainly for the :ref:`developers <hacking>` reference.
OS-level dependencies can be installed like so:

In general:
.. code-block:: shell

letsencrypt-auto/letsencrypt-auto --os-packages-only
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think we should just make it simpler and revert all changes to this file for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can be confident in the le-auto-based bootstrapping at this point. It's textually almost identical to the old scripts, and we've vetted it on the test farm and in the community. Furthermore, we're talking to devs here, and they should be using the latest, not least because they know how to file bugs at us if it doesn't work. :-)


In general...

* ``sudo`` is required as a suggested way of running privileged process
* `Augeas`_ is required for the Python bindings
* ``virtualenv`` and ``pip`` are used for managing other python library
dependencies

What follow are OS-specific notes for the :ref:`developers <hacking>` reference.

.. _Augeas: http://augeas.net/
.. _Virtualenv: https://virtualenv.pypa.io

Ubuntu
------

.. code-block:: shell

sudo ./bootstrap/ubuntu.sh


Debian
------

.. code-block:: shell

sudo ./bootstrap/debian.sh

For squeeze you will need to:

- Use ``virtualenv --no-site-packages -p python`` instead of ``-p python2``.


.. _`#280`: https://github.com/letsencrypt/letsencrypt/issues/280


Mac OSX
-------

.. code-block:: shell

./bootstrap/mac.sh


Fedora
------

.. code-block:: shell

sudo ./bootstrap/fedora.sh


Centos 7
--------

.. code-block:: shell

sudo ./bootstrap/centos.sh


FreeBSD
-------

.. code-block:: shell

sudo ./bootstrap/freebsd.sh

Bootstrap script for FreeBSD uses ``pkg`` for package installation,
i.e. it does not use ports.
Package installation for FreeBSD uses ``pkg``, not ports.

FreeBSD by default uses ``tcsh``. In order to activate virtualenv (see
below), you will need a compatible shell, e.g. ``pkg install bash &&
Expand Down