Skip to content

Commit

Permalink
[#2536] Installation and relese process doc tweaks
Browse files Browse the repository at this point in the history 
* Include setting `site_url` on the package install instructions to avoid getting an exception
* Improve the release process docs: overview, packaging, announce email, etc
  • Loading branch information
@amercader
amercader committed Jul 16, 2015
1 parent db19a58 commit 56d25db
Show file tree
Hide file tree
Showing 3 changed files with 117 additions and 35 deletions.
128 changes: 96 additions & 32 deletions doc/contributing/release-process.rst
View file
Expand Up @@ -11,6 +11,31 @@ new CKAN release.
An overview of the different kinds of CKAN release, and the process for
upgrading a CKAN site to a new version.

----------------
Process overview
----------------

The process of a new release starts with the creation of a new release branch.
A release branch is the one that will be stabilized and eventually become the actual
released version. Release branches are always named ``release-vM.m.p``, after the
:ref:`major, minor and patch versions <releases>` they include. Major and minor versions are
always branched from master. Patch releases are always branched from the most recent tip
of the previous patch release branch.

::

+--+---------------------------------------+-------------> Master
| |
+-----------------> release-v2.4.0 +----------> release-v2.5.0
|
+---------> release-v2.4.1
|
+------> release-v2.4.2

Once a release branch has been created there is generally a three-four week period until
the actual release. During this period the branch is tested and fixes cherry-picked. The whole
process is described in the following sections.


.. _beta-release:

Expand All @@ -23,14 +48,14 @@ become stable releases.

#. Create a new release branch::

git checkout -b release-v1.8
git checkout -b release-v2.5.0

Update ``ckan/__init__.py`` to change the version number to the new version
with a *b* after it, e.g. *1.8b*.
with a *b* after it, e.g. *2.5.0b*.
Commit the change and push the new branch to GitHub::

git commit -am "Update version number"
git push origin release-v1.8
git push origin release-v2.5.0

You will probably need to update the same file on master to increase the
version number, in this case ending with an *a* (for alpha).
Expand All @@ -53,8 +78,26 @@ become stable releases.

There will be a final front-end build before the actual release.

#. The beta staging site (http://beta.ckan.org, currently on s084) should be
updated regularly to allow user testing.
#. The beta staging site (http://beta.ckan.org, currently on s084) must be
set to track the latest beta release branch to allow user testing. This site
is updated nightly.

#. Once a week create a deb package with the latest release branch, using ``betaX``
iterations. Deb packages are built using Ansible_ scripts located at the
following repo:

https://github.com/ckan/ckan-packaging

The repository contains furhter instructions on how to run the scripts, but essentially
you will need access to the packaging server, and then run something like::

ansible-playbook package.yml -u your_user -s

You will be prompted for the CKAN version to package (eg ``2.4.0``), the iteration (eg ``beta1``)
and whether to package the DataPusher (always do it on release packages).

Packages are created by default on the `/build` folder of the publicly accessible directory of
the packaging server.

#. Once the translation freeze is in place (ie no changes to the translatable
strings are allowed), strings need to be extracted and uploaded to
Expand Down Expand Up @@ -96,7 +139,7 @@ become stable releases.

e. Edit ``.tx/config``, on line 4 to set the Transifex 'resource' to the new
major release name (if different), using dashes instead of dots.
For instance v1.2, v1.2.1 and v1.2.2 all share: ``[ckan.1-2]``.
For instance v2.4.0, v2.4.1 and v2.4.2 all share: ``[ckan.2-4]``.

f. Update the ``ckan.po`` files with the new strings from the ``ckan.pot`` file::

Expand Down Expand Up @@ -148,6 +191,11 @@ become stable releases.
git commit -am " Update translations from Transifex"
git push

#. A week before the actual release, send an email to the
`ckan-announce mailing list <http://lists.okfn.org/mailman/listinfo/ckan-announce>`_,
so CKAN instance maintainers can be aware of the upcoming releases. List any patch releases
that will be also available. Here's an `example <https://lists.okfn.org/pipermail/ckan-announce/2015-July/000013.html>`_ email.


----------------------
Doing a proper release
Expand All @@ -156,16 +204,16 @@ Doing a proper release
Once the release branch has been thoroughly tested and is stable we can do
a release.

1. Run the most thorough tests::
#. Run the most thorough tests::

nosetests ckan/tests --ckan --ckan-migration --with-pylons=test-core.ini

2. Do a final build of the front-end and commit the changes::
#. Do a final build of the front-end and commit the changes::

paster front-end-build
git commit -am "Rebuild front-end"

3. Update the CHANGELOG.txt with the new version changes:
#. Update the CHANGELOG.txt with the new version changes:

* Add the release date next to the version number
* Add the following notices at the top of the release, reflecting whether
Expand All @@ -176,7 +224,12 @@ a release.
Note: This version does not require a Solr schema upgrade

* Check the issue numbers on the commit messages for information about
the changes. These are some helpful git commands::
the changes. The following gist has a script that uses the GitHub API to
aid in getting the merged issues between releases:

https://gist.github.com/amercader/4ec55774b9a625e815bf

Other helpful commands are::

git branch -a --merged > merged-current.txt
git branch -a --merged ckan-1.8.1 > merged-previous.txt
Expand All @@ -185,23 +238,33 @@ a release.
git log --no-merges release-v1.8.1..release-v2.0
git shortlog --no-merges release-v1.8.1..release-v2.0

4. Check that the docs compile correctly::
#. Check that the docs compile correctly::

rm build/sphinx -rf
python setup.py build_sphinx

5. Remove the beta letter in the version number in ``ckan/__init__.py``
#. Remove the beta letter in the version number in ``ckan/__init__.py``
(eg 1.1b -> 1.1) and commit the change::

git commit -am "Update version number for release X.Y"

6. Tag the repository with the version number, and make sure to push it to
#. Tag the repository with the version number, and make sure to push it to
GitHub afterwards::

git tag -a -m '[release]: Release tag' ckan-X.Y
git push --tags

7. Upload the release to PyPI::
#. Create the final deb package and move it to the root of the
`publicly accessible folder <http://packaging.ckan.org/>`_ of
the packaging server from the `/build` folder.

Make sure to rename it so it follows the deb packages name convention::

python-ckan_Major.minor_amd64.deb

Note that we drop any patch version or iteration from the package name.

#. Upload the release to PyPI::

python setup.py sdist upload

Expand All @@ -211,7 +274,7 @@ a release.
If you make a mistake, you can always remove the release file on PyPI and
re-upload it.

8. Enable the new version of the docs on Read the Docs (you will need an admin
#. Enable the new version of the docs on Read the Docs (you will need an admin
account):

a. Go to the `Read The Docs`_ versions page
Expand All @@ -221,31 +284,32 @@ a release.
b. If it is the latest stable release, set it to be the Default Version and
check it is displayed on http://docs.ckan.org.

9. Write a `CKAN Blog post <http://ckan.org/wp-admin>`_ and send an email to
#. Write a `CKAN Blog post <http://ckan.org/wp-admin>`_ and send an email to
the mailing list announcing the release, including the relevant bit of
changelog.

10. Cherry-pick the i18n changes from the release branch onto master.
#. Cherry-pick the i18n changes from the release branch onto master.

Generally we don't merge or cherry-pick release branches into master, but
the files in ckan/i18n are an exception. These files are only ever changed
on release branches following the :ref:`beta-release` instructions above,
and after a release has been finalized the changes need to be cherry-picked
onto master.
We don't generally merge or cherry-pick release branches into master, but
the files in ckan/i18n are an exception. These files are only ever changed
on release branches following the :ref:`beta-release` instructions above,
and after a release has been finalized the changes need to be cherry-picked
onto master.

To find out what i18n commits there are on the release-v* branch that are
not on master, do::
To find out what i18n commits there are on the release-v* branch that are
not on master, do::

git log master..release-v* ckan/i18n
git log master..release-v* ckan/i18n

Then ``checkout`` the master branch, do a ``git status`` and a ``git pull``
to make sure you have the latest commits on master and no local changes.
Then use ``git cherry-pick`` when on the master branch to cherry-pick these
commits onto master. You should not get any merge conflicts. Run the
``check-po-files`` command again just to be safe, it should not report any
problems. Run CKAN's tests, again just to be safe. Then do ``git push
origin master``.
Then ``checkout`` the master branch, do a ``git status`` and a ``git pull``
to make sure you have the latest commits on master and no local changes.
Then use ``git cherry-pick`` when on the master branch to cherry-pick these
commits onto master. You should not get any merge conflicts. Run the
``check-po-files`` command again just to be safe, it should not report any
problems. Run CKAN's tests, again just to be safe. Then do ``git push
origin master``.


.. _Transifex: https://www.transifex.com/projects/p/ckan
.. _`Read The Docs`: http://readthedocs.org/dashboard/ckan/versions/
.. _Ansible: http://ansible.com/
2 changes: 2 additions & 0 deletions doc/maintaining/configuration.rst
View file
Expand Up @@ -53,6 +53,8 @@ details on how to this check :doc:`/extensions/remote-config-update`.



.. _config_file:

CKAN configuration file
***********************

Expand Down
22 changes: 19 additions & 3 deletions doc/maintaining/installing/install-from-package.rst
View file
Expand Up @@ -35,7 +35,7 @@ CKAN:

.. parsed-literal::
wget \http://packaging.ckan.org/|latest_package_name|
wget \http://packaging.ckan.org/|latest_package_name|
.. note:: If ``wget`` is not present, you can install it
via::
Expand Down Expand Up @@ -87,6 +87,22 @@ CKAN:
then edit the :ref:`sqlalchemy.url` option in your |production.ini| file and
set the correct password, database and database user.

-------------------------------------------------------
3. Update the configuration and initialize the database
-------------------------------------------------------

#. Edit the :ref:`config_file` (|production.ini|) to set up the following options:

site_id
Each CKAN site should have a unique ``site_id``, for example::

ckan.site_id = default

site_url
Provide the site's URL. For example::

ckan.site_url = http://demo.ckan.org

#. Initialize your CKAN database by running this command in a terminal::

sudo ckan db init
Expand All @@ -98,7 +114,7 @@ CKAN:
instructions in :doc:`/maintaining/filestore`.

---------------------------
3. Restart Apache and Nginx
4. Restart Apache and Nginx
---------------------------

Restart Apache and Nginx by running this command in a terminal::
Expand All @@ -107,7 +123,7 @@ Restart Apache and Nginx by running this command in a terminal::
sudo service nginx restart

---------------
4. You're done!
5. You're done!
---------------

Open http://localhost in your web browser. You should see the CKAN front
Expand Down

0 comments on commit 56d25db

Please sign in to comment.