From 56d25db60210d856aee8420485125bdc76a5b7f4 Mon Sep 17 00:00:00 2001 From: amercader Date: Thu, 16 Jul 2015 11:44:42 +0100 Subject: [PATCH] [#2536] Installation and relese process doc tweaks * Include setting `site_url` on the package install instructions to avoid getting an exception * Improve the release process docs: overview, packaging, announce email, etc --- doc/contributing/release-process.rst | 128 +++++++++++++----- doc/maintaining/configuration.rst | 2 + .../installing/install-from-package.rst | 22 ++- 3 files changed, 117 insertions(+), 35 deletions(-) diff --git a/doc/contributing/release-process.rst b/doc/contributing/release-process.rst index ff83535b07d..ea57e55361d 100644 --- a/doc/contributing/release-process.rst +++ b/doc/contributing/release-process.rst @@ -11,6 +11,31 @@ new CKAN release. An overview of the different kinds of CKAN release, and the process for upgrading a CKAN site to a new version. +---------------- +Process overview +---------------- + +The process of a new release starts with the creation of a new release branch. +A release branch is the one that will be stabilized and eventually become the actual +released version. Release branches are always named ``release-vM.m.p``, after the +:ref:`major, minor and patch versions ` they include. Major and minor versions are +always branched from master. Patch releases are always branched from the most recent tip +of the previous patch release branch. + + :: + + +--+---------------------------------------+-------------> Master + | | + +-----------------> release-v2.4.0 +----------> release-v2.5.0 + | + +---------> release-v2.4.1 + | + +------> release-v2.4.2 + +Once a release branch has been created there is generally a three-four week period until +the actual release. During this period the branch is tested and fixes cherry-picked. The whole +process is described in the following sections. + .. _beta-release: @@ -23,14 +48,14 @@ become stable releases. #. Create a new release branch:: - git checkout -b release-v1.8 + git checkout -b release-v2.5.0 Update ``ckan/__init__.py`` to change the version number to the new version - with a *b* after it, e.g. *1.8b*. + with a *b* after it, e.g. *2.5.0b*. Commit the change and push the new branch to GitHub:: git commit -am "Update version number" - git push origin release-v1.8 + git push origin release-v2.5.0 You will probably need to update the same file on master to increase the version number, in this case ending with an *a* (for alpha). @@ -53,8 +78,26 @@ become stable releases. There will be a final front-end build before the actual release. -#. The beta staging site (http://beta.ckan.org, currently on s084) should be - updated regularly to allow user testing. +#. The beta staging site (http://beta.ckan.org, currently on s084) must be + set to track the latest beta release branch to allow user testing. This site + is updated nightly. + +#. Once a week create a deb package with the latest release branch, using ``betaX`` + iterations. Deb packages are built using Ansible_ scripts located at the + following repo: + + https://github.com/ckan/ckan-packaging + + The repository contains furhter instructions on how to run the scripts, but essentially + you will need access to the packaging server, and then run something like:: + + ansible-playbook package.yml -u your_user -s + + You will be prompted for the CKAN version to package (eg ``2.4.0``), the iteration (eg ``beta1``) + and whether to package the DataPusher (always do it on release packages). + + Packages are created by default on the `/build` folder of the publicly accessible directory of + the packaging server. #. Once the translation freeze is in place (ie no changes to the translatable strings are allowed), strings need to be extracted and uploaded to @@ -96,7 +139,7 @@ become stable releases. e. Edit ``.tx/config``, on line 4 to set the Transifex 'resource' to the new major release name (if different), using dashes instead of dots. - For instance v1.2, v1.2.1 and v1.2.2 all share: ``[ckan.1-2]``. + For instance v2.4.0, v2.4.1 and v2.4.2 all share: ``[ckan.2-4]``. f. Update the ``ckan.po`` files with the new strings from the ``ckan.pot`` file:: @@ -148,6 +191,11 @@ become stable releases. git commit -am " Update translations from Transifex" git push +#. A week before the actual release, send an email to the + `ckan-announce mailing list `_, + so CKAN instance maintainers can be aware of the upcoming releases. List any patch releases + that will be also available. Here's an `example `_ email. + ---------------------- Doing a proper release @@ -156,16 +204,16 @@ Doing a proper release Once the release branch has been thoroughly tested and is stable we can do a release. -1. Run the most thorough tests:: +#. Run the most thorough tests:: nosetests ckan/tests --ckan --ckan-migration --with-pylons=test-core.ini -2. Do a final build of the front-end and commit the changes:: +#. Do a final build of the front-end and commit the changes:: paster front-end-build git commit -am "Rebuild front-end" -3. Update the CHANGELOG.txt with the new version changes: +#. Update the CHANGELOG.txt with the new version changes: * Add the release date next to the version number * Add the following notices at the top of the release, reflecting whether @@ -176,7 +224,12 @@ a release. Note: This version does not require a Solr schema upgrade * Check the issue numbers on the commit messages for information about - the changes. These are some helpful git commands:: + the changes. The following gist has a script that uses the GitHub API to + aid in getting the merged issues between releases: + + https://gist.github.com/amercader/4ec55774b9a625e815bf + + Other helpful commands are:: git branch -a --merged > merged-current.txt git branch -a --merged ckan-1.8.1 > merged-previous.txt @@ -185,23 +238,33 @@ a release. git log --no-merges release-v1.8.1..release-v2.0 git shortlog --no-merges release-v1.8.1..release-v2.0 -4. Check that the docs compile correctly:: +#. Check that the docs compile correctly:: rm build/sphinx -rf python setup.py build_sphinx -5. Remove the beta letter in the version number in ``ckan/__init__.py`` +#. Remove the beta letter in the version number in ``ckan/__init__.py`` (eg 1.1b -> 1.1) and commit the change:: git commit -am "Update version number for release X.Y" -6. Tag the repository with the version number, and make sure to push it to +#. Tag the repository with the version number, and make sure to push it to GitHub afterwards:: git tag -a -m '[release]: Release tag' ckan-X.Y git push --tags -7. Upload the release to PyPI:: +#. Create the final deb package and move it to the root of the + `publicly accessible folder `_ of + the packaging server from the `/build` folder. + + Make sure to rename it so it follows the deb packages name convention:: + + python-ckan_Major.minor_amd64.deb + + Note that we drop any patch version or iteration from the package name. + +#. Upload the release to PyPI:: python setup.py sdist upload @@ -211,7 +274,7 @@ a release. If you make a mistake, you can always remove the release file on PyPI and re-upload it. -8. Enable the new version of the docs on Read the Docs (you will need an admin +#. Enable the new version of the docs on Read the Docs (you will need an admin account): a. Go to the `Read The Docs`_ versions page @@ -221,31 +284,32 @@ a release. b. If it is the latest stable release, set it to be the Default Version and check it is displayed on http://docs.ckan.org. -9. Write a `CKAN Blog post `_ and send an email to +#. Write a `CKAN Blog post `_ and send an email to the mailing list announcing the release, including the relevant bit of changelog. -10. Cherry-pick the i18n changes from the release branch onto master. +#. Cherry-pick the i18n changes from the release branch onto master. - Generally we don't merge or cherry-pick release branches into master, but - the files in ckan/i18n are an exception. These files are only ever changed - on release branches following the :ref:`beta-release` instructions above, - and after a release has been finalized the changes need to be cherry-picked - onto master. + We don't generally merge or cherry-pick release branches into master, but + the files in ckan/i18n are an exception. These files are only ever changed + on release branches following the :ref:`beta-release` instructions above, + and after a release has been finalized the changes need to be cherry-picked + onto master. - To find out what i18n commits there are on the release-v* branch that are - not on master, do:: + To find out what i18n commits there are on the release-v* branch that are + not on master, do:: - git log master..release-v* ckan/i18n + git log master..release-v* ckan/i18n - Then ``checkout`` the master branch, do a ``git status`` and a ``git pull`` - to make sure you have the latest commits on master and no local changes. - Then use ``git cherry-pick`` when on the master branch to cherry-pick these - commits onto master. You should not get any merge conflicts. Run the - ``check-po-files`` command again just to be safe, it should not report any - problems. Run CKAN's tests, again just to be safe. Then do ``git push - origin master``. + Then ``checkout`` the master branch, do a ``git status`` and a ``git pull`` + to make sure you have the latest commits on master and no local changes. + Then use ``git cherry-pick`` when on the master branch to cherry-pick these + commits onto master. You should not get any merge conflicts. Run the + ``check-po-files`` command again just to be safe, it should not report any + problems. Run CKAN's tests, again just to be safe. Then do ``git push + origin master``. .. _Transifex: https://www.transifex.com/projects/p/ckan .. _`Read The Docs`: http://readthedocs.org/dashboard/ckan/versions/ +.. _Ansible: http://ansible.com/ diff --git a/doc/maintaining/configuration.rst b/doc/maintaining/configuration.rst index dab12ee9895..57830ff41c8 100644 --- a/doc/maintaining/configuration.rst +++ b/doc/maintaining/configuration.rst @@ -53,6 +53,8 @@ details on how to this check :doc:`/extensions/remote-config-update`. +.. _config_file: + CKAN configuration file *********************** diff --git a/doc/maintaining/installing/install-from-package.rst b/doc/maintaining/installing/install-from-package.rst index 78d680fc638..c9dd9be0817 100644 --- a/doc/maintaining/installing/install-from-package.rst +++ b/doc/maintaining/installing/install-from-package.rst @@ -35,7 +35,7 @@ CKAN: .. parsed-literal:: - wget \http://packaging.ckan.org/|latest_package_name| + wget \http://packaging.ckan.org/|latest_package_name| .. note:: If ``wget`` is not present, you can install it via:: @@ -87,6 +87,22 @@ CKAN: then edit the :ref:`sqlalchemy.url` option in your |production.ini| file and set the correct password, database and database user. +------------------------------------------------------- +3. Update the configuration and initialize the database +------------------------------------------------------- + +#. Edit the :ref:`config_file` (|production.ini|) to set up the following options: + + site_id + Each CKAN site should have a unique ``site_id``, for example:: + + ckan.site_id = default + + site_url + Provide the site's URL. For example:: + + ckan.site_url = http://demo.ckan.org + #. Initialize your CKAN database by running this command in a terminal:: sudo ckan db init @@ -98,7 +114,7 @@ CKAN: instructions in :doc:`/maintaining/filestore`. --------------------------- -3. Restart Apache and Nginx +4. Restart Apache and Nginx --------------------------- Restart Apache and Nginx by running this command in a terminal:: @@ -107,7 +123,7 @@ Restart Apache and Nginx by running this command in a terminal:: sudo service nginx restart --------------- -4. You're done! +5. You're done! --------------- Open http://localhost in your web browser. You should see the CKAN front