Skip to content

Commit 0158404

Browse files
augustoPerezaugustoPerez
committed
[ADD] Upgrade documentation: Migration scripts and Util package
1 parent e6a17c0 commit 0158404

File tree

3 files changed

+365
-2
lines changed

3 files changed

+365
-2
lines changed

‎content/developer/howtos/upgrade/migration_scripts.rst‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
=================
2+
Migration scripts
3+
=================
4+
5+
A migration script is a Python file containing a function called :file:`migrate`, which the upgrade
6+
process invokes at the appropriate time.
7+
8+
The :file:`migrate` function receives two parameters:
9+
10+
- :file:`cr`: The current database cursor.
11+
- :file:`version`: The installed version of the module.
12+
13+
Typically, this function executes one or multiple SQL queries and can also access Odoo's ORM, as
14+
well as the :doc:`../upgrade/upgrade_util`.
15+
16+
As described in :ref:`upgrade_custom/upgraded_database/migrate_data`, you might have to use
17+
migration scripts to reflect changes from the source code to their corresponding data.
18+
19+
20+
Writing migration scripts
21+
=========================
22+
23+
To write what we refer as a **custom migration script**, the path of the file should follow this
24+
template: :file:`<module_name>/migrations/<major_version>.<minor_version>/{pre|post|end}-*.py`.
25+
26+
Follow this steps to create the directory's structure and the Python file:
27+
28+
#. Create a :file:`migrations` directory inside the custom module.
29+
#. Create a :file:`version` directory with the format: :file:`<major_version>.<minor_version>`
30+
inside the :file:`migrations` directory.
31+
32+
- :file:`<major_version>` corresponds to the major version of Odoo (e.g., :file:`16.0` for
33+
Odoo 16).
34+
- :file:`<minor_version>` corresponds to the updated version declared on the module's
35+
manifest.
36+
37+
Migration scripts are only executed when the module is being updated. Therefore, the
38+
:file:`minor_version` set in the directory needs to be higher than the module's installed
39+
version on the database and equal or lower to the updated version of the module.
40+
For example, in an `Odoo 16` database, with a custom module installed in version `1.1`, and an
41+
updated module version equal to `1.2`; the version directory should be `16.0.1.2`.
42+
#. Create a :file:`Python file` inside the :file:`version` directory named according to the
43+
:ref:`Phase <upgrade/migration-scripts/phases>` on which it needs to be executed. It should
44+
follow the template `{pre|post|end}-*.py`. The file name will determine the order in which it
45+
is executed for that module, version and phase.
46+
#. Create a :file:`migrate` function in the Python file that receives as parameters
47+
:file:`(cr, version)`.
48+
#. Add the logic needed inside the Python file.
49+
50+
.. example::
51+
52+
Directory structure of a migration script for a custom module named `awesome_partner` upgraded
53+
to version `2.0.0` on Odoo 17
54+
55+
.. code-block:: text
56+
57+
awesome_partner/
58+
|-- migrations/
59+
| |-- 17.0.2.0.0/
60+
| | |-- pre-exclamation.py
61+
62+
Two migration scripts examples with the content of the :file:`pre-exclamation.py`, file adding
63+
"!" at the end of partners' names:
64+
65+
.. code-block:: python
66+
67+
import logging
68+
69+
_logger = logging.getLogger(__name__)
70+
71+
72+
def migrate(cr, version):
73+
cr.execute("UPDATE res_partner SET name = name || '!'")
74+
_logger.info("Updated %s partners", cr.rowcount)
75+
76+
.. code-block:: python
77+
78+
import logging
79+
from odoo.upgrade import util
80+
81+
_logger = logging.getLogger(__name__)
82+
83+
84+
def migrate(cr, version):
85+
env = util.env(cr)
86+
87+
partners = env["res.partner"].search([])
88+
for partner in partners:
89+
partner.name += "!"
90+
91+
_logger.info("Updated %s partners", len(partners))
92+
93+
Note that in the second example, the script takes advantage of the :doc:`../upgrade/upgrade_util`
94+
to access the ORM. Check the documentation to find out more about this library.
95+
96+
97+
Running and testing migration scripts
98+
=====================================
99+
100+
.. tabs::
101+
102+
.. group-tab:: Odoo Online
103+
104+
As the instalation of custom modules containing Python files is not allowed on Odoo Online
105+
databases, it is not possible to run migration scripts on this platform.
106+
107+
.. group-tab:: Odoo.sh
108+
109+
As explained on the `Odoo.sh` tab of :ref:`upgrade/request-test-database`, Odoo.sh is
110+
integrated with the upgrade platform.
111+
112+
Once the upgrade of a staging branch is on "Update on commit" mode, each time a commit is
113+
pushed on the branch, the upgraded backup is restored and all the custom modules are updated.
114+
This update includes the execution of the migration scripts.
115+
116+
When upgrading the production database, the execution of the migration scripts is also part of
117+
the update of the custom modules done by the platform when the upgraded database is restored.
118+
119+
.. group-tab:: On-premise
120+
121+
Once you receive the upgraded dump of the database from the `Upgrade platform
122+
<https://upgrade.odoo.com>`_, deploy the database and update all the custom modules by
123+
invoking the command :doc:`odoo-bin </developer/reference/cli>` in the shell.
124+
To update the custom modules, use the option: `-u <modules>,
125+
--update <modules>`.
126+
127+
.. important::
128+
As mentioned in the :doc:`CLI documentation </developer/reference/cli>`, the command used
129+
to call the CLI depends on how you installed Odoo.
130+
131+
132+
.. _upgrade/migration-scripts/phases:
133+
134+
Phases of migration scripts
135+
===========================
136+
137+
The upgrade process consists of three phases for each version of each module:
138+
139+
#. The pre-phase, before the module is loaded.
140+
#. The post-phase, after the module and its dependencies are loaded and upgraded.
141+
#. The end-phase, after all modules have been loaded and upgraded for that version.
142+
143+
Migration scripts are grouped according to the first part of their filenames into the corresponding
144+
phase. Within each phase, the files are executed according to their lexical order.
145+
146+
.. note::
147+
If you are unsure which phase to use, use the end-phase.
148+
149+
.. spoiler:: Execution order of example scripts for one module in one version
150+
151+
- :file:`pre-10-do_something.py`
152+
- :file:`pre-20-something_else.py`
153+
- :file:`post-do_something.py`
154+
- :file:`post-something.py`
155+
- :file:`end-01-migrate.py`
156+
- :file:`end-migrate.py`

‎content/developer/howtos/upgrade/upgrade_util.rst‎

Lines changed: 187 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
====================
2+
Upgrade Util package
3+
====================
4+
5+
The `Upgrade Util package <https://github.com/odoo/upgrade-util/>`_ is a library that contains
6+
helper functions to facilitate the writing of upgrade scripts. This package, used by Odoo for the
7+
migration scripts of standard modules, provides reliability and helps speed up the upgrade process:
8+
9+
.. todo:: Check if needed - and syntaxis
10+
11+
- The helper functions make sure the data is consitent in your database.
12+
- It takes care of indirect references.
13+
- Allows calling functions and avoid writing code.
14+
- Helpers allow to focus on what is important for the upgrade and not think of details.
15+
16+
17+
Installation
18+
============
19+
20+
Once you have clone this repository locally, just start `odoo` with the `src` directory prepended to
21+
the `--upgrade-path` option.
22+
23+
.. code-block:: console
24+
25+
$ ./odoo-bin --upgrade-path=/path/to/upgrade-util/src,/path/to/other/upgrade/script/directory [...]
26+
27+
On platforms where you do not manage Odoo yourself, you can install this package via `pip`:
28+
29+
.. code-block:: console
30+
31+
$ python3 -m pip install git+https://github.com/odoo/upgrade-util@master
32+
33+
On `Odoo.sh <https://www.odoo.sh/>`_ it is recommended to add it to the :file:`requirements.txt` of
34+
your repository. For this, add the following line inside the file::
35+
36+
odoo_upgrade @ git+https://github.com/odoo/upgrade-util@master
37+
38+
Using the Util package
39+
======================
40+
41+
Once installed, the following packages are available for the migration scripts:
42+
43+
- :file:`odoo.upgrade.util`: the helper themself.
44+
- :file:`odoo.upgrade.testing`: base TestCase classes.
45+
46+
To use it migration scripts, simply import it:
47+
48+
.. code-block:: python
49+
50+
from odoo.upgrade import util
51+
52+
53+
def migrate(cr, version):
54+
# Rest of the script
55+
56+
Now, the helper functions are available to be called through `util`.
57+
58+
Util functions
59+
==============
60+
61+
The util package provides many useful functions to ease the upgrade process. Here, we will describe
62+
some of the most useful ones. You can find them by going to the `upgrade-util repository
63+
<https://github.com/odoo/upgrade-util/tree/master/src/util>`_.
64+
65+
.. todo:: Check how to add technical information
66+
.. Link like official Odoo doc, to repo and auto-generated ?
67+
68+
.. todo:: How to display the information?
69+
.. todo:: What functions to show?
70+
71+
Fields
72+
------
73+
74+
- remove_field
75+
- move_field_to_module
76+
- rename_field
77+
- convert_field_to_html
78+
79+
Models
80+
------
81+
82+
- remove_model
83+
- rename_model
84+
- merge_model
85+
86+
Modules
87+
-------
88+
89+
- remove_module
90+
- rename_module
91+
- merge_module
92+
93+
ORM
94+
---
95+
96+
- env
97+
- recompute_fields
98+
99+
Misc
100+
----
101+
102+
- skippable_cm
103+
104+
PostgreSQL
105+
----------
106+
107+
- parallel_execute
108+
- explode_query_range
109+
- create_column
110+
- column_exists
111+
- remove_column
112+
- table_exists
113+
- create_index
114+
- rename_table
115+
- create_m2m
116+
117+
Records
118+
-------
119+
120+
- remove_record
121+
- remove_menus
122+
- remove_group
123+
- rename_xmlid
124+
- ref
125+
- ensure_xmlid_match_record
126+
- update_record_from_xml
127+
- reset_cowed_views
128+
- convert_jinja_to_qweb
129+
130+
.. tabs::
131+
132+
.. tab:: Fields
133+
134+
- remove_field
135+
- move_field_to_module
136+
- rename_field
137+
- convert_field_to_html
138+
139+
.. tab:: Models
140+
141+
:attr:`remove_model`
142+
remove_model explanation
143+
:attr:`rename_model`
144+
rename_model explanation
145+
:attr:`merge_model`
146+
merge_model explanation
147+
148+
.. tab:: Modules
149+
150+
- remove_module
151+
- rename_module
152+
- merge_module
153+
154+
.. tab:: ORM
155+
156+
- env
157+
- recompute_fields
158+
159+
.. tab:: Misc
160+
161+
- skippable_cm
162+
163+
.. tab:: PostgreSQL
164+
165+
- parallel_execute
166+
- explode_query_range
167+
- create_column
168+
- column_exists
169+
- remove_column
170+
- table_exists
171+
- create_index
172+
- rename_table
173+
- create_m2m
174+
175+
.. tab:: Records
176+
177+
- remove_record
178+
- remove_menus
179+
- remove_group
180+
- rename_xmlid
181+
- ref
182+
- ensure_xmlid_match_record
183+
- update_record_from_xml
184+
- reset_cowed_views
185+
- convert_jinja_to_qweb
186+
187+
.. todo:: Add examples

0 commit comments

Comments
 (0)