[FW][FIX] core: propagate update of modifiers attributes from base terms #151932

fw-bot · 2024-01-30T23:27:18Z

When we update an inline translated element (like ) for the base
language en_US we should update the modifiers attributes for all
languages.

For example when updating from (16.0)
https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/views/ir_ui_view_views.xml#L127-L129
https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/i18n/fr.po#L7233-L7235
to (17.0)
https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/views/ir_ui_view_views.xml#L126-L128
https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/i18n/fr.po#L12087-L12089

The base term, en_US, is updated since the text matches but the
attributes of the translated terms, fr_FR for example, are not
updated. Later when the PO file is loaded in non-overwrite mode the
translated terms for fr_FR is not updated. This causes all sort of
issues during an upgrade for inline-translated terms -- like .
More so since the recent change that converts domain-based attributes
into inline Python expressions.

In this patch we propagate modifiers attributes from inline-translated
items in the new base term into all translated terms when the base term
is updated. In that way we ensure the attributes are correct in all
languages even if later the loading of their corresponding PO file
doesn't update the term.

For a detailed example, let's see what happens when loading the view
above during an upgrade 16->17, right at the first load of the XML file
at https://github.com/odoo/odoo/blob/b1461d28/odoo/fields.py#L1864

(Pdb) p old_term
'<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">This view has no previous version.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">This view is not coming from a file.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">You need two views to compare.</span>'
(Pdb) p closest_term
'<span invisible="reset_mode != \'soft\'">This view has no previous version.</span>\n                        <span invisible="reset_mode != \'hard\'">This view is not coming from a file.</span>\n                        <span invisible="reset_mode != \'other_view\'">You need two views to compare.</span>'
(Pdb) p translation_dictionary[old_term]
defaultdict(<class 'dict'>, {'fr_FR': '<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">Cette vue n\'a pas de version antérieure.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">Cette vue ne provient pas d\'un fichier.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">Vous avez besoin de deux vues pour comparer.</span>'})

As we can see the new term will get an updated value for its invisible
attribute, while also removing attrs. The translated terms will be
still keep the old modifier though.
Now later when the fr_FR.po file is loaded we reach this point
https://github.com/odoo/odoo/blob/b1461d28/odoo/tools/translate.py#L1442

(Pdb) p term_en
'<span invisible="reset_mode != \'soft\'">This view has no previous version.</span>\n                        <span invisible="reset_mode != \'hard\'">This view is not coming from a file.</span>\n                        <span invisible="reset_mode != \'other_view\'">You need two views to compare.</span>'
(Pdb) p translation_dictionary[term_en]
defaultdict(<function DeepDefaultDict at 0x7fc9640cdfc0>, {'fr_FR': '<span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'soft\')]}">Cette vue n\'a pas de version antérieure.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'hard\')]}">Cette vue ne provient pas d\'un fichier.</span>\n                        <span attrs="{\'invisible\': [(\'reset_mode\', \'!=\', \'other_view\')]}">Vous avez besoin de deux vues pour comparer.</span>'})

Thus the translated values are NOT updated, keeping the wrong
modifiers. This is later fixed during the upgrade in a clumsy way.
C.f. the warnings like this one in runbot:

Incomplete conversion for view(id=77, lang=fr_FR) at
<span attrs="{'invisible': [('reset_mode', '!=', 'soft')]}">Cette vue n'a pas de version ant&https://github.com/odoo-dev/odoo/pull/233;rieure.</span>

Note that such warnings are gone in current PR CI.
The root issue here is that when the fr_FR translation is loaded the
terms are not updated due to a combination of factors:

The text content of the term didn't change
There is no override flag set for translations

Option 2 is not a valid option during upgrades because we want to keep
custom translations. We could instead of the current patch tweak how
option 1 works and perhaps make the closest term more restricted. This
would lead to the update of the whole translation though while the
actual issue here is just the modifiers. Moreover if the translations
are out of sync the translated terms will still keep the wrong values
that could still be essential for the correct functioning of the record
they belong too (view archs -- for example).

Finally this is a more extreme case (16.0):
https://github.com/odoo/enterprise/blob/1e63b4a8/sale_subscription/views/sale_order_views.xml#L142
In this case during the upgrade we fix the modifier value (refer to
runbot warning above -- it's the same script that fixes it) and set

invisible="(subscription_management == 'upsell') or (recurrence_id == False)"

for translations, which is wrong. The correct value is (17.0):

invisible="not plan_id or subscription_state == '7_upsell'"

https://github.com/odoo/enterprise/blob/530ba3ad/sale_subscription/views/sale_order_views.xml#L136

Forward-Port-Of: #150152

`get_text_content` will transform contiguous space chars into single spaces, plus translate special HTML elements ``` >>> " ".join(html.fromstring(f"a\n b & c").text_content().split()) 'a b & c' ``` In order the correctly verify if a term is text-only we need to use the HTML parser. Note that to be resilient against bad XML, but valid HTML, we cannot use the default XML parser. X-original-commit: 86e3c3c

When we update an inline translated element (like ``) for the base language `en_US` we should update the modifiers attributes for all languages. For example when updating from (16.0) https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/views/ir_ui_view_views.xml#L127-L129 https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/i18n/fr.po#L7233-L7235 to (17.0) https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/views/ir_ui_view_views.xml#L126-L128 https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/i18n/fr.po#L12087-L12089 The base term, `en_US`, is updated since the text matches but the attributes of the translated terms, `fr_FR` for example, are not updated. Later when the PO file is loaded in non-overwrite mode the translated terms for `fr_FR` is not updated. This causes all sort of issues during an upgrade for inline-translated terms -- like ``. More so since the recent change that converts domain-based attributes into inline Python expressions. In this patch we propagate modifiers attributes from inline-translated items in the new base term into all translated terms when the base term is updated. In that way we ensure the attributes are correct in all languages even if later the loading of their corresponding PO file doesn't update the term. For a detailed example, let's see what happens when loading the view above during an upgrade 16->17, right at the first load of the XML file at https://github.com/odoo/odoo/blob/b1461d28/odoo/fields.py#L1864 ``` (Pdb) p old_term 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p closest_term 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p translation_dictionary[old_term] defaultdict(<class 'dict'>, {'fr_FR': 'Cette vue n\'a pas de version antérieure.\n Cette vue ne provient pas d\'un fichier.\n Vous avez besoin de deux vues pour comparer.'}) ``` As we can see the new term will get an updated value for its `invisible` attribute, while also removing `attrs`. The translated terms will be still keep the old modifier though. Now later when the fr_FR.po file is loaded we reach this point https://github.com/odoo/odoo/blob/b1461d28/odoo/tools/translate.py#L1442 ``` (Pdb) p term_en 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p translation_dictionary[term_en] defaultdict(<function DeepDefaultDict at 0x7fc9640cdfc0>, {'fr_FR': 'Cette vue n\'a pas de version antérieure.\n Cette vue ne provient pas d\'un fichier.\n Vous avez besoin de deux vues pour comparer.'}) ``` Thus the translated values are NOT updated, keeping the _wrong_ modifiers. This is later fixed during the upgrade in a clumsy way. C.f. the warnings like this one in runbot: ``` Incomplete conversion for view(id=77, lang=fr_FR) at Cette vue n'a pas de version antérieure. ``` Note that such warnings are gone in current PR CI. The root issue here is that when the fr_FR translation is loaded the terms are not updated due to a combination of factors: 1. The text content of the term didn't change 2. There is no override flag set for translations Option 2 is not a valid option during upgrades because we want to keep custom translations. We could instead of the current patch tweak how option 1 works and perhaps make the closest term more restricted. This would lead to the update of the whole translation though while the actual issue here is _just_ the modifiers. Moreover if the translations are out of sync the translated terms will still keep the wrong values that could still be essential for the correct functioning of the record they belong too (view archs -- for example). Finally this is a more extreme case (16.0): https://github.com/odoo/enterprise/blob/1e63b4a8/sale_subscription/views/sale_order_views.xml#L142 In this case during the upgrade we fix the modifier value (refer to runbot warning above -- it's the same script that fixes it) and set ``` invisible="(subscription_management == 'upsell') or (recurrence_id == False)" ``` for translations, which is wrong. The correct value is (17.0): ``` invisible="not plan_id or subscription_state == '7_upsell'" ``` https://github.com/odoo/enterprise/blob/530ba3ad/sale_subscription/views/sale_order_views.xml#L136 X-original-commit: ce3aac8

robodoo · 2024-01-30T23:28:12Z

.

fw-bot · 2024-01-30T23:28:13Z

@aj-fuentes @rco-odoo this PR targets master and is the last of the forward-port chain containing:

[FW][FIX] core: propagate update of modifiers attributes from base terms #151927

To merge the full chain, use

@fw-bot r+

More info at https://github.com/odoo/odoo/wiki/Mergebot#forward-port

fw-bot · 2024-01-30T23:35:15Z

@aj-fuentes @rco-odoo ci/runbot failed on this forward-port PR

aj-fuentes · 2024-01-31T10:52:18Z

@robodoo r+

aj-fuentes · 2024-01-31T10:52:40Z

@fw-bot r+

`get_text_content` will transform contiguous space chars into single spaces, plus translate special HTML elements ``` >>> " ".join(html.fromstring(f"a\n b & c").text_content().split()) 'a b & c' ``` In order the correctly verify if a term is text-only we need to use the HTML parser. Note that to be resilient against bad XML, but valid HTML, we cannot use the default XML parser. X-original-commit: 86e3c3c Part-of: #151932

When we update an inline translated element (like ``) for the base language `en_US` we should update the modifiers attributes for all languages. For example when updating from (16.0) https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/views/ir_ui_view_views.xml#L127-L129 https://github.com/odoo/odoo/blob/7ecd9413/odoo/addons/base/i18n/fr.po#L7233-L7235 to (17.0) https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/views/ir_ui_view_views.xml#L126-L128 https://github.com/odoo/odoo/blob/b1461d28/odoo/addons/base/i18n/fr.po#L12087-L12089 The base term, `en_US`, is updated since the text matches but the attributes of the translated terms, `fr_FR` for example, are not updated. Later when the PO file is loaded in non-overwrite mode the translated terms for `fr_FR` is not updated. This causes all sort of issues during an upgrade for inline-translated terms -- like ``. More so since the recent change that converts domain-based attributes into inline Python expressions. In this patch we propagate modifiers attributes from inline-translated items in the new base term into all translated terms when the base term is updated. In that way we ensure the attributes are correct in all languages even if later the loading of their corresponding PO file doesn't update the term. For a detailed example, let's see what happens when loading the view above during an upgrade 16->17, right at the first load of the XML file at https://github.com/odoo/odoo/blob/b1461d28/odoo/fields.py#L1864 ``` (Pdb) p old_term 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p closest_term 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p translation_dictionary[old_term] defaultdict(<class 'dict'>, {'fr_FR': 'Cette vue n\'a pas de version antérieure.\n Cette vue ne provient pas d\'un fichier.\n Vous avez besoin de deux vues pour comparer.'}) ``` As we can see the new term will get an updated value for its `invisible` attribute, while also removing `attrs`. The translated terms will be still keep the old modifier though. Now later when the fr_FR.po file is loaded we reach this point https://github.com/odoo/odoo/blob/b1461d28/odoo/tools/translate.py#L1442 ``` (Pdb) p term_en 'This view has no previous version.\n This view is not coming from a file.\n You need two views to compare.' (Pdb) p translation_dictionary[term_en] defaultdict(<function DeepDefaultDict at 0x7fc9640cdfc0>, {'fr_FR': 'Cette vue n\'a pas de version antérieure.\n Cette vue ne provient pas d\'un fichier.\n Vous avez besoin de deux vues pour comparer.'}) ``` Thus the translated values are NOT updated, keeping the _wrong_ modifiers. This is later fixed during the upgrade in a clumsy way. C.f. the warnings like this one in runbot: ``` Incomplete conversion for view(id=77, lang=fr_FR) at Cette vue n'a pas de version antérieure. ``` Note that such warnings are gone in current PR CI. The root issue here is that when the fr_FR translation is loaded the terms are not updated due to a combination of factors: 1. The text content of the term didn't change 2. There is no override flag set for translations Option 2 is not a valid option during upgrades because we want to keep custom translations. We could instead of the current patch tweak how option 1 works and perhaps make the closest term more restricted. This would lead to the update of the whole translation though while the actual issue here is _just_ the modifiers. Moreover if the translations are out of sync the translated terms will still keep the wrong values that could still be essential for the correct functioning of the record they belong too (view archs -- for example). Finally this is a more extreme case (16.0): https://github.com/odoo/enterprise/blob/1e63b4a8/sale_subscription/views/sale_order_views.xml#L142 In this case during the upgrade we fix the modifier value (refer to runbot warning above -- it's the same script that fixes it) and set ``` invisible="(subscription_management == 'upsell') or (recurrence_id == False)" ``` for translations, which is wrong. The correct value is (17.0): ``` invisible="not plan_id or subscription_state == '7_upsell'" ``` https://github.com/odoo/enterprise/blob/530ba3ad/sale_subscription/views/sale_order_views.xml#L136 closes #151932 X-original-commit: ce3aac8 Signed-off-by: Raphael Collet <rco@odoo.com> Signed-off-by: Alvaro Fuentes Suarez (afu) <afu@odoo.com>

randall-vx · 2024-02-05T06:20:37Z

@moylop260 maybe we can setup into a new lint in pre-commit-vauxoo.

aj-fuentes added 2 commits January 31, 2024 00:27

robodoo added the forwardport This PR was created by @fw-bot label Jan 30, 2024

C3POdoo added the RD research & development, internal work label Jan 31, 2024

robodoo temporarily deployed to merge February 1, 2024 14:20 Inactive

robodoo closed this Feb 1, 2024

robodoo added the 17.2 label Feb 1, 2024

aj-fuentes deleted the master-17.0-fix_update_unstranslated_attribs-afu-pZeu-fw branch February 1, 2024 14:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[FW][FIX] core: propagate update of modifiers attributes from base terms #151932

[FW][FIX] core: propagate update of modifiers attributes from base terms #151932

Uh oh!

fw-bot commented Jan 30, 2024

Uh oh!

robodoo commented Jan 30, 2024 •

edited

Loading

Uh oh!

fw-bot commented Jan 30, 2024

Uh oh!

fw-bot commented Jan 30, 2024

Uh oh!

aj-fuentes commented Jan 31, 2024

Uh oh!

aj-fuentes commented Jan 31, 2024

Uh oh!

randall-vx commented Feb 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

[FW][FIX] core: propagate update of modifiers attributes from base terms #151932

[FW][FIX] core: propagate update of modifiers attributes from base terms #151932

Uh oh!

Conversation

fw-bot commented Jan 30, 2024

Uh oh!

robodoo commented Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fw-bot commented Jan 30, 2024

Uh oh!

fw-bot commented Jan 30, 2024

Uh oh!

aj-fuentes commented Jan 31, 2024

Uh oh!

aj-fuentes commented Jan 31, 2024

Uh oh!

randall-vx commented Feb 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

robodoo commented Jan 30, 2024 •

edited

Loading