Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration to Python3 #2203

Closed
thomasdanan opened this issue Jan 23, 2020 · 0 comments · Fixed by #2839
Closed

Migration to Python3 #2203

thomasdanan opened this issue Jan 23, 2020 · 0 comments · Fixed by #2839
Assignees
Labels
complexity:hard Something that may require up to a week to fix kind:enhancement New feature or request kind:epic High-level description of a feature iteration priority:medium Medium priority issues, should only be postponed if no other option python Pull requests that update Python code topic:salt Everything related to SaltStack in our product

Comments

@thomasdanan
Copy link
Contributor

thomasdanan commented Jan 23, 2020

TBD

Component:

Why this is needed:

What should be done:

  • Make salt custom modules, states, runners, pillars, ... works with python3
  • Install Salt python3
  • Update Salt unit test to work in python3

Implementation proposal (strongly recommended):

Test plan:

@thomasdanan thomasdanan added moonshot kind:epic High-level description of a feature iteration labels Jan 23, 2020
@TeddyAndrieux TeddyAndrieux self-assigned this Oct 8, 2020
@TeddyAndrieux TeddyAndrieux added complexity:hard Something that may require up to a week to fix kind:enhancement New feature or request priority:medium Medium priority issues, should only be postponed if no other option python Pull requests that update Python code topic:deployment Bugs in or enhancements to deployment stages topic:salt Everything related to SaltStack in our product kind:epic High-level description of a feature iteration and removed kind:epic High-level description of a feature iteration topic:deployment Bugs in or enhancements to deployment stages labels Oct 8, 2020
TeddyAndrieux added a commit that referenced this issue Oct 9, 2020
Make all custom salt modules compatible with Python3 using six:
- replace `basestring` with `six.string_types`
- replace `urlparse` with `six.moves.urllib.parse`
- do not try to decode string object
- do not convert string to unicode
- do not use `yum` Python API as it's only available with Python2
- encode and decode string object when using `base64`

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Oct 9, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
@TeddyAndrieux TeddyAndrieux mentioned this issue Oct 9, 2020
7 tasks
TeddyAndrieux added a commit that referenced this issue Oct 12, 2020
Make all custom salt modules compatible with Python3 using six:
- replace `basestring` with `six.string_types`
- replace `urlparse` with `six.moves.urllib.parse`
- do not try to decode string object
- do not convert string to unicode
- do not use `yum` Python API as it's only available with Python2
- encode and decode string object when using `base64`

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Oct 12, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Oct 12, 2020
Make all custom salt modules, states, ... compatible with Python3 using
six:
- replace `basestring` with `six.string_types`
- replace `urlparse` with `six.moves.urllib.parse`
- do not try to decode string object
- do not convert string to unicode
- do not use `yum` Python API as it's only available with Python2
- encode and decode string object when using `base64`

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Oct 12, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Oct 12, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Oct 13, 2020
Make all custom salt modules, states, ... compatible with Python3 using
six:
- replace `basestring` with `six.string_types`
- replace `urlparse` with `six.moves.urllib.parse`
- do not try to decode string object
- do not convert string to unicode
- do not use `yum` Python API as it's only available with Python2
- encode and decode string object when using `base64`

NOTE: We do this change before migrating to Python3, so that futur
downgrade will be simpler

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 19, 2020
With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 19, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Nov 20, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 20, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 23, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Add an entry in changelog file about Salt migration to Python3 and bump
to Salt version 3002.2

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 26, 2020
Add an entry in changelog file about Salt migration to Python3 and bump
to Salt version 3002.2

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Change salt-master image to use Python3 salt-master package and also
install Python3 dependencies, embed Python3 salt-minion version instead
of Python2.

In order to install Python3 salt-minion we need:
- to install python3
- to install python36-rpm as by default version comparaison for package
  installation is wrong
- to install python3 on nodes for being able to use `salt-ssh`

Sees: saltstack/salt#58039
Sees: saltstack/salt#57972
Fixes: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Add an entry in changelog file about Salt migration to Python3 and bump
to Salt version 3002.2

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
With Python3 we can no longer use `cmp` in `sort` and `sorted` function
so use our salt module `metalk8s.cmp_sorted` in Downgrade orchestrate if
we are running with Python3, so that it works well with Python2 and
Pyton3

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Since we migrate from Python2 to Python3 a "classic" salt states cannot
handle it properly as the Python version change during the state
execution.
Add a dedicated orchestrate that handle the migration from Salt Python2
to Salt Python3 and also the migration from Salt Python3 to Salt
Python2, call this new orchestrate during Upgrade and Downgrade if
needed

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Since we migrate to Salt Python3 we have Jinja 2.11 so we can
use `loop.previtem` instead of ugly hack using a jinja variable.
NOTE: This hack is still needed in downgrade orchestrate when we
downgrade to `2.6.x` as we are running with Salt master metalk8s-2.6.x
that use Salt Python2 and Jinja 2.7

Refs: #2203
TeddyAndrieux added a commit that referenced this issue Nov 27, 2020
Add an entry in changelog file about Salt migration to Python3 and bump
to Salt version 3002.2

Refs: #2203
@bert-e bert-e closed this as completed in b53d936 Nov 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
complexity:hard Something that may require up to a week to fix kind:enhancement New feature or request kind:epic High-level description of a feature iteration priority:medium Medium priority issues, should only be postponed if no other option python Pull requests that update Python code topic:salt Everything related to SaltStack in our product
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants