New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enh] Migrate moulinette to python 3 #172

Open
wants to merge 28 commits into
base: stretch-unstable
from

Conversation

Projects
None yet
2 participants
@zamentur
Copy link
Contributor

zamentur commented Oct 5, 2018

The problem

Python 2 will reach its end of support in 1 year.

Solution

Convert moulinette into python3

PR Status

WIP

How to test

apt install python3-yaml python3-bottle python3-gnupg python3-argcomplete python3-psutil
rm -Rf /usr/lib/python3/dist-packages/moulinette; cp -R /vagrant/moulinette/moulinette  /usr/lib/python3/dist-packages/moulinette
cp /vagrant/moulinette/data/actionsmap/test.yml /usr/share/moulinette/actionsmap/ 
mkdir /usr/lib/moulinette/test
cp /vagrant/moulinette/lib/test/test.py /usr/lib/moulinette/test/ 
cp /vagrant/moulinette/bin/test-moulinette /usr/bin/test-moulinette
chmod u+x /usr/bin/test-moulinette

# Install ldap python3 module
apt install python3-pyldap

test-moulinette test non-auth

Validation

  • Principle agreement 0/2 :
  • Quick review 0/1 :
  • Simple test 0/1 :
  • Deep review 0/1 :

zamentur and others added some commits Oct 4, 2018

Introducing first test for moulinette/utils/filesystem
- try to read a non existent file raises an Exception
Introducing first test for moulinette/utils/filesystem
- try to read a non existent file raises an Exception
Tests for utils.filesystem.read_file
- non existent file
- non readable file
- readable file with content

Introducing a fake_open function to easily use mocking plumbery for
mocking the opening and reading of text files in tests.
Testing utils.filesystem.read_json
- read proper json file return python value from json
- read improper json file raise an MoulinetteError
Improving style of utils.test_filesystem
- documenting the use of fake_open
- improving readability of read_file tests
Testing utils.filesystem.read_yaml
- reading a yaml file returns its content
- reading a badly formatted yaml file raise an MoulinetteError

yaml may return yaml.scanner.ScannerError in addition to ValueError on
improper content of the yaml file.
Tests for utils.filesystem.write_to_file
- non existent folder
- non readable file
- writable file with content

Introducing a fake_open_for_write helper function to easily use mocking
plumbery for mocking the opening and writing of text files in tests.

Renamed fake_open helper to fake_open_for_read (one helper similarly
named for read and for write)
Testing utils.filesystem.write_to_file
- rolled back to existing code from exception to assertions in function
- added tests for various situations with base folder and file being
  existant or not and being folders or not
Testing utils.filesystem.write_to_json
- test writing of dict to json file
- test writing of list to json file
- test error on writing something else

Updated the implementation to re-use write_to_file for proper writing of
content instead of duplicating code
@zamentur

This comment has been minimized.

Copy link
Contributor

zamentur commented Oct 5, 2018

root@yunohost:~# caniusepython3 --projects /vagrant/moulinette/moulinette
Finding and checking dependencies ...
[WARNING] problem fetching /vagrant/moulinette/moulinette, assuming ported (404)

🎉 You have 0 projects blocking you from using Python 3!

@zamentur

This comment has been minimized.

Copy link
Contributor

zamentur commented Oct 5, 2018

Is this dependencies available in python 3 ?

  • python-ldap,
  • python-yaml,
  • python-bottle (>= 0.12),
  • python-gnupg,
  • python-gevent-websocket,
  • python-argcomplete,
  • python-psutil

Is this dependencies available in stretch debian repo ?

  • [NOK] python-ldap,
  • [?] python-gevent-websocket,

zamentur and others added some commits Oct 5, 2018

Testing utils.filesystem.rm
- removing existing file with proper permissions
- removing existing folder with proper permissions

- non existant or no permissions prevent removing file
- non existant or no permissions prevent removing folder

Fixed potential OSError handling from shutil.rmtree
Testing utils.filesystem.chmod
- changing existing file mode with proper permissions
- changing existing folder mode recursively with proper permissions
- changing existing folder mode recursively with mode for files and mode
  for folders with proper permissions

- non existant or no permissions prevent changing mode of file / folder
@alexAubin

This comment has been minimized.

Copy link
Member

alexAubin commented Oct 5, 2018

ytkwrkk

Raphael Marvie and others added some commits Oct 5, 2018

Testing utils.filesystem.chown
- changing existing file owner with proper permissions
- changing existing folder owner recursively with proper permissions

- non existant or no permission prevent changing owner of file / folder

- new test for reading yaml from file, raising a parser error

Updated read_yaml to take care of ParserError from yaml lib
Testing utils.filesystem.mkdir
- creating single folder
- creating folder with parent
- creating folder with specific owner

- no permission prevent creating folder

Update filesyste.mkdir to not raise FileExistsError when force=True is
used as parameter.
Testing utils.filesystem.mkdir
- creating single folder
- creating folder with parent
- creating folder with specific owner
- creating existing folder do not fail if using force=True

- existance of folder prevent creation if not using force
- no permission prevent creating folder

Update filesyste.mkdir to not raise FileExistsError when force=True is
used as parameter.

@alexAubin alexAubin added this to the Futur milestone Oct 31, 2018

@alexAubin alexAubin changed the title Enh python 3 [enh] Migrate moulinette to python 3 Nov 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment