Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix py3 errors and warnings #748

Conversation

JeanChristopheMorinPerso
Copy link
Member

This is a batch of commits that fixes errors and warnings that we have in the master branch. Each commit contains a unique change.

  • There was an error with bez with python 3 due to the usage of iteritems.
  • Importing collections.MutableMapping is deprecated and it will stop to work in Python 3.8. The import needs to be collections.abc.MutableMapping instead.
  • When subclassing MutableMapping in python 3, we need to re-implement the abstract methods __delitem__, __iter__ and __len__.
  • Fixed a deprecation warning inside rezconfig.py that was to do with a \ in the docstring.
  • Fixed a crqash in platform_ due to the method decode that doesn't exit on type str in python 3.
  • Module imp is deprecated in Python 3. So I changed it to use importlib when in python 3.

I ran the tests under Linux with python 2.7, 3.4, 3.5, 3.6 and 3.7. If someone could confirm that everything is working in Windows, that would be great.

The only change I'm not sure of is the decode. No comments was made in the commit that introduced it to explain why it was needed.

Once that is merged, I'll do a pass at updating the vendored libs that cause other warnings.

@maxnbk
Copy link
Contributor

maxnbk commented Sep 16, 2019

The decode commit is causing this, for me, on linux, with python-3.6.1.
ignore the "2.33" in the traceback, that's just because I haven't updated my little "install this rez over here" convenience wrapper lately. It's from this branch.

Traceback (most recent call last):
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/tests/test_suites.py", line 100, in test_3
    c_foo = ResolvedContext(["foo"])
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/resolved_context.py", line 230, in __init__
    self.os = system.os
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/data_utils.py", line 191, in __get__
    result = self.func(instance)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/system.py", line 52, in os
    r = platform_.os
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/data_utils.py", line 191, in __get__
    result = self.func(instance)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/platform_mapped.py", line 31, in inner
    result = func(*args, **kwargs)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/platform_.py", line 33, in os
    return self._os()
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/platform_.py", line 236, in _os
    "Release:")
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.6/site-packages/rez/utils/platform_.py", line 204, in _parse
    lines = txt.strip().split('\n')
TypeError: a bytes-like object is required, not 'str'

@JeanChristopheMorinRodeoFX
Copy link
Contributor

That is extra weird... Maybe we could force convert it to a byte string?

@maxnbk
Copy link
Contributor

maxnbk commented Sep 16, 2019

This isn't a result of your changes, but I'm noticing a warning I was missing because it's not failing any tests, but is appearing very far at the top of the tests, specifically for the portion of tests run by rez-self --logging
This is on py3.6/3.7.

--- Logging error ---
Traceback (most recent call last):
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/utils/colorize.py", line 278, in emit
    message = self.format(record)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 869, in format
    return fmt.format(record)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 608, in format
    record.message = record.getMessage()
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not enough arguments for format string
Call stack:
  File "/sw/dev/smackenzie/rez/2.33.0.test/bin/rez/rez-selftest", line 8, in <module>
    sys.exit(run_rez_selftest())
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/cli/_entry_points.py", line 222, in run_rez_selftest
    return run("selftest")
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/cli/_main.py", line 152, in run
    returncode = run_cmd()
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/cli/_main.py", line 144, in run_cmd
    return opts.func(opts, opts.parser, extra_arg_groups)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/cli/selftest.py", line 69, in command
    main(module=None, argv=argv, verbosity=opts.verbose)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/main.py", line 101, in __init__
    self.runTests()
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/main.py", line 271, in runTests
    self.result = testRunner.run(self.test)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/runner.py", line 176, in run
    test(result)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 122, in run
    test(result)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 122, in run
    test(result)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/suite.py", line 122, in run
    test(result)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/case.py", line 676, in __call__
    return self.run(*args, **kwds)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/tests/test_logging.py", line 25, in test_print
    logging_.print_critical(msg, *nargs)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/utils/logging_.py", line 26, in print_critical
    logger.critical(msg, *nargs)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 1425, in critical
    self._log(CRITICAL, msg, args, **kwargs)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 1514, in _log
    self.handle(record)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 1524, in handle
    self.callHandlers(record)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 1586, in callHandlers
    hdlr.handle(record)
  File "/sw/packages/external/python/3.7.4/platform-linux/arch-x86_64/os-CentOS-7/lib/python3.7/logging/__init__.py", line 894, in handle
    self.emit(record)
  File "/sw/dev/smackenzie/rez/2.33.0.test/lib/python3.7/site-packages/rez/utils/colorize.py", line 292, in emit
    self.handleError(record)
Message: 'Hello %s %s'
Arguments: ('foo',)

@JeanChristopheMorinRodeoFX
Copy link
Contributor

Yeah, this one I already saw it. It's simply due to how the test is written. The same happens in python 2. I'll commit a fix for this as it's annoying.

@nerdvegas nerdvegas merged commit 38d1373 into AcademySoftwareFoundation:master Sep 17, 2019
@JeanChristopheMorinPerso JeanChristopheMorinPerso deleted the dev_fix_py3_errors_and_warnings branch September 22, 2019 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic:py3 Python 3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants