Migrate katcp to Python 3 #215

mmphego · 2019-09-11T15:23:28Z

This PR does the following:

JIRA: MT-737

Demo

Simulated 4A4k subarray benchmark

Original version of `katcp`

Migrated version of `katcp`

Simulated 5A4k(64 X-Fhosts) subarray benchmark

Original version of `katcp`

Migrated version of `katcp`

* Stage 1: Migrate katcp-python to Python 3 and ensure PY2 tests pass This PR does the following: - Updated `Jenkinsfile` and added `tox.ini` - Updated `setup.py` with changes similar to `katportalclient`. - Renamed `COPYING` to `LICENSE` and `CHANGELOG` to `CHANGELOG.md` - Added/Updated copyright and licence information on `*.py` files - Migrated `README.rst` to `README.md`, why - Markdown is better (Opinionated review) - Skipping `test_until_some_kwargs` test as it fails on tox, will investigate later. - Commented out static analysis, speed up ci - Added flake8 config on setup.cfg - Ran futurize on katcp-tests, added minor fixes for tests to pass - Tests passes on tox py27

…est pass

…ror and bytes vs str

…x escape \e

…ing Nonetypes and int's)

…pport py2&py3 compartibility at the same time supports different versions of tornado. This becomes tricky as tornado api has significant changes accross different versions

… protocol in order to you access original function object.

…reix 'test' as an actual 'test'

…joubertza

…joubertza had concerns mainly regarding multiple requests handling and memory usage

Raises TypeError as Py3 doesn't support comparing instances of an object and etc.

…object has no attribute 'TimeoutError'

…patiblity

User/mmphego/mt 737/stage3 make py3 pass

- Fixed `ValueError: Attempted relative import in non-package` when importing `docsrape` module. - Updated package installation instructions, fixed grammatic and header issues.

- Updated document hyperlinks - Bullet(s/ing) related fixes

…es not exist

- Inline emphasis start-string without end-string. - Unknown target name: "agg". - Block quote ends without a blank line; unexpected unindent. - Explicit markup ends without a blank line; unexpected unindent

@bmerry

- Removed `docs` generation dependencies from `setup.py`, and moved them to `tox.ini`. (see: https://github.com/ska-sa/katcp-python/pull/215/files#r325371314) - Added 'Py3.8' classifier as per @bmerry's recommendations. (see: https://github.com/ska-sa/katcp-python/pull/215/files#r336985014)

mmphego · 2019-10-24T13:06:42Z

Jenkinsfile

@@ -79,6 +79,19 @@ pipeline {
                }
            }
        }
+        stage('Generate documentation.') {


ajoubertza · 2019-10-25T13:47:43Z

AQF run on CAM OK. 1 minor change needed for a float comparison that was using == instead of a tolerance. So 123.00000000000001 != 123.0. This is due to using the repr for float conversion instead of "%.15g".

ajoubertza

Good to go! 🎉
(Only took 90 commits and 245 comments...)

Thanks to everyone that worked on this mega PR. We'll make a new version for PyPI soon.

tockards

🉑 ❗️

ofaletse · 2019-10-25T13:55:13Z

Woot!

mmphego · 2019-10-25T13:55:56Z

🍰 🍰 🍰 🍰 🍰 🍰 🍰 🍰 🍰

mmphego · 2019-10-25T20:20:07Z

CAM AQF Success!!!!

http://ci.camlab.kat.ac.za/blue/organizations/jenkins/ci.aqf.mkat/detail/ci.aqf.mkat/211/pipeline

Mpho Mphego and others added 23 commits September 5, 2019 12:34

Updated logic to make test_kattypes.py:TestDecorator.test_katcpv4 t…

4a30979

…est pass

Updated logic to make `test_kattypes.py1 pass, failures due to TypeEr…

f1db831

…ror and bytes vs str

Updated logic to make test_katcp_bnf.py pass, error due to bad rege…

b7201bd

…x escape \e

Unicode related errors fix for core.py and test_core.py

257a7f5

Fix for 'TestProtocolFlags' ( raise a TypeError exception when compar…

af4d98a

…ing Nonetypes and int's)

Fix error: 'WaitIterator' object is not an iterator on 'TestUntilSome'

d921072

assertEquals deprecated, using assertEqual

e3ce958

Handle tornado timeouts for various versions

fcad6ad

Pinning tornado=4.5.3, for obvious reasons - doesn't make sense to su…

826a8bf

…pport py2&py3 compartibility at the same time supports different versions of tornado. This becomes tricky as tornado api has significant changes accross different versions

Fixed test_fake_clients.py : Using .__dict__ to bypass the descriptor…

30c3751

… protocol in order to you access original function object.

Renamed base class and method, unittest considers class/method with p…

8e11c0f

…reix 'test' as an actual 'test'

dict.iteritems not supported in Py3, instead using dict.items. @a…

bee5965

…joubertza had concerns mainly regarding multiple requests handling and memory usage

built-in sorted() uses __cmp__ which break compartibility with Py2.

95dba2c

Raises TypeError as Py3 doesn't support comparing instances of an object and etc.

Ensure that data is of type byte before sending/writtin on sockets

7bc9b61

Better exception handling and reporting- >AttributeError: 'NoneType' …

8388f31

…object has no attribute 'TimeoutError'

Accessing bounded method instead of unbounded method, py2 and py3 com…

bd98ff2

…patiblity

Import fixes and minor cleanup

e711b0b

Ensure that data is of type byte before sending/writtin on sockets

95698b0

Removed unittest2 medule, unused

2d428ce

Added compartibility on example scripts

df78cd3

User/mmphego/mt 737/stage3 make py3 pass (#214)

c0263ae

User/mmphego/mt 737/stage3 make py3 pass

Removed supports for handling multiple versions of tornado

d7afc42

mmphego requested review from ajoubertza, sw00, amakhaba, kmadisa and lanceWilliams and removed request for ajoubertza September 11, 2019 15:23

mmphego added 6 commits October 24, 2019 08:09

Updated docs and doc generator.

8ec9752

- Fixed `ValueError: Attempted relative import in non-package` when importing `docsrape` module. - Updated package installation instructions, fixed grammatic and header issues.

Address sphinx document generation warnings.

df44fb0

- Updated document hyperlinks - Bullet(s/ing) related fixes

html_static_path entry u'/home/mmphego/CAM_Work/katcp/doc/_static' do…

b5ed19d

…es not exist

Docutils and sphinx warning fixes.

a64534d

- Inline emphasis start-string without end-string. - Unknown target name: "agg". - Block quote ends without a blank line; unexpected unindent. - Explicit markup ends without a blank line; unexpected unindent

Added document generation stage

9ef46e1

mmphego force-pushed the user/mmphego/MT-737/Add-Py2-Py3-compatibility branch from 2f1836f to 20b0714 Compare October 24, 2019 12:44

Fixed sphinx/docutils warnings regarding sections and indentations.

51650a7

mmphego force-pushed the user/mmphego/MT-737/Add-Py2-Py3-compatibility branch from 20b0714 to 51650a7 Compare October 24, 2019 12:59

mmphego commented Oct 24, 2019

View reviewed changes

Jenkinsfile

@@ -79,6 +79,19 @@ pipeline {

}

}

}

stage('Generate documentation.') {

Copy link

Contributor Author

mmphego Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmphego requested review from amakhaba, bngcebetsha, bxaia, kmadisa, lanceWilliams, samueltwum1, sw00 and tockards October 24, 2019 13:11

bmerry approved these changes Oct 25, 2019

View reviewed changes

ajoubertza approved these changes Oct 25, 2019

View reviewed changes

ludwigschwardt approved these changes Oct 25, 2019

View reviewed changes

tockards approved these changes Oct 25, 2019

View reviewed changes

mmphego merged commit 653d69e into master Oct 25, 2019

mmphego deleted the user/mmphego/MT-737/Add-Py2-Py3-compatibility branch October 25, 2019 20:22

ajoubertza removed Blocked Do Not Merge labels Oct 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate katcp to Python 3 #215

Migrate katcp to Python 3 #215

mmphego commented Sep 11, 2019 •

edited

Loading

mmphego Oct 24, 2019

ajoubertza commented Oct 25, 2019

ajoubertza left a comment

tockards left a comment

ofaletse commented Oct 25, 2019

mmphego commented Oct 25, 2019

mmphego commented Oct 25, 2019 •

edited

Loading

Migrate katcp to Python 3 #215

Migrate katcp to Python 3 #215

Conversation

mmphego commented Sep 11, 2019 • edited Loading

Demo

Simulated 4A4k subarray benchmark

Original version of katcp

Migrated version of katcp

Simulated 5A4k(64 X-Fhosts) subarray benchmark

Original version of katcp

Migrated version of katcp

mmphego Oct 24, 2019

Choose a reason for hiding this comment

ajoubertza commented Oct 25, 2019

ajoubertza left a comment

Choose a reason for hiding this comment

tockards left a comment

Choose a reason for hiding this comment

ofaletse commented Oct 25, 2019

mmphego commented Oct 25, 2019

mmphego commented Oct 25, 2019 • edited Loading

mmphego commented Sep 11, 2019 •

edited

Loading

Original version of `katcp`

Migrated version of `katcp`

Original version of `katcp`

Migrated version of `katcp`

mmphego commented Oct 25, 2019 •

edited

Loading