Python should support VxWorks RTOS

[kuhlenough]

BPO	31904
Nosy	@gpshead, @vstinner, @tiran, @bitdancer, @asvetlov, @ambv, @JimJJewett, @izbyshev, @kuhlenough, @miss-islington, @pxinwr
PRs	bpo-31904 : Python should support VxWorks RTOS  #4179 bpo-31904 : Add support for VxWorks RTOS #4184 bpo-31904: Add support for VxWorks RTOS #11951 bpo-31904: Add cross-build support for VxWorks RTOS #11968 bpo-31904: Add encoding support for VxWorks RTOS #12051 bpo-31904: Add posix module support for VxWorks RTOS #12118 bpo-31904: Add subprocess module support for VxWorks RTOS #12157 bpo-31904: Adapt the _signal module to VxWorks RTOS #12304 bpo-31904: Add time module support for VxWorks RTOS #12305 bpo-31904: Fix test_os failures for VxWorks RTOS #12319 bpo-31904: Add _crypt module support for VxWorks RTOS #12321 bpo-31904: fix test_utf8_mode fail issue on VxWorks #12428 bpo-31904: Fix file not exist error string #12646 bpo-31904 : skip test_cmd_line case for VxWorks doesn't support preexec #12648 bpo-31904: fix signalmodule issue in VxWorks #12670 bpo-31904: Skip setting RLIMIT_FSIZE and RLIMIT_CPU test case for VxWorks #12719 bpo-31904: remove _crypt extension for VxWorks RTOS #12833 bpo-31904: Only UTF-8 encoding is supported on VxWorks #13486 bpo-31904: Skip zoneinfo tests on VxWorks #13535 bpo-31904: Increase LOOPBACK_TIMEOUT to 10 for VxWorks RTOS #19447 bpo-31904: set expected default encoding in test_c_locale_coercion.py for VxWorks RTOS #19448 bpo-31904: fix fifo test cases for VxWorks RTOS #20254 bpo-31904: Fix test_getcwd_long_path failures for VxWorks RTOS #20256 bpo-31904: Fix test_netrc for VxWorks RTOS #21675 bpo-31904: Fix site and sysconfig modules for VxWorks RTOS  #21821 bpo-31904: support signal module for VxWorks #23391 bpo-31904: remove libnet dependency from detect_socket() for VxWorks #23394 bpo-31904: fix test_doctest.py failures for VxWorks #23419 bpo-31904: skip some tests related to fifo on VxWorks #23473 bpo-31904: disable os.popen and impacted test cases on VxWorks #21687 bpo-31904: add shell requirement for test_pipes #23489 bpo-31904: skip test_test of test_mailcap on VxWorks #23507 bpo-31904: skip test_getaddrinfo_ipv6_scopeid_symbolic and test_getnameinfo_ipv6_scopeid_symbolic on VxWorks #23518 bpo-31904: posixpath.expanduser() handles user home of None #23530 bpo-31904: skip some tests of changing owner in _test_all_chown_common() on VxWorks #23716 bpo-31904: Define THREAD_STACK_SIZE for VxWorks #23718 bpo-31904: Enable libpython3.so share library for VxWorks #23741 bpo-31904: Skip test_expanduser and test_expanduser_pwd on VxWorks #23776 bpo-31904: Skip some asyncio tests on VxWorks #23815 bpo-27640: Add --disable-test-modules configure option #23886 bpo-44097: add --enable-pyc-build option to the configure script #23920 bpo-31904: add library search path by wr-cc in add_cross_compiling_paths() #24191 bpo-31904: Correct error string in test_file_not_exists() for VxWorks #25965 [3.10] bpo-31904: Correct error string in test_file_not_exists() for VxWorks (GH-25965) #25983

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-10-30.19:56:33.760>
labels = ['type-feature', '3.10', 'build']
title = 'Python should support VxWorks RTOS'
updated_at = <Date 2021-05-08.11:20:17.167>
user = 'https://github.com/kuhlenough'

bugs.python.org fields:

activity = <Date 2021-05-08.11:20:17.167>
actor = 'Alex.Willmer'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2017-10-30.19:56:33.760>
creator = 'Brian Kuhl'
dependencies = []
files = []
hgrepos = []
issue_num = 31904
keywords = ['patch']
message_count = 46.0
messages = ['305249', '305511', '305657', '305690', '305869', '305877', '315365', '336680', '336742', '337083', '337867', '337909', '338964', '339059', '339898', '340244', '340245', '340310', '340385', '340406', '340409', '343004', '343008', '366073', '374361', '374460', '374469', '382014', '382020', '382021', '382023', '382198', '382218', '382276', '382525', '382793', '382807', '383009', '383096', '383097', '383098', '383226', '383259', '383467', '385461', '393247']
nosy_count = 11.0
nosy_names = ['gregory.p.smith', 'vstinner', 'christian.heimes', 'r.david.murray', 'asvetlov', 'lukasz.langa', 'Jim.Jewett', 'izbyshev', 'Brian Kuhl', 'miss-islington', 'pxinwr']
pr_nums = ['4179', '4184', '11951', '11968', '12051', '12118', '12157', '12304', '12305', '12319', '12321', '12428', '12646', '12648', '12670', '12719', '12833', '13486', '13535', '19447', '19448', '20254', '20256', '21675', '21821', '23391', '23394', '23419', '23473', '21687', '23489', '23507', '23518', '23530', '23716', '23718', '23741', '23776', '23815', '23886', '23920', '24191', '25965', '25983']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31904'
versions = ['Python 3.10']

[kuhlenough]

With the trend to use REST APIs between the cloud and the IoT there is increasing interest in Python on embedded devices. Cloud developer’s typical release a reference REST implementation of JSON and/or Python on Linux and leave it to the device developer to adapt it to their platform. While many devices use eLinux, others with IP and/or hard real-time constraints need a commercial RTOS platform.

Currently the automake configure explicitly prevents configuration of VxWorks as a build target.

I'll provide a pull request referencing this issue with the required changes.

[terryjreedy]

The following might be relevant to this issue:
https://www.python.org/dev/peps/pep-0011/#supporting-platforms

[vstinner]

To support a new platform, you need a developer who can support this platform next years, a working buildbot, etc. You can start a discussion on python-dev to get a first feedback.

Without a strong support, this issue should be fixed a REJECTED and a patch should be maintainted out of the tree. Since the PR seems small, it should be "easy" to keep a fork of CPython up to date.

[vstinner]

FYI I already started a thread on python-dev:
[Python-Dev] Partial support of a platform
https://mail.python.org/pipermail/python-dev/2017-November/150238.html

[kuhlenough]

I'm quite happy to take on maintainer role for Python on VxWorks, so I think we can get that one solved.

Enabling a build bot for cross compile of propitiatory OS presents a number of legal licensing issues that outside my control. And I'll discuss it internally at Wind River. However I think it is in line with where our customers want us to go, so well worth pursuing.

I'll keep this pull request active and up to date, till the broader issues you have raised can be resolved.

I'll post a proposal on the mailing list after I consulted within Wind River.

Many thanks for your interest and support.

[bitdancer]

I'm not sure what licensing issues you are talking about, but setting up a buildbot shouldn't normally run into any. As long as you have a license to the run the OS, the fact that you are using it to receive jobs from our build master and run them shouldn't be a problem. You can keep the whole thing behind a firewall in a DMZ: the slave makes outbound connections to pick up its jobs.

On the other hand, the logistics of setting up a cross compile buildbot might be a bit complex, I've never done that. You might need specific support from our build master. In any case, the python-buldbots mailing list is the place to talk if you want to/can pursue this.

[ned-deily]

As I commented on the PR, I think this PR should not be merged until and if there is a consensus that this support belongs in the standard cpython repo and there is an agreed-upon plan how this platform would be supported on going. I think it needs an approved PEP. We've allowed ourselves in the past to do a long-term disservice to our downstream users by merging in support for platforms that we were not equipped to support. In any case, it would need to wait for 3.8.

[vstinner]

Kuhl, Brian started a new discussion: [Python-Dev] VxWorks and cpython?
https://mail.python.org/pipermail/python-dev/2019-January/156024.html

PR 11968 and PR 12051 are small and reasonable.

IMHO we can take decisions on a case by case basic. But WindRiver plans to provide a buildbot and is already showing their will to propose PRs, so it seems like things are moving on.

[vstinner]

New changeset 32f5fdd by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Add cross-build support for VxWorks RTOS (GH-11968)
32f5fdd

[vstinner]

New changeset f4b0a1c by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Add encoding support for VxWorks RTOS (GH-12051)
f4b0a1c

[vstinner]

New changeset 8b5bdda by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Adapt the _signal module to VxWorks RTOS (GH-12304)
8b5bdda

[vstinner]

Please stop to send new PRs which depend on other PRs which are not merged yet, it becomes too hard to follow :-( Let's focus on first PRs.

[vstinner]

New changeset f4333d0 by Victor Stinner (hliu0) in branch 'master':
bpo-31904: Fix test_utf8_mode on VxWorks (GH-12428)
f4333d0

[tiran]

I'm against implementing crypt on top of OpenSSL's DES_crypt. Please don't support the module at all instead of just supporting a completely broken implementation.

[vstinner]

Please stop adding more pull requests, I cannot review too many at the same time. I would prefer to have a limit of 4 open PRs. I don't propose to close existing ones. Just stop to add more :-)

[vstinner]

New changeset 236d0b7 by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Don't build the _crypt extension on VxWorks (GH-12833)
236d0b7

[vstinner]

New changeset f1464f4 by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Port the time module on VxWorks (GH-12305)
f1464f4

[pxinwr]

@vstinner I've informed WRS team of temporarily not creating new PRs in until less than 4 PRs are in the open state. Next we will keep open PRs less than 4. Thanks for your effort on them.

[vstinner]

New changeset 36c41bc by Victor Stinner (Lihua Zhao) in branch 'master':
bpo-31904: Fix test_tabnanny on VxWorks (GH-12646)
36c41bc

[vstinner]

New changeset 2954550 by Victor Stinner (Lihua Zhao) in branch 'master':
bpo-31904: Port test_cmd_line to VxWorks (bpo-12648)
2954550

[vstinner]

New changeset 693c104 by Victor Stinner (Lihua Zhao) in branch 'master':
bpo-31904: Port test_resource to VxWorks (GH-12719)
693c104

[vstinner]

New changeset f2d7ac7 by Victor Stinner (pxinwr) in branch 'master':
bpo-31904: Add posix module support for VxWorks (GH-12118)
f2d7ac7

[vstinner]

Please update PR 12670: see my comment there.

[vstinner]

New changeset 5cd2803 by pxinwr in branch 'master':
bpo-31904: Fix test_c_locale_coercion encodings for VxWorks RTOS (GH-19448)
5cd2803

[methane]

New changeset 855e688 by pxinwr in branch 'master':
bpo-31904: Fix test_ftplib failures for VxWorks RTOS (GH-19447)
855e688

[jimjjewett]

Is it safe to say that there is an now intent to support VxWorks within the main tree, with Wind River agreeing to be primary support?

And this ticket has become a tracking ticket for the status on getting it there, small PR by small PR plus buildbot?

[pxinwr]

@Jim.Jewett Yes. We have got most modules passed testing locally. Now we want to get the patches upstream. So VxWorks platform can be officially supported.

[vstinner]

New changeset aa1b8a1 by pxinwr in branch 'master':
bpo-31904: Fix test_os.test_getcwd_long_path() failure for VxWorks (GH-20256)
aa1b8a1

[miss-islington]

New changeset a86a274 by pxinwr in branch 'master':
bpo-31904: add shell requirement for test_pipes (GH-23489)
a86a274

[miss-islington]

New changeset 6a273fd by pxinwr in branch 'master':
bpo-31904: skip some tests related to fifo on VxWorks (GH-23473)
6a273fd

[miss-islington]

New changeset 00a6568 by pxinwr in branch 'master':
bpo-31904: remove libnet dependency from detect_socket() for VxWorks (GH-23394)
00a6568

[vstinner]

New changeset 1244c81 by pxinwr in branch 'master':
bpo-31904: Support signal module on VxWorks (GH-23391)
1244c81

[vstinner]

New changeset b2d0c66 by pxinwr in branch 'master':
bpo-31904: Fix fifo test cases for VxWorks (GH-20254)
b2d0c66

[vstinner]

New changeset e483d28 by pxinwr in branch 'master':
bpo-31904: Fix test_netrc for VxWorks RTOS (GH-21675)
e483d28

[gpshead]

New changeset 8d4f57d by pxinwr in branch 'master':
bpo-31904: fix test_doctest.py failures for VxWorks (GH-23419)
8d4f57d

[wlf100220]

test comment

[vstinner]

New changeset d5dcb65 by pxinwr in branch 'master':
bpo-31904: Define THREAD_STACK_SIZE for VxWorks (GH-23718)
d5dcb65

[vstinner]

New changeset c117426 by pxinwr in branch 'master':
bpo-31904: Enable libpython3.so shared library for VxWorks (GH-23741)
c117426

[vstinner]

New changeset e1e3c2d by pxinwr in branch 'master':
bpo-31904: Disable os.popen and popen test cases on VxWorks (GH-21687)
e1e3c2d

[vstinner]

New changeset 9a0dea6 by pxinwr in branch 'master':
bpo-31904: Skip some tests of changing owner in _test_all_chown_common() on VxWorks (GH-23716)
9a0dea6

[vstinner]

New changeset b230409 by pxinwr in branch 'master':
bpo-31904: Skip os.path.expanduser() tests on VxWorks (GH-23776)
b230409

[asvetlov]

New changeset ba760f3 by pxinwr in branch 'master':
bpo-31904: Skip some asyncio tests on VxWorks (bpo-23815)
ba760f3

[vstinner]

New changeset 75dabfe by pxinwr in branch 'master':
bpo-31904: posixpath.expanduser() handles None user home on VxWorks (GH-23530)
75dabfe

[vstinner]

New changeset ab74c01 by pxinwr in branch 'master':
bpo-31904: Fix site and sysconfig modules for VxWorks RTOS (GH-21821)
ab74c01

[vstinner]

New changeset 5e45f1c by pxinwr in branch 'master':
bpo-31904: setup.py: fix cross-compilation on VxWorks (GH-24191)
5e45f1c

[miss-islington]

New changeset 6e7fe19 by Miss Islington (bot) in branch '3.10':
[3.10] bpo-31904: Correct error string in test_file_not_exists() for VxWorks (GH-25965) (GH-25983)
6e7fe19