The select and time modules uses libm functions without linking against it

[fornwall]

BPO	21668
Nosy	@vstinner, @freakboy3742, @xdegaye, @yan12125
Files	time_and_select_module_link_with_libm.patch audioop_ctypes_test_link_with_libm.patch: Additional patch to deal with missing -lm flag for audioop and _ctypes_test. time_select_audioop_ctypes_test_link_with_libm.patch: Combined patches to add '-lm' flag. python-hg-modules-link-libm.patch ext_modules_libm.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2016-04-19.14:00:07.302>
created_at = <Date 2014-06-05.12:56:33.447>
labels = ['build', 'type-crash']
title = 'The select and time modules uses libm functions without linking against it'
updated_at = <Date 2016-04-20.08:02:15.650>
user = 'https://bugs.python.org/fornwall'

bugs.python.org fields:

activity = <Date 2016-04-20.08:02:15.650>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2016-04-19.14:00:07.302>
closer = 'vstinner'
components = ['Build', 'Cross-Build']
creation = <Date 2014-06-05.12:56:33.447>
creator = 'fornwall'
dependencies = []
files = ['35490', '37041', '37042', '41168', '42514']
hgrepos = []
issue_num = 21668
keywords = ['patch']
message_count = 15.0
messages = ['219813', '219814', '230076', '230077', '230080', '255410', '255411', '263716', '263718', '263738', '263739', '263803', '263811', '263812', '263813']
nosy_count = 7.0
nosy_names = ['vstinner', 'freakboy3742', 'xdegaye', 'python-dev', 'fornwall', 'WanderingLogic', 'yan12125']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue21668'
versions = ['Python 3.5', 'Python 3.6']

[fornwall]

The select and time modules use functions from libm, but do not link against it.

• selectmodule.c calls ceil(3) in pyepoll_poll()
• timemodule.c calls fmod(3) and floor(3) in floatsleep()

[fornwall]

Note: This causes problems at least when running on android, where the system is unable to find the symbols when loading the modules at runtime.

[WanderingLogic]

Additionally,

• audioop calls floor()
• _ctypes_test calls sqrt()

Patch attached.

[vstinner]

audioop_ctypes_test_link_with_libm.patch

+ libraries=['m'])

Why not using math_libs here? It would also be nice to add a comment explaining why libm is needed in each module.

Can someone please combine both patches?

[WanderingLogic]

> audioop_ctypes_test_link_with_libm.patch

•                         libraries=['m'])
  

Why not using math_libs here?

math_libs is defined in detect_modules(). But the _ctypes_test
extension is defined in a different function: detect_ctypes().

The other option, would be to define math_libs=['m'] directly above this line and then use it once. I didn't think that added clarity, but I'd be happy to do it that way if it fits better with standard style.

It would also be nice to add a comment explaining why libm is needed in each module.

Done.

Can someone please combine both patches?

Done.

[yan12125]

Bump. For rev c6930661599b timemodule.c and selectmodule.c no longer calls libm functions, while audioop.c and _ctypes_test.c still do. I have my updated patch here based on previous patches. Note that the introduction of detect_math_libs() is my naive try. Its naming and position in setup.py may need more consideration.

Here's the result before and after my patch.

Before:
shell@GT-N7000:/data/local/tmp $ python3
Python 3.6.0a0 (default:6a8fbb97c8d8+, Nov 26 2015, 18:42:29) 
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import select 
>>> import time
>>> import audioop
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen failed: cannot locate symbol "floor" referenced by "audioop.cpython-36m-arm-linux-gnueabi.so"...
>>> import _ctypes_test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen failed: cannot locate symbol "sqrt" referenced by "_ctypes_test.cpython-36m-arm-linux-gnueabi.so"...
>>> 

After:

shell@GT-N7000:/data/local/tmp $ python3
Python 3.6.0a0 (default:c6930661599b+, Nov 26 2015, 19:10:15) 
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> import selectr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'selectr'
>>> import select 
>>> import audioop
>>> import _ctypes_test
>>> 

Tested on Samsung Galaxy Note GT-N7000 with my custom build of CyanogenMod 11.0

[yan12125]

Well, the hg revision of Python in two tests are different, because I hg pull -u each time I build.

[xdegaye]

The _datetime module also calls functions in libm: delta_new() uses
round() and accum() uses modf().

The attached patch updates setup.py for all the modules that use libm.

[yan12125]

The approach with a helper function is better. See https://github.com/yan12125/python3-android/blob/038271d/mk/python/modules-link-libm.patch.

By the way, I have verified there are no libraries referencing libm functions with https://github.com/yan12125/python3-android/blob/cpython-hg/devscripts/import_all.py

[python-dev]

New changeset f92fea23161d by Victor Stinner in branch '3.5':
setup.py: add missing libm dependency
https://hg.python.org/cpython/rev/f92fea23161d

New changeset 3a9b47b062b9 by Victor Stinner in branch 'default':
Merge 3.5: Issue bpo-21668
https://hg.python.org/cpython/rev/3a9b47b062b9

[vstinner]

I pushed ext_modules_libm.patch with the helper function proposed by Chi Hsuan Yen in https://github.com/yan12125/python3-android/blob/038271d/mk/python/modules-link-libm.patch.

Tell me if it's not enough. I hope that it will make CPython a little bit more portable ;-)

[xdegaye]

The patch should be acknowledged in Misc/NEWS to Chi Hsuan Yen and not to me.

[python-dev]

New changeset f76753f26982 by Victor Stinner in branch 'default':
Issue bpo-21668: Fix author of the patch.
https://hg.python.org/cpython/rev/f76753f26982

[python-dev]

New changeset 7530caa5ed1a by Victor Stinner in branch 'default':
Issue bpo-21668: Add also Chi Hsuan Yen to Misc/ACKS
https://hg.python.org/cpython/rev/7530caa5ed1a

[vstinner]

Xavier de Gaye: "The patch should be acknowledged in Misc/NEWS to Chi Hsuan Yen and not to me."

Oh sorry. I see 4 authors in attached patches, I picked the latest. Thank you to all authors for your contribution on this issue ;-) I fixed the author in Misc/NEWS.