Slowdown and high memory usage when adding a new module in embedded Python 3.4 on 64bit Windows

[MrValdez]

BPO	21110
Nosy	@vstinner, @methane, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-03.00:53:11.248>
created_at = <Date 2014-03-31.10:35:23.167>
labels = ['extension-modules', 'build', 'performance']
title = 'Slowdown and high memory usage when adding a new module in embedded Python 3.4 on 64bit Windows'
updated_at = <Date 2019-06-03.00:53:11.248>
user = 'https://bugs.python.org/MrValdez'

bugs.python.org fields:

activity = <Date 2019-06-03.00:53:11.248>
actor = 'MrValdez'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-03.00:53:11.248>
closer = 'MrValdez'
components = ['Build', 'Extension Modules']
creation = <Date 2014-03-31.10:35:23.167>
creator = 'MrValdez'
dependencies = []
files = []
hgrepos = []
issue_num = 21110
keywords = []
message_count = 9.0
messages = ['215227', '238892', '340010', '340030', '340093', '340149', '340159', '340162', '344349']
nosy_count = 4.0
nosy_names = ['vstinner', 'methane', 'steve.dower', 'MrValdez']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue21110'
versions = ['Python 3.4']

[MrValdez]

The example in the "Extending Embedded Python" (https://docs.python.org/3/extending/embedding.html#extending-embedded-python) takes up a lot of memory and slow downs the computer. I am using Windows 7 64bit, Python 3.4 64bit and gcc (rubenvb-4.8.0) 4.8.0.

Solution:

Apparently, you need to add this to your C program:

#define HAVE_SSIZE_T

I found the solution at (https://forums.embarcadero.com/message.jspa?messageID=581594). The page said that when adding the above, "this ensures that the type Py_ssize_t becomes __int64 instead of int."

Expected:

The programmer should not see a simple program (that came from the main documentation, no less) causing slowdowns and high memory usage. The 64bit version of the Python installer was downloaded so the programmer should, at least, not need to know that they need this flag.

Can the Python.h in the 64 bit builds automatically set the flag? Or, if this isn't feasible, add a note to the documentation that this flag exist.

[BreamoreBoy]

Who is best placed to address this issue?

[methane]

Is this issue still alive?
May I close this issue as "out of date"?

[vstinner]

Steve: Are you aware of this issue? "Apparently, you need to add this to your C program: #define HAVE_SSIZE_T"

[zooba]

Never heard of it, but perhaps there are some preprocessor checks for Windows that assume you are using MSVC and not gcc?

We don't support compilers other than MSVC on Windows, but if someone has a fix for this I'm happy to consider it.

[methane]

• rubenvb is not maintained.
• https://forums.embarcadero.com/message.jspa?messageID=581594 is dead link.
• When PY_SSIZE_T is not defined, we use intptr_t, not int.

Original issue report doesn't make sense to me.
But we can't confirm it for now.

[vstinner]

In case of doubt, I suggest to close the issue. If the issue strikes back,
it is trivial to reopen the issue or open a new one.

[MrValdez]

Its a shame the link is now gone. Its also been a long time that I don't remember all the details.

I still have my code from back when I found this bug. I can try to replicate it.

We don't support compilers other than MSVC on Windows
[...]
rubenvb is not maintained.

I don't remember why I used rubenvb back then. I'll see if I can get MSVC working. Maybe this is the cause for the slow down.

[MrValdez]

I couldn't replicate since my old dev machine didn't have rubenvb.

Since this issue has been around for 5 years already, rubenvb isn't maintained and I couldn't replicate, I think its fine to close this issue.