New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of PyNEST and SLI commandline arguments #840

Merged
merged 7 commits into from Mar 7, 2018

Conversation

Projects
None yet
4 participants
@jougs
Contributor

jougs commented Oct 13, 2017

In particular, this makes the following changes:

  • Replace the the command line argument --sli-debug by --debug only right before loading the pynestkernel
  • Make all given command line arguments available in the statusdict
  • introduce --sli-debug to avoid confusion with Python's --debug command line argument
  • Remove mutation of argv by PyNEST

This fixes #779 and fixes #784.

jougs added some commits Oct 13, 2017

Ignore --debug command line argument and add --sli-debug, fixing #784
* Replace the the command line argument --sli-debug by --debug only right before loading the pynestkernel
* Make all given commandline arguments available in the statusdict
* introduce --sli-debug to avoid confusion with Python's --debug argument
@heplesser

@jougs Thanks, this looks pretty good! I just have a few small comments in the code. Is there any sensible way to actually test that this all works as part of our test suite?

Show outdated Hide outdated pynest/nest/__init__.py
Show outdated Hide outdated pynest/nest/__init__.py
@jougs

This comment has been minimized.

Show comment
Hide comment
@jougs

jougs Feb 6, 2018

Contributor

@heplesser, @apeyser: I've updated the PR to reply to the review comments. Can you please have another look? Thanks!

Contributor

jougs commented Feb 6, 2018

@heplesser, @apeyser: I've updated the PR to reply to the review comments. Can you please have another look? Thanks!

@terhorstd terhorstd added this to the NEST 2.16 milestone Mar 5, 2018

@apeyser

Some questions regarding python and SLI

Show outdated Hide outdated pynest/nest/__init__.py
nest_argv.remove("--debug")
if "--sli-debug" in nest_argv:
nest_argv.remove("--sli-debug")
nest_argv.append("--debug")

This comment has been minimized.

@apeyser

apeyser Mar 5, 2018

Contributor

Does command line ordering matter? If so, this re-orders operations.

@apeyser

apeyser Mar 5, 2018

Contributor

Does command line ordering matter? If so, this re-orders operations.

This comment has been minimized.

@jougs

jougs Mar 5, 2018

Contributor

I don't see any reason why it should.

@jougs

jougs Mar 5, 2018

Contributor

I don't see any reason why it should.

Show outdated Hide outdated pynest/nest/__init__.py
Show outdated Hide outdated pynest/nest/tests/test_sp/test_growth_curves.py
@@ -176,23 +176,20 @@ cdef class NESTEngine(object):
raise NESTError("argv can't be empty")
# Create c-style argv arguments from sys.argv
cdef char* arg0 = "pynest\0"

This comment has been minimized.

@apeyser

apeyser Mar 5, 2018

Contributor

So, is the crazy SLI requirement gone?

@apeyser

apeyser Mar 5, 2018

Contributor

So, is the crazy SLI requirement gone?

This comment has been minimized.

@jougs

jougs Mar 5, 2018

Contributor

Yes.

@jougs

jougs Mar 5, 2018

Contributor

Yes.

for i, argvi in enumerate(argv_bytes):
argv_chars_off[i] = argvi # c-string ref extracted
argv_chars[i] = argvi # c-string ref extracted

This comment has been minimized.

@apeyser

apeyser Mar 5, 2018

Contributor

Does

argv_chars = [<char*> argvi in argv_bytes]

work? That's simpler
Or even maybe

argv_chars[:] = argv_bytes[:]
@apeyser

apeyser Mar 5, 2018

Contributor

Does

argv_chars = [<char*> argvi in argv_bytes]

work? That's simpler
Or even maybe

argv_chars[:] = argv_bytes[:]

This comment has been minimized.

@jougs

jougs Mar 5, 2018

Contributor

Unfortunately this does not work, so I've left it unchanged. Here's the error messages for reference.

Error compiling Cython file:
------------------------------------------------------------
...
            # Need to keep a reference to encoded bytes issue #377
            # argv_bytes = [byte...] which internally holds a reference
            # to the c string in argv_char = [c-string... NULL]
            # the `byte` is the utf-8 encoding of sys.argv[...]
            argv_bytes = [argvi.encode() for argvi in argv]
            argv_chars[:] = argv_bytes[:]
                                     ^
------------------------------------------------------------

/home/jochen/work/src/nest-simulator/pynest/pynestkernel.pyx:191:38: Cannot convert Python object to 'char **'

Error compiling Cython file:
------------------------------------------------------------
...
            # Need to keep a reference to encoded bytes issue #377
            # argv_bytes = [byte...] which internally holds a reference
            # to the c string in argv_char = [c-string... NULL]
            # the `byte` is the utf-8 encoding of sys.argv[...]
            argv_bytes = [argvi.encode() for argvi in argv]
            argv_chars[:] = argv_bytes[:]
                                     ^
------------------------------------------------------------

/home/jochen/work/src/nest-simulator/pynest/pynestkernel.pyx:191:38: Storing unsafe C derivative of temporary Python reference

@jougs

jougs Mar 5, 2018

Contributor

Unfortunately this does not work, so I've left it unchanged. Here's the error messages for reference.

Error compiling Cython file:
------------------------------------------------------------
...
            # Need to keep a reference to encoded bytes issue #377
            # argv_bytes = [byte...] which internally holds a reference
            # to the c string in argv_char = [c-string... NULL]
            # the `byte` is the utf-8 encoding of sys.argv[...]
            argv_bytes = [argvi.encode() for argvi in argv]
            argv_chars[:] = argv_bytes[:]
                                     ^
------------------------------------------------------------

/home/jochen/work/src/nest-simulator/pynest/pynestkernel.pyx:191:38: Cannot convert Python object to 'char **'

Error compiling Cython file:
------------------------------------------------------------
...
            # Need to keep a reference to encoded bytes issue #377
            # argv_bytes = [byte...] which internally holds a reference
            # to the c string in argv_char = [c-string... NULL]
            # the `byte` is the utf-8 encoding of sys.argv[...]
            argv_bytes = [argvi.encode() for argvi in argv]
            argv_chars[:] = argv_bytes[:]
                                     ^
------------------------------------------------------------

/home/jochen/work/src/nest-simulator/pynest/pynestkernel.pyx:191:38: Storing unsafe C derivative of temporary Python reference

@apeyser

Edit with better comments

@jougs

This comment has been minimized.

Show comment
Hide comment
@jougs

jougs Mar 5, 2018

Contributor

@apeyser: I've addressed all your comments in the most recent commit and in the replies above.

Contributor

jougs commented Mar 5, 2018

@apeyser: I've addressed all your comments in the most recent commit and in the replies above.

@apeyser

apeyser approved these changes Mar 5, 2018

Agreed.

@heplesser heplesser merged commit 47846f5 into nest:master Mar 7, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment