Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Modern Command Line interface for Cython #53

Open
wants to merge 55 commits into from

4 participants

Stefano Sanfilippo Robert Bradshaw Vitja Makarov scoder
Stefano Sanfilippo

NOTE: this replicates a previous pull request from a deleted repo.

Here is my modern command line interface, which performs a little slower if compared to old custom code, but is clear and easily extensible. It supports both argparse and optparse, though the "legacy" version has some 10 HACKs . Additionally, argparse performs better with complicated command lines (like those generated by autotools &co.) - especially if source files are not at the end of the argument list - and allows "read arguments from file" syntax.

This pull includes some changes to surrounding code (specifically, Cython.Compiler.Main), either for integration or performance issues.

The only potential issue is with --embed argument, since it allows its value to be implied:

cython --embed myfile

is ok, but:

cython --embed=mymain myfile

is correctly understood only by argparse. Secondly,

cython --embed mymain myfile

is ambigous. Since the parser is greedy for source files, mymain will be considered a source file, actually breaking the case above with a "more than one file" error. If anyone liked that syntax, it may use:

cython --embed mymain -- myfile

This syntax also allows for explicit main() name specification in optparse version.

This is hard to be fixed. At least, I could have optparse to ignore it and parse --embed form manually, but that would be a huge HACK.


NOTE: my branch has passed all tests from testsuite, including a purposedly written one (see Cython.Compiler.Tests.TestCommandLine) but, please, double check this ;)

and others added some commits July 31, 2011
Vitja Makarov Add Entry.cf_used attribute 231aabf
Vitja Makarov Set entry.cf_used to False when argument is unused 5db9f26
Vitja Makarov Mark unused cdef-function args with CYTHON_UNUSED 594e3ba
Vitja Makarov Don't initialize unused cdef-function optional args f871cad
Vitja Makarov Add generic python args test 480798a
Vitja Makarov Code cleanup 8d4c8f4
Vitja Makarov Add runtime test with unused arguments 6cf7ea9
Vitja Makarov Don't remove optional args when function is overridable 8a3aa54
Vitja Makarov Only mark skip_dispatch arg as unused if function has OverrideCheckNode a084a7b
Vitja Makarov Mark unused vars with CYTHON_UNUSED qualifier cecd09c
Vitja Makarov Mark unused METH_O arguments 7306e1e
scoder Merge pull request #50 from vitek/_unused_args
Silence warning about unused args
5ace507
Robert Bradshaw Some Python 2.3 cleanup. a0fff9f
Robert Bradshaw Refuse to compile for Python < 2.4 517c36d
Robert Bradshaw More Py2.3 cleanup. 6ff3b8e
AttributeNode.as_cython_attribute(): make method return value explicit 3769067
test case for ticket #653 bb90984
fix type inheritance check for builtin types ddba4f1
fix support for redeclaring builtin types as external extension types…
…, add a big FIXME to mark for eventual removal
ec2a95d
implement may_be_none() for CloneNode 7faf9c6
fix ticket #653
- generate correct code for optimised override calls to methods of builtin types in subtypes
- optimise calls for 'final' subtypes
- disable optimised C-API calls for non-final subtypes
9ff18ae
drop unused import e739f25
Stefano Sanfilippo Command line parsing now uses argparse module. Still a work in progre…
…ss, though.
5d55f8c
Stefano Sanfilippo Turned flags structure into a readable dictionary. c3f7f82
Stefano Sanfilippo Completed transition to argparse module. Start adding optparse compat…
…ibilty.
5508b3d
Stefano Sanfilippo Added optparse fallback. 3872ecb
Stefano Sanfilippo Added complete test case for Cython.Compiler.CmdLine module. 72b09e1
Stefano Sanfilippo Completed test case for debug options & directives. 2a81407
Stefano Sanfilippo Completed -X and --debug parsing options. Partial regression for optp…
…arse

compatibility.
e4303dc
Stefano Sanfilippo Small functional adds (to reflect changes in the CmdLine module) 0ef7868
Stefano Sanfilippo Fixed small error in alternate Parser() implementation b2113c1
Stefano Sanfilippo Now default options are no more overridden in Option a0b637c
Stefano Sanfilippo Removed deprecated internal functions. 3bff312
Stefano Sanfilippo Corrected other lack in alternate implementation (optparse) ef7bbc5
Stefano Sanfilippo Turned c-style booleans (0 as False, 1 as True) into real python bools. bdd47a7
Stefano Sanfilippo Fixed bug in optparse's --embed recognition 27b72e4
Stefano Sanfilippo Replaced conditional import of argparse with a try: catch: block 10e0231
Stefano Sanfilippo Replaced failUnlessEqual with assertEqual wherever possible. 68a0277
Stefano Sanfilippo Adequated some tests to new CmdLine.py internals. 4a20f39
Stefano Sanfilippo Now dictionaries and lists belonging to Options are no more overridden. 411838a
Stefano Sanfilippo Added a HACK to prevent unwanted behaviour of --embed in some test ca…
…ses.
404451e
Stefano Sanfilippo Updated tests for Cython.Compiler.CmdLine 6f0f931
Stefano Sanfilippo Added secure list for options to insert into Options module (so that …
…no unwanted

change is applied if a duplicate - but indipendent - variable will be added)
60aec93
Stefano Sanfilippo Added cleanup code to avoid a long series of nasty errors. BIG TODO: …
…restructure

Options module.
990487f
Stefano Sanfilippo Modified optparse code to support variable length --embed flag 2f30832
Stefano Sanfilippo Removed a test case, added a remark in error messages. 6f121c9
Stefano Sanfilippo Added one more hack for optparse, now both versions passed all the te…
…sts.
e8be8a5
Stefano Sanfilippo Removed fake import to force optparse version 80564a3
Stefano Sanfilippo Modified debug flags syntax. b5d44cd
Stefano Sanfilippo Removed obsoleted code. 291df3b
Stefano Sanfilippo Updated copyright line. 6bfa85a
Stefano Sanfilippo esseks commented on the diff August 11, 2011
Cython/Compiler/CmdLine.py
... ...
@@ -1,181 +1,293 @@
1 1
 #
2  
-#   Cython - Command Line Parsing
  2
+#   Copyright 2005-2006 The Pyrex team
  3
+#   Copyright 2006-2011 The Cython team
  4
+#   Copyright 2011 Stefano Sanfilippo <satufk on GitHub>
1
Stefano Sanfilippo
esseks added a note August 11, 2011

Actually, I rewrote this file from scratches, but I've updated the copyright line to Pyrex&Cython team. This is because there is no Cython Foundation to enforce our rights.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Stefano Sanfilippo

Ok, I figured out a small HACK for --embed... I'll sketch it as soon as I'm back, and I'll commit :)

Robert Bradshaw robertwb commented on the diff August 15, 2011
Cython/Compiler/CmdLine.py
... ...
@@ -1,181 +1,292 @@
1 1
 #
2  
-#   Cython - Command Line Parsing
  2
+#   Copyright 2005-2006 The Pyrex team
1
Robert Bradshaw Owner
robertwb added a note August 15, 2011

Any change to the licensing wording should be discussed on the list, not slipped into an unrelated pull request such as this. The top-level license and copying files have been sufficient without the need for per-file boilerplate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Robert Bradshaw
Owner

This seems to add a lot of complexity, e.g. spanning optparse/argparse, without any clear benefit (and perhaps a regression, or a lot of hackery, regarding --embed). Also, the current omission of certain flags in usage was intentional. The current setup works fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 55 unique commits by 5 authors.

Aug 03, 2011
Vitja Makarov Add Entry.cf_used attribute 231aabf
Vitja Makarov Set entry.cf_used to False when argument is unused 5db9f26
Vitja Makarov Mark unused cdef-function args with CYTHON_UNUSED 594e3ba
Vitja Makarov Don't initialize unused cdef-function optional args f871cad
Vitja Makarov Add generic python args test 480798a
Vitja Makarov Code cleanup 8d4c8f4
Vitja Makarov Add runtime test with unused arguments 6cf7ea9
Vitja Makarov Don't remove optional args when function is overridable 8a3aa54
Vitja Makarov Only mark skip_dispatch arg as unused if function has OverrideCheckNode a084a7b
Vitja Makarov Mark unused vars with CYTHON_UNUSED qualifier cecd09c
Vitja Makarov Mark unused METH_O arguments 7306e1e
Aug 05, 2011
scoder Merge pull request #50 from vitek/_unused_args
Silence warning about unused args
5ace507
Robert Bradshaw Some Python 2.3 cleanup. a0fff9f
Robert Bradshaw Refuse to compile for Python < 2.4 517c36d
Robert Bradshaw More Py2.3 cleanup. 6ff3b8e
Aug 07, 2011
AttributeNode.as_cython_attribute(): make method return value explicit 3769067
test case for ticket #653 bb90984
fix type inheritance check for builtin types ddba4f1
fix support for redeclaring builtin types as external extension types…
…, add a big FIXME to mark for eventual removal
ec2a95d
implement may_be_none() for CloneNode 7faf9c6
fix ticket #653
- generate correct code for optimised override calls to methods of builtin types in subtypes
- optimise calls for 'final' subtypes
- disable optimised C-API calls for non-final subtypes
9ff18ae
Aug 08, 2011
drop unused import e739f25
Aug 11, 2011
Stefano Sanfilippo Command line parsing now uses argparse module. Still a work in progre…
…ss, though.
5d55f8c
Stefano Sanfilippo Turned flags structure into a readable dictionary. c3f7f82
Stefano Sanfilippo Completed transition to argparse module. Start adding optparse compat…
…ibilty.
5508b3d
Stefano Sanfilippo Added optparse fallback. 3872ecb
Stefano Sanfilippo Added complete test case for Cython.Compiler.CmdLine module. 72b09e1
Stefano Sanfilippo Completed test case for debug options & directives. 2a81407
Stefano Sanfilippo Completed -X and --debug parsing options. Partial regression for optp…
…arse

compatibility.
e4303dc
Stefano Sanfilippo Small functional adds (to reflect changes in the CmdLine module) 0ef7868
Stefano Sanfilippo Fixed small error in alternate Parser() implementation b2113c1
Stefano Sanfilippo Now default options are no more overridden in Option a0b637c
Stefano Sanfilippo Removed deprecated internal functions. 3bff312
Stefano Sanfilippo Corrected other lack in alternate implementation (optparse) ef7bbc5
Stefano Sanfilippo Turned c-style booleans (0 as False, 1 as True) into real python bools. bdd47a7
Stefano Sanfilippo Fixed bug in optparse's --embed recognition 27b72e4
Stefano Sanfilippo Replaced conditional import of argparse with a try: catch: block 10e0231
Stefano Sanfilippo Replaced failUnlessEqual with assertEqual wherever possible. 68a0277
Stefano Sanfilippo Adequated some tests to new CmdLine.py internals. 4a20f39
Stefano Sanfilippo Now dictionaries and lists belonging to Options are no more overridden. 411838a
Stefano Sanfilippo Added a HACK to prevent unwanted behaviour of --embed in some test ca…
…ses.
404451e
Stefano Sanfilippo Updated tests for Cython.Compiler.CmdLine 6f0f931
Stefano Sanfilippo Added secure list for options to insert into Options module (so that …
…no unwanted

change is applied if a duplicate - but indipendent - variable will be added)
60aec93
Stefano Sanfilippo Added cleanup code to avoid a long series of nasty errors. BIG TODO: …
…restructure

Options module.
990487f
Stefano Sanfilippo Modified optparse code to support variable length --embed flag 2f30832
Stefano Sanfilippo Removed a test case, added a remark in error messages. 6f121c9
Stefano Sanfilippo Added one more hack for optparse, now both versions passed all the te…
…sts.
e8be8a5
Stefano Sanfilippo Removed fake import to force optparse version 80564a3
Stefano Sanfilippo Modified debug flags syntax. b5d44cd
Stefano Sanfilippo Removed obsoleted code. 291df3b
Stefano Sanfilippo Updated copyright line. 6bfa85a
Stefano Sanfilippo Fixing small typo. 0b9da27
Aug 14, 2011
Stefano Sanfilippo Small fixes, tuples for all menu dictionary keys. 4768053
Stefano Sanfilippo Removed "check if option key is a string". 9e62cef
Aug 15, 2011
Stefano Sanfilippo Polishing code. Added quick --embed form recognition. 3491880
This page is out of date. Refresh to see the latest.
449  Cython/Compiler/CmdLine.py 100644 → 100755
... ...
@@ -1,181 +1,292 @@
1 1
 #
2  
-#   Cython - Command Line Parsing
  2
+#   Copyright 2005-2006 The Pyrex team
  3
+#   Copyright 2006-2011 The Cython team
  4
+#   Copyright 2011 Stefano Sanfilippo <satufk on GitHub>
3 5
 #
  6
+#   Licensed under the Apache License, Version 2.0 (the "License");
  7
+#   you may not use this file except in compliance with the License.
  8
+#   You may obtain a copy of the License at
  9
+#
  10
+#       http://www.apache.org/licenses/LICENSE-2.0
  11
+#
  12
+#   Unless required by applicable law or agreed to in writing, software
  13
+#   distributed under the License is distributed on an "AS IS" BASIS,
  14
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  15
+#   See the License for the specific language governing permissions and
  16
+#   limitations under the License.
4 17
 
5  
-import os
6  
-import sys
7  
-import Options
8  
-
9  
-usage = """\
10  
-Cython (http://cython.org) is a compiler for code written in the
11  
-Cython language.  Cython is based on Pyrex by Greg Ewing.
12  
-
13  
-Usage: cython [options] sourcefile.{pyx,py} ...
14  
-
15  
-Options:
16  
-  -V, --version                  Display version number of cython compiler
17  
-  -l, --create-listing           Write error messages to a listing file
18  
-  -I, --include-dir <directory>  Search for include files in named directory
19  
-                                 (multiple include directories are allowed).
20  
-  -o, --output-file <filename>   Specify name of generated C file
21  
-  -t, --timestamps               Only compile newer source files
22  
-  -f, --force                    Compile all source files (overrides implied -t)
23  
-  -q, --quiet                    Don't print module names in recursive mode
24  
-  -v, --verbose                  Be verbose, print file names on multiple compilation
25  
-  -p, --embed-positions          If specified, the positions in Cython files of each
26  
-                                 function definition is embedded in its docstring.
27  
-  --cleanup <level>              Release interned objects on python exit, for memory debugging.
28  
-                                 Level indicates aggressiveness, default 0 releases nothing.
29  
-  -w, --working <directory>      Sets the working directory for Cython (the directory modules
30  
-                                 are searched from)
31  
-  --gdb                          Output debug information for cygdb
32  
-
33  
-  -D, --no-docstrings            Strip docstrings from the compiled module.
34  
-  -a, --annotate                 Produce a colorized HTML version of the source.
35  
-  --line-directives              Produce #line directives pointing to the .pyx source
36  
-  --cplus                        Output a C++ rather than C file.
37  
-  --embed[=<method_name>]        Generate a main() function that embeds the Python interpreter.
38  
-  -2                             Compile based on Python-2 syntax and code semantics.
39  
-  -3                             Compile based on Python-3 syntax and code semantics.
40  
-  --fast-fail                    Abort the compilation on the first error
41  
-  --warning-error, -Werror       Make all warnings into errors
42  
-  --warning-extra, -Wextra       Enable extra warnings
43  
-  -X, --directive <name>=<value>[,<name=value,...] Overrides a compiler directive
44  
-"""
45  
-
46  
-# The following is broken http://trac.cython.org/cython_trac/ticket/379
47  
-#  -r, --recursive                Recursively find and compile dependencies (implies -t)
48  
-
49  
-
50  
-#The following experimental options are supported only on MacOSX:
51  
-#  -C, --compile    Compile generated .c file to .o file
52  
-#  --link           Link .o file to produce extension module (implies -C)
53  
-#  -+, --cplus      Use C++ compiler for compiling and linking
54  
-#  Additional .o files to link may be supplied when using -X."""
55  
-
56  
-def bad_usage():
57  
-    sys.stderr.write(usage)
58  
-    sys.exit(1)
  18
+'''Cython - Command Line Parsing'''
59 19
 
60  
-def parse_command_line(args):
  20
+__all__ = ['parser', 'parse_command_line']
61 21
 
62  
-    from Cython.Compiler.Main import \
63  
-        CompilationOptions, default_options
  22
+from Cython import __version__ as version
  23
+import Cython.Compiler.Options as Options
64 24
 
65  
-    def pop_arg():
66  
-        if args:
67  
-            return args.pop(0)
68  
-        else:
69  
-            bad_usage()
  25
+try:
  26
+    # Just to see if it was implemented (Py >= 3.0)
  27
+    'test'.isidentifier()
  28
+except AttributeError:
  29
+    import tokenize
  30
+    import re
  31
+    identifierre = re.compile('^%s$' % tokenize.Name)
  32
+    isidentifier = identifierre.match
  33
+else:
  34
+    isidentifier = lambda s: s.isidentifier()
  35
+
  36
+menu = {
  37
+    'default' : {
  38
+        ('-V', '--version') : {
  39
+            'dest' : 'show_version', 'action' : 'version',
  40
+            'version' : 'Cython version %s' % version,
  41
+            'help' : 'Display version number of cython compiler'},
  42
+        ('sources',) : {
  43
+            'nargs' : '+', 'metavar' : 'sourcefile.{py,pyx}',
  44
+            'help' : 'Source file(s) to be compiled.' }},
  45
+    'environment setup': {
  46
+        ('-l', '--create-listing') : {
  47
+            'dest' : 'use_listing_file', 'action' : 'store_true',
  48
+            'help' : 'Write error messages to a listing file'},
  49
+        ('-I', '--include-dir') : {
  50
+            'dest' : 'include_path', 'action' : 'append',
  51
+            'metavar' : '<directory>',
  52
+            'help' : '''Search for include files in named <directory>
  53
+                        (multiple include directories are allowed).'''},
  54
+        ('-o', '--output-file') : {
  55
+            'dest' : 'output_file', 'metavar' : '<output file>',
  56
+            'help' : 'Specify name of generated C file'},
  57
+        ('-w', '--working') : {
  58
+            'dest' : 'working_path', 'metavar' : '<directory>',
  59
+            'help' : '''Sets the working directory for Cython
  60
+                        (the directory modules are searched from)'''}},
  61
+    'compilation' : {
  62
+        ('-t', '--timestamps') : {
  63
+            'dest' : 'timestamps', 'action' : 'store_true',
  64
+            'help' : 'Only compile newer source files (implied)'},
  65
+        ('-f', '--force') : {
  66
+            'dest' : 'timestamps', 'action' : 'store_false',
  67
+            'help' : 'Compile all source files (overrides -t)'},
  68
+        ('-z', '--pre-import') : {
  69
+            'dest' : 'pre_import', 'metavar' : '<module>',
  70
+            'help' : 'Import <module> before compilation.' },
  71
+        ('--fast-fail',) : {
  72
+            'dest' : 'fast_fail', 'action' : 'store_true',
  73
+            'help' : 'Abort the compilation on the first error.' },
  74
+        ('-Werror', '--warning-errors'): {
  75
+            'dest' : 'warning_errors', 'action' : 'store_true',
  76
+            'help' : 'Make all warnings into errors.' },
  77
+        ('-Wextra', '--warning-extra'): {
  78
+            'dest' : 'compiler_directives', 'const' : Options.extra_warnings,
  79
+            'action' : 'append_const',
  80
+            'help' : 'Enable extra warnings' }},
  81
+    'debugging' : {
  82
+        ('--cleanup',) : {
  83
+            'dest' : 'generate_cleanup_code', 'type' : int, 'metavar' : '<level>',
  84
+            'default' : 0,
  85
+            'help' : '''Release interned objects on python exit, for memory debugging.
  86
+                        <level> indicates aggressiveness, default 0 releases nothing.'''},
  87
+        ('--gdb',) : {
  88
+            'dest' : 'gdb_debug', 'action' : 'store_true',
  89
+            'help' : 'Output debug information for cygdb'}},
  90
+    'Python compatibility options' : {
  91
+        ('-2',) : {
  92
+            'dest' : 'language_level', 'const' : 2, 'action' : 'store_const',
  93
+            'help' : 'Compile based on Python-2 syntax and code semantics.'},
  94
+        ('-3',) : {
  95
+            'dest' : 'language_level', 'const' : 3, 'action' : 'store_const',
  96
+            'help' : 'Compile based on Python-3 syntax and code semantics.'}},
  97
+    'recursive compilation': {
  98
+        ('-r', '--recursive') : {
  99
+            'dest' : 'recursive', 'action' : 'store_true',
  100
+            'help' : 'Recursively find and compile dependencies (implies -t)'},
  101
+        ('-q', '--quiet') : {
  102
+            'dest' : 'quiet', 'action' : 'store_true',
  103
+            'help' : "Don't print module names in recursive mode" },
  104
+        ('-v', '--verbose') : {
  105
+            'dest' : 'verbose', 'action' : 'count',
  106
+            'help' : 'Be verbose, print file names on multiple compilation' }},
  107
+    'code annotation' : {
  108
+        ('-a', '--annotate') : {
  109
+            'dest' : 'annotate', 'action' : 'store_true',
  110
+            'help' : 'Produce a colorized HTML version of the source.' },
  111
+        ('--line-directives',) : {
  112
+            'dest' : 'emit_linenums', 'action' : 'store_true',
  113
+            'help' : 'Produce #line directives pointing to the .pyx source' }},
  114
+    'code generation' : {
  115
+        ('--embed',) : {
  116
+            'dest' : 'embed', 'const' : 'main', 'nargs' : '?',
  117
+            'metavar' : '<function name>',
  118
+            'help' : 'Generate a main() function that embeds the Python interpreter.'},
  119
+        ('-+', '--cplus') : {
  120
+            'dest' : 'cplus', 'action' : 'store_true',
  121
+            'help' : 'Output a C++ rather than C file.' },
  122
+        ('-p', '--embed-positions') : {
  123
+            'dest' : 'embed_pos_in_docstring', 'action' : 'store_true',
  124
+            'help' : '''If specified, the positions in Cython files of each
  125
+                        function definition is embedded in its docstring.''' },
  126
+        ('--no-c-in-traceback',) : {
  127
+            'dest' : 'c_line_in_traceback', 'action' : 'store_false',
  128
+            'help' : 'Omit C source code line when printing tracebacks.' },
  129
+        ('--convert-range',) : {
  130
+            'dest' : 'convert_range', 'action' : 'store_true',
  131
+            'help' : '''Convert `for x in range():` statements in pure C for(),
  132
+                      whenever possibile.''' },
  133
+        ('-D', '--no-docstrings') : {
  134
+            'dest' : 'docstrings', 'action' : 'store_false',
  135
+            'help' : 'Strip docstrings from the compiled module.' },
  136
+        ('--disable-function-redefinition',) : {
  137
+            'dest' : 'disable_function_redefinition', 'action' : 'store_true',
  138
+            'help' : 'For legacy code only, needed for some circular imports.' },
  139
+        ('--old-style-globals',) : {
  140
+            'dest' : 'old_style_globals', 'action' : 'store_true',
  141
+            'help' : '''Makes globals() give the first non-Cython module
  142
+                        globals in the call stack. For SAGE compatibility''' }},
  143
+   'behavioural options' : {
  144
+       ('-X', '--directive') : {
  145
+           'dest' : 'compiler_directives', 'action' : 'append',
  146
+           'metavar' : '<name>=<value>[,<name>=<value>...]',
  147
+           'help' : 'Overrides a #pragma compiler directive'}},
  148
+#    'experimental options' : { # MacOS X only
  149
+#        ('-C', '--compile') : {
  150
+#            'dest' : 'compile', 'action' : 'store_true',
  151
+#            'help' : 'Compile generated .c file to .o file' },
  152
+#        ('-l', '--link') : {
  153
+#            'dest' : 'link', 'action' : 'store_true',
  154
+#            'help' : '''Link .o file to produce extension module (implies -C)
  155
+#                        Additional .o files to link may be supplied when using -X.''' }},
  156
+}
  157
+
  158
+epilog = '''\
  159
+Cython (http://cython.org) is a compiler for code written
  160
+in the Cython language. Cython is based on Pyrex by Greg Ewing.
  161
+
  162
+WARNING: RECURSIVE COMPILATION IS STILL BROKEN
  163
+(see http://trac.cython.org/cython_trac/ticket/379).'''
70 164
 
71  
-    def get_param(option):
72  
-        tail = option[2:]
73  
-        if tail:
74  
-            return tail
  165
+def makedebuglist():
  166
+    '''Reads flags from DebugFlags, and turn them into boolean options.'''
  167
+    import Cython.Compiler.DebugFlags as DebugFlags
  168
+    flags = {}
  169
+    for var in vars(DebugFlags):
  170
+        if var.startswith('debug'):
  171
+            name = '--' + var.replace('_', '-')
  172
+            flags.update({(name,) : dict(dest=var, action='store_true')})
  173
+    return menu.update({'compiler debugging options' : flags})
  174
+
  175
+class BasicParser:
  176
+    '''Basic command line parser functions, used by all parsers.'''
  177
+    def refine(self, options):
  178
+        # Separate source files list from all other options
  179
+        sources = options.sources
  180
+        del options.sources
  181
+
  182
+        # Basic sanity check
  183
+        if options.gdb_debug:
  184
+            import os
  185
+            options.output_dir = os.curdir
  186
+        if not options.include_path: #HACK for optparse compatibility
  187
+            options.include_path = []
  188
+        if not sources:
  189
+            self.error('no source file specified.')
  190
+        if options.output_file and len(sources) > 1:
  191
+            self.error('only one source file allowed when using -o')
  192
+        if getattr(options, 'embed', False) and len(sources) > 1: #HACK
  193
+            self.error('only one source file allowed when using --embed. Maybe you placed --embed after sources?')
  194
+
  195
+        # Parse compiler directives into a dictionary
  196
+        if options.compiler_directives:
  197
+            dirs = ','.join(options.compiler_directives)
  198
+            try:
  199
+                options.compiler_directives = Options.parse_directive_list(
  200
+                    dirs, relaxed_bool=True,
  201
+                    current_settings=Options.directive_defaults)
  202
+            except ValueError, error:
  203
+                parser.error('compiler directive: %s' % error.args[0])
75 204
         else:
76  
-            return pop_arg()
77  
-
78  
-    options = CompilationOptions(default_options)
79  
-    sources = []
80  
-    while args:
81  
-        if args[0].startswith("-"):
82  
-            option = pop_arg()
83  
-            if option in ("-V", "--version"):
84  
-                options.show_version = 1
85  
-            elif option in ("-l", "--create-listing"):
86  
-                options.use_listing_file = 1
87  
-            elif option in ("-+", "--cplus"):
88  
-                options.cplus = 1
89  
-            elif option == "--embed":
90  
-                Options.embed = "main"
91  
-            elif option.startswith("--embed="):
92  
-                Options.embed = options[8:]
93  
-            elif option.startswith("-I"):
94  
-                options.include_path.append(get_param(option))
95  
-            elif option == "--include-dir":
96  
-                options.include_path.append(pop_arg())
97  
-            elif option in ("-w", "--working"):
98  
-                options.working_path = pop_arg()
99  
-            elif option in ("-o", "--output-file"):
100  
-                options.output_file = pop_arg()
101  
-            elif option in ("-r", "--recursive"):
102  
-                options.recursive = 1
103  
-            elif option in ("-t", "--timestamps"):
104  
-                options.timestamps = 1
105  
-            elif option in ("-f", "--force"):
106  
-                options.timestamps = 0
107  
-            elif option in ("-v", "--verbose"):
108  
-                options.verbose += 1
109  
-            elif option in ("-p", "--embed-positions"):
110  
-                Options.embed_pos_in_docstring = 1
111  
-            elif option in ("-z", "--pre-import"):
112  
-                Options.pre_import = pop_arg()
113  
-            elif option == "--cleanup":
114  
-                Options.generate_cleanup_code = int(pop_arg())
115  
-            elif option in ("-D", "--no-docstrings"):
116  
-                Options.docstrings = False
117  
-            elif option in ("-a", "--annotate"):
118  
-                Options.annotate = True
119  
-            elif option == "--convert-range":
120  
-                Options.convert_range = True
121  
-            elif option == "--line-directives":
122  
-                options.emit_linenums = True
123  
-            elif option == "--no-c-in-traceback":
124  
-                options.c_line_in_traceback = False
125  
-            elif option == "--gdb":
126  
-                options.gdb_debug = True
127  
-                options.output_dir = os.curdir
128  
-            elif option == '-2':
129  
-                options.language_level = 2
130  
-            elif option == '-3':
131  
-                options.language_level = 3
132  
-            elif option == "--fast-fail":
133  
-                Options.fast_fail = True
134  
-            elif option in ('-Werror', '--warning-errors'):
135  
-                Options.warning_errors = True
136  
-            elif option in ('-Wextra', '--warning-extra'):
137  
-                options.compiler_directives.update(Options.extra_warnings)
138  
-            elif option == "--disable-function-redefinition":
139  
-                Options.disable_function_redefinition = True
140  
-            elif option == "--old-style-globals":
141  
-                Options.old_style_globals = True
142  
-            elif option == "--directive" or option.startswith('-X'):
143  
-                if option.startswith('-X') and option[2:].strip():
144  
-                    x_args = option[2:]
145  
-                else:
146  
-                    x_args = pop_arg()
  205
+            options.compiler_directives = {} #HACK
  206
+
  207
+        # Sets gathered Option and Debug options XXX
  208
+        import Cython.Compiler.DebugFlags as DebugFlags
  209
+        for module in Options, DebugFlags:
  210
+            for flag in vars(module):
147 211
                 try:
148  
-                    options.compiler_directives = Options.parse_directive_list(
149  
-                        x_args, relaxed_bool=True,
150  
-                        current_settings=options.compiler_directives)
151  
-                except ValueError, e:
152  
-                    sys.stderr.write("Error in compiler directive: %s\n" % e.args[0])
153  
-                    sys.exit(1)
154  
-            elif option.startswith('--debug'):
155  
-                option = option[2:].replace('-', '_')
156  
-                import DebugFlags
157  
-                if option in dir(DebugFlags):
158  
-                    setattr(DebugFlags, option, True)
  212
+                    setattr(module, flag, getattr(options, flag))
  213
+                except AttributeError:
  214
+                    pass
  215
+
  216
+        return vars(options), sources
  217
+
  218
+try:
  219
+    #import suca
  220
+    import argparse
  221
+
  222
+    class Parser(BasicParser, argparse.ArgumentParser):
  223
+        def __init__(self):
  224
+            argparse.ArgumentParser.__init__(self, epilog=epilog,
  225
+                formatter_class=argparse.RawDescriptionHelpFormatter,
  226
+                fromfile_prefix_chars='@')
  227
+            newgroup = lambda t: self.add_argument_group(title=t)
  228
+            for grouptitle, flaglist in menu.iteritems():
  229
+                group = self if grouptitle == 'default' else newgroup(grouptitle)
  230
+                for flag, parameters in flaglist.iteritems():
  231
+                    group.add_argument(*flag, **parameters)
  232
+
  233
+        def parse(self, args):
  234
+            #HACK: prevents first source file from being absorbed by --embed XXX
  235
+            try:
  236
+                pos = args.index('--embed')
  237
+            except ValueError:
  238
+                pass
  239
+            else:
  240
+                for arg in args[pos+1:]:
  241
+                    if arg.startswith('-'): break
159 242
                 else:
160  
-                    sys.stderr.write("Unknown debug flag: %s\n" % option)
161  
-                    bad_usage()
162  
-            elif option in ('-h', '--help'):
163  
-                sys.stdout.write(usage)
164  
-                sys.exit(0)
  243
+                    args.insert(pos + 1, '--')
  244
+
  245
+            results = self.parse_args(args)
  246
+            return self.refine(results)
  247
+
  248
+except ImportError:
  249
+    import optparse
  250
+
  251
+    class Parser(BasicParser, optparse.OptionParser):
  252
+        def __init__(self):
  253
+            optparse.OptionParser.__init__(self, epilog=epilog,
  254
+                version=menu['default'][('-V', '--version')]['version'],
  255
+                usage="%prog [options] sourcefile.{pyx,py} ...")
  256
+            newgroup = lambda t: optparse.OptionGroup(self, t)
  257
+            for grouptitle, flaglist in menu.iteritems():
  258
+                group = self if grouptitle == 'default' else newgroup(grouptitle)
  259
+                for flag, parameters in flaglist.iteritems():
  260
+                    try:
  261
+                        if '--embed' in flag:
  262
+                            del parameters['const'], parameters['nargs']
  263
+                            parameters['type'] = 'str'
  264
+                            parameters['action'] = 'callback'
  265
+                            parameters['callback'] = self.__embed_callback
  266
+
  267
+                        group.add_option(*flag, **parameters)
  268
+                    except optparse.OptionError:
  269
+                        pass
  270
+
  271
+                if group is not self: self.add_option_group(group)
  272
+
  273
+        def parse(self, args):
  274
+            results, others = self.parse_args(args)
  275
+            results.sources = others
  276
+            return self.refine(results)
  277
+
  278
+        # super HACK to support implicit --embed arg 'main' value
  279
+        @staticmethod
  280
+        def __embed_callback(option, opt_str, value, parser):
  281
+            if isidentifier(value):
  282
+                main = value
165 283
             else:
166  
-                sys.stderr.write("Unknown compiler flag: %s\n" % option)
167  
-                sys.exit(1)
168  
-        else:
169  
-            sources.append(pop_arg())
170  
-    if options.use_listing_file and len(sources) > 1:
171  
-        sys.stderr.write(
172  
-            "cython: Only one source file allowed when using -o\n")
173  
-        sys.exit(1)
174  
-    if len(sources) == 0 and not options.show_version:
175  
-        bad_usage()
176  
-    if Options.embed and len(sources) > 1:
177  
-        sys.stderr.write(
178  
-            "cython: Only one source file allowed when using -embed\n")
179  
-        sys.exit(1)
180  
-    return options, sources
  284
+                main = 'main'
  285
+                parser.rargs.insert(0, value)
  286
+            setattr(parser.values, option.dest, main)
  287
+
  288
+makedebuglist()
  289
+parser = Parser()
181 290
 
  291
+def parse_command_line(args):
  292
+    return parser.parse(args)
20  Cython/Compiler/Main.py
@@ -509,12 +509,12 @@ class CompilationOptions(object):
509 509
 
510 510
     def __init__(self, defaults = None, **kw):
511 511
         self.include_path = []
512  
-        if defaults:
513  
-            if isinstance(defaults, CompilationOptions):
514  
-                defaults = defaults.__dict__
  512
+        if isinstance(defaults, CompilationOptions):
  513
+            options = defaults.__dict__
515 514
         else:
516  
-            defaults = default_options
517  
-        self.__dict__.update(defaults)
  515
+            options = default_options
  516
+
  517
+        self.__dict__.update(options)
518 518
         self.__dict__.update(kw)
519 519
 
520 520
     def create_context(self):
@@ -643,14 +643,14 @@ def main(command_line = 0):
643 643
     any_failures = 0
644 644
     if command_line:
645 645
         from CmdLine import parse_command_line
646  
-        options, sources = parse_command_line(args)
  646
+        flags, sources = parse_command_line(args)
647 647
     else:
648  
-        options = CompilationOptions(default_options)
  648
+        flags = {}
649 649
         sources = args
650 650
 
651  
-    if options.show_version:
652  
-        sys.stderr.write("Cython version %s\n" % Version.version)
653  
-    if options.working_path!="":
  651
+    options = CompilationOptions(default_options, **flags)
  652
+
  653
+    if options.working_path:
654 654
         os.chdir(options.working_path)
655 655
     try:
656 656
         result = compile(sources, options)
145  Cython/Compiler/Tests/TestCommandLine.py
... ...
@@ -0,0 +1,145 @@
  1
+#
  2
+#   Copyright 2011 Stefano Sanfilippo <satufk on GitHub>
  3
+#
  4
+#   Licensed under the Apache License, Version 2.0 (the "License");
  5
+#   you may not use this file except in compliance with the License.
  6
+#   You may obtain a copy of the License at
  7
+#
  8
+#       http://www.apache.org/licenses/LICENSE-2.0
  9
+#
  10
+#   Unless required by applicable law or agreed to in writing, software
  11
+#   distributed under the License is distributed on an "AS IS" BASIS,
  12
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13
+#   See the License for the specific language governing permissions and
  14
+#   limitations under the License.
  15
+
  16
+import os
  17
+import multiprocessing
  18
+
  19
+from Cython.TestUtils import CythonTest
  20
+from Cython.Compiler.CmdLine import parse_command_line
  21
+
  22
+import Cython.Compiler.Options as Options
  23
+import Cython.Compiler.DebugFlags as DebugFlags
  24
+
  25
+def parse(string):
  26
+    return parse_command_line(string.split())
  27
+
  28
+def parser_return_code(string):
  29
+    '''Return code of Cython, if it had been invoked with <string> as cline.'''
  30
+    p = multiprocessing.Process(target=parse_command_line, args=[string.split()])
  31
+    p.start()
  32
+    p.join()
  33
+    return p.exitcode
  34
+
  35
+FAILURE = 2
  36
+
  37
+class TestCommandLine(CythonTest):
  38
+    def test_source_recognition(self):
  39
+        opt, src = parse('-z test -t -f -I. source.pyx')
  40
+        self.assertEqual(src, ['source.pyx'])
  41
+        opt, src = parse('-z=test -t -f -I. sourceXX.pyx')
  42
+        self.assertEqual(src, ['sourceXX.pyx'])
  43
+        opt, src = parse('-I. source.pyx source2.pyx -f -D')
  44
+        self.assertEqual(src, ['source.pyx', 'source2.pyx'])
  45
+        #opt, src = parse('-I. source.pyx -f -D source2.pyx')
  46
+        #self.assertEqual(src, ['source.pyx', 'source2.pyx'])
  47
+        opt, src = parse('-I. -f -D source.pyx source2.pyx')
  48
+        self.assertEqual(src, ['source.pyx', 'source2.pyx'])
  49
+
  50
+    def test_embed_recognition(self):
  51
+        opt, src = parse('--embed --gdb -2  test.pyx')
  52
+        self.assertEqual(Options.embed, 'main')
  53
+        self.assertEqual(src, ['test.pyx'])
  54
+        opt, src = parse('--gdb -2 --embed -- test.pyx')
  55
+        self.assertEqual(Options.embed, 'main')
  56
+        self.assertEqual(src, ['test.pyx'])
  57
+        opt, src = parse('--gdb -2 --embed test.pyx')
  58
+        self.assertEqual(Options.embed, 'main')
  59
+        self.assertEqual(src, ['test.pyx'])
  60
+        opt, src = parse('--fast-fail --embed mainX -f --gdb s.py')
  61
+        self.assertEqual(Options.embed, 'mainX')
  62
+        #XXX should not be used
  63
+        #opt, src = parse('--fast-fail -f --gdb s.py --embed mainZ')
  64
+        #self.assertEqual(src, ['s.py'])
  65
+        #self.assertEqual(Options.embed, 'mainZ')
  66
+        self.assertEqual(
  67
+            parser_return_code('--fast-fail -f --embed a.pyx b.py'),
  68
+            FAILURE)
  69
+        self.assertEqual(
  70
+            parser_return_code('--fast-fail -f --embed a.pyx b.py c.py'),
  71
+            FAILURE)
  72
+
  73
+    def test_embed_explicit_recognition(self):
  74
+        opt, src = parse('--fast-fail -f --embed=mainY --gdb s.py')
  75
+        self.assertEqual(Options.embed, 'mainY')
  76
+        self.assertEqual(
  77
+            parser_return_code('--fast-fail -f --embed=mainY --gdb'),
  78
+            FAILURE)
  79
+        self.assertEqual(
  80
+            parser_return_code('--fast-fail -f --embed=mainY --gdb a.pyx b.py'),
  81
+            FAILURE)
  82
+
  83
+    def test_include_recognition(self):
  84
+        opt, src = parse('-v -r -Igreedy -w x test.pyx')
  85
+        self.assertEqual(opt['include_path'], ['greedy'])
  86
+        self.assertEqual(src, ['test.pyx'])
  87
+        opt, src = parse('-I/usr/include/mistery/dot test.pyx')
  88
+        self.assertEqual(opt['include_path'], ['/usr/include/mistery/dot'])
  89
+        opt, src = parse('-I /usr/include/mistery/dot test.pyx')
  90
+        self.assertEqual(opt['include_path'], ['/usr/include/mistery/dot'])
  91
+
  92
+    def test_directive_options(self):
  93
+        def issubset(universe, dic):
  94
+            '''True if <dic> is a subset of <universe>'''
  95
+            for k, v in dic.iteritems():
  96
+                try:
  97
+                    if universe[k] != v:
  98
+                        return False
  99
+                except KeyError:
  100
+                    return False
  101
+            return True
  102
+
  103
+        opt, src = parse('-Xboundscheck=False test.pyx')
  104
+        self.failIf(opt['compiler_directives']['boundscheck'])
  105
+        opt, src = parse('-Xnonecheck=True test.pyx')
  106
+        self.failUnless(opt['compiler_directives']['nonecheck'])
  107
+        opt, src = parse('-Xcdivision=True test.pyx')
  108
+        self.failUnless(opt['compiler_directives']['cdivision'])
  109
+        opt, src = parse('-Xlanguage_level=2,profile=True test.pyx')
  110
+        self.failUnless(issubset(opt['compiler_directives'],
  111
+            {'profile': True, 'language_level': 2}))
  112
+        opt, src = parse('--directive callspec=stdcall,final=True test.pyx')
  113
+        self.failUnless(issubset(opt['compiler_directives'],
  114
+            {'callspec': 'stdcall', 'final': True}))
  115
+        opt, src = parse('-Xauto_cpdef=True,profile=True -I. -Xinternal=True test.pyx')
  116
+        self.failUnless(issubset(opt['compiler_directives'],
  117
+            {'profile': True, 'internal': True, 'auto_cpdef': True} ))
  118
+
  119
+    def test_output_options(self):
  120
+        opt, src = parse('-I. -I /usr -o mytestexec test.pyx')
  121
+        self.assertEqual(opt['output_file'], 'mytestexec')
  122
+        opt, src = parse('-I. -o2mytestexec -I /usr test.pyx')
  123
+        self.assertEqual(opt['output_file'], '2mytestexec')
  124
+
  125
+    def test_if_arguments_are_required_correctly(self):
  126
+        self.assertEqual(parser_return_code('-I'), FAILURE)
  127
+        self.assertEqual(parser_return_code(''), FAILURE)
  128
+        self.assertEqual(parser_return_code('-X source.py'), FAILURE)
  129
+
  130
+    def test_debug_options(self):
  131
+        opt, src = parse('-I. --debug-temp-code-comments test.pyx')
  132
+        self.failUnless(DebugFlags.debug_temp_code_comments)
  133
+        opt, src = parse('--debug-trace-code-generation test.pyx')
  134
+        self.failUnless(DebugFlags.debug_trace_code_generation)
  135
+        opt, src = parse('--debug-verbose-pipeline test.pyx')
  136
+        self.failUnless(DebugFlags.debug_verbose_pipeline)
  137
+
  138
+    @classmethod
  139
+    def tearDownClass(cls):
  140
+        # Clean up permanent options
  141
+        reload(Options)
  142
+
  143
+if __name__ == '__main__':
  144
+    import unittest
  145
+    unittest.main()
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.