Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

array expressions #144

Open
wants to merge 56 commits into from

2 participants

@markflorisson
Collaborator

Element-wise array expressions for Cython

markflorisson added some commits
@markflorisson markflorisson Split buffer and memoryview operations from IndexNode fbae57b
@markflorisson markflorisson Split analyse_types of IndexNode some more 26d7c28
@markflorisson markflorisson Fix copy/copy_fortran return type & tests 1df5077
@markflorisson markflorisson Refactor memoryview dtype validation 98d0d60
@markflorisson markflorisson Fix transpose memoryview type 36f037b
@markflorisson markflorisson Refactor slice assignment and copying 922af21
@markflorisson markflorisson Fix python 3 import 9536b62
@markflorisson markflorisson Start on elementwise expressions 6244394
@markflorisson markflorisson Allow strided memoryviews to be copied to contiguous memoryviews 148c290
@markflorisson markflorisson Specialize for contiguous operands 9b0fc9e
@markflorisson markflorisson Add minivect as submodule 1217a97
@markflorisson markflorisson Choose typedefed type in widest_numeric_type & support tyepdef Cython…
… types in vector expressions
3fdcc52
@markflorisson markflorisson Allow arbitrary Cython types compatible with C expressions as vector …
…expression dtypes
31460e8
@markflorisson markflorisson Support complex numbers in vector expressions 1e53ae2
@markflorisson markflorisson Support objects in vector expressions 451c802
@markflorisson markflorisson Use array funcarg method f4496bc
@markflorisson markflorisson Select specialization at runtime & support broadcasting b8ab0f3
@markflorisson markflorisson Support overlapping memory and broadcasting assignment
    (Avoid recomputation along broadcasting axes)
    (Need some tests)
c7db788
@markflorisson markflorisson DECREF lhs in slice assignment w/ dtype object c4e72cf
@markflorisson markflorisson Don't choose contig specialization for broadcasting operations known …
…at compile time
e86e5aa
@markflorisson markflorisson Remove broadcasting leading dimensions from RHS 48672b0
@markflorisson markflorisson Split vector expression tests & add runtime broadcasting test bc22f9f
@markflorisson markflorisson Select specialization at runtime 32e6671
@markflorisson markflorisson Support tiling specializations aae2d35
@markflorisson markflorisson Let the LHS participate in determining array layout order b13c9e2
@markflorisson markflorisson Implement a code cache 46fe298
@markflorisson markflorisson Support scalar arguments a02e733
@markflorisson markflorisson Support unary operations d640297
@markflorisson markflorisson Support non-slice array expression assignment 62f8783
@markflorisson markflorisson Support non-assignment array expressions ce10532
@markflorisson markflorisson Fix broadcasting & add omitted tests 0653f04
@markflorisson markflorisson Support restrict and const qualifiers for array expression arguments 6a6e9a7
@markflorisson markflorisson Support inner-contiguous specialization selection 94ddebd
@markflorisson markflorisson Map more types to minivect, change qualify to use immutable types e19b8a7
@markflorisson markflorisson Support OpenMP in array expressions d22b3c4
@markflorisson markflorisson Filter duplicate expression arguments ef46dfd
@markflorisson markflorisson Add auto-tuner for square tiling blocksize (+caching) 2182b34
@markflorisson markflorisson Name mangle all variable and function names in the tiling utility code 842952f
@markflorisson markflorisson Make tiling blocksize type signed (avoid compiler warnings about sign…
…ed/unsigned comparison)
cbe47f8
@markflorisson markflorisson Divide tiling blocksize by itemsize 5c8409c
@markflorisson markflorisson Auto-tune OpenMP size e413b45
@markflorisson markflorisson Test various binary operators dd455de
@markflorisson markflorisson Support (partial) elementwise function calls b42b157
@markflorisson markflorisson Avoid tiled specializations if all operands are C or F contig d41d090
@markflorisson markflorisson Support vectorized specializations ebc8850
@markflorisson markflorisson Add support for graphviz AST visualisation c596986
@markflorisson markflorisson Use more sensible output filenames for graphviz .dot files 7c5f2f2
@markflorisson markflorisson Only print debug code for expression 1ef2ebd
@markflorisson markflorisson Add variable resolving mixin class adbb911
@markflorisson markflorisson Fix code caching for array expressions 53b77f7
@markflorisson markflorisson Omit type information for graphviz 1719bdb
@markflorisson markflorisson Resolve non-external typedefs for array expressions d146ac7
@markflorisson markflorisson Add some more documentation 4ae867f
@markflorisson markflorisson Create minivect function type for function calls bdaeea5
@scoder scoder referenced this pull request
Closed

refactor indexnode2 #137

@robertwb robertwb commented on the diff
Cython/Compiler/Vector.py
((24 lines not shown))
+
+# Print generated minivect code
+_debug = False
+# Generate debug calls from specialized functions
+_context_debug = False
+
+#
+### Graphviz related things. .dot files are only written when write_graphviz
+### is true.
+#
+graphviz_out_filename_unspecialized = os.path.expanduser("~/ast.dot")
+graphviz_out_filename = os.path.expanduser("~/ast%s.dot")
+write_graphviz = False
+
+# Macro that should be defined to enable explicit vectorization
+cython_vector_size = "CYTHON_VECTOR_SIZE"
@robertwb Owner

Meaning of this value?

@markflorisson Collaborator

Compiling with -DCYTHON_VECTOR_SIZE=X enables vectorization, with 4 meaning SSE2 (4 floats), and 8 meaning AVX (8 floats).

The new branch is called _array_expressions_rebased, but the py3k build segfaults with a refcount error. I still need to investigate that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@robertwb robertwb commented on the diff
Cython/Compiler/Vector.py
((43 lines not shown))
+ Map Cython types to minitypes.
+ """
+
+ def map_type(self, type, wrap=False):
+ if type.is_typedef:
+ if type.typedef_is_external:
+ return minitypes.TypeWrapper(type, self.context)
+ else:
+ type = type.resolve()
+
+ if type.is_memoryviewslice:
+ dtype = self.map_type(type.dtype, wrap=wrap)
+ return minitypes.ArrayType(dtype, len(type.axes),
+ is_c_contig=type.is_c_contig,
+ is_f_contig=type.is_f_contig)
+ elif type.is_float:
@robertwb Owner

Would these be better expressed as a dictionary?

@markflorisson Collaborator

Yes I think that would be nicer.

@markflorisson Collaborator

Oh I remember why I didn't do that now. It's because you can create new types, and for instance c_sint_type and c_int_type are the same, but don't compare equal (I'm not sure why?).

@robertwb Owner

Like typedefs I guess. Does this then assume you know the exact size at compile time? Hmm...

@markflorisson Collaborator

No, it needs that only for the LLVM backend (which isn't used by Cython). It uses the external typedef name if the typedef is external, otherwise it uses the actual types to allow code reuse of expressions of equivalent types. I got everything working, except for one pyximport test, but I think that might be my test setup. I'll try a new PR next weekend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@robertwb robertwb commented on the diff
Cython/Compiler/Vector.py
((959 lines not shown))
+ code.putln("else {")
+
+ code.putln("/* Strided specializations */")
+ self.put_ordered_specializations(code, specializers.StridedSpecializer,
+ specializers.StridedFortranSpecializer)
+
+ if if_clause != "if":
+ code.putln("}")
+
+ def generate_result_code(self, code):
+ "Generate a branch and call to each specialization"
+ contig, mixed_contig, c_contig, f_contig = all_c_or_f_contig(self.operands)
+
+ self.context.original_cython_code = code
+
+ if_clause = "if"
@robertwb Owner

unused code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@robertwb
Owner

In my (albeit somewhat cursory) perusal of the code, I think it looks good. Is there any way to test that your code is being used (i.e. unnecessary intermediates are not being created) as well as getting the correct output? (Perhaps an object type that prints out arithmetic operations could verify the expected sequence of operations, though this might be too much subject to change). Also, it'd be good to have a link somewhere to your thesis or other high-level overview.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 23, 2012
  1. @markflorisson
  2. @markflorisson
  3. @markflorisson
  4. @markflorisson
  5. @markflorisson
  6. @markflorisson
  7. @markflorisson

    Fix python 3 import

    markflorisson authored
  8. @markflorisson
  9. @markflorisson
  10. @markflorisson
  11. @markflorisson
  12. @markflorisson
  13. @markflorisson
  14. @markflorisson
  15. @markflorisson
  16. @markflorisson

    Use array funcarg method

    markflorisson authored
  17. @markflorisson
  18. @markflorisson

    Support overlapping memory and broadcasting assignment

    markflorisson authored
        (Avoid recomputation along broadcasting axes)
        (Need some tests)
  19. @markflorisson
  20. @markflorisson
  21. @markflorisson
  22. @markflorisson
  23. @markflorisson
  24. @markflorisson
  25. @markflorisson
  26. @markflorisson

    Implement a code cache

    markflorisson authored
  27. @markflorisson

    Support scalar arguments

    markflorisson authored
  28. @markflorisson

    Support unary operations

    markflorisson authored
  29. @markflorisson
  30. @markflorisson
  31. @markflorisson
  32. @markflorisson
  33. @markflorisson
  34. @markflorisson
  35. @markflorisson
  36. @markflorisson
  37. @markflorisson
  38. @markflorisson
  39. @markflorisson
  40. @markflorisson
  41. @markflorisson

    Auto-tune OpenMP size

    markflorisson authored
  42. @markflorisson
  43. @markflorisson
  44. @markflorisson
  45. @markflorisson
  46. @markflorisson
  47. @markflorisson
Commits on Aug 1, 2012
  1. @markflorisson
Commits on Aug 6, 2012
  1. @markflorisson
Commits on Aug 11, 2012
  1. @markflorisson
Commits on Aug 13, 2012
  1. @markflorisson
Commits on Aug 21, 2012
  1. @markflorisson
Commits on Aug 24, 2012
  1. @markflorisson
Commits on Sep 16, 2012
  1. @markflorisson
Commits on Oct 14, 2012
  1. @markflorisson
  2. @markflorisson
Something went wrong with that request. Please try again.