Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve non-trivial datatypes #1808

Merged
merged 145 commits into from
Apr 16, 2024
Merged

Improve non-trivial datatypes #1808

merged 145 commits into from
Apr 16, 2024

Conversation

EmilyBourne
Copy link
Member

@EmilyBourne EmilyBourne commented Mar 27, 2024

Move the storage of rank and order from ast.basic.TypedAstNode to an internal property of ast.datatypes.PyccelType objects (the properties are still exposed via ast.basic.TypedAstNode). This allows mixed ordered objects to be described and therefore created (e.g. list[float[:,:](order=F)]). A test is added for this case. This change should make #1583 simple to implement.
Making the rank and order part of the type fixes #1821 . This provides enough information to allow lists of arrays. Fixes #1721

In order to correctly index types such as list[float[:,:](order=F)] the creation of an IndexedElement in the syntactic stage is modified. Instead of collapsing all the indices into one IndexedElement an IndexedElement is created for each ast.Subscript. In the semantic stage this is collapsed into one IndexedElement for each container.

E.g. for the following code:

import numpy as np
a = (np.ones((3,4)), np.zeros((3,4)))
a[0][1][2]

On the devel branch after the syntactic stage the last line is described as IndexedElement('a', (0, 1, 2)). This persists to the codegen stage.
On this branch after the syntactic stage the last line is described as IndexedElement(IndexedElement(IndexedElement('a', (0,)), (1,)), (2,)). After the semantic stage it becomes IndexedElement(IndexedElement('a', (0,)), (1,2)). The printers are modified to handle this. The generated code is not changed (as support for lists has not yet been added) so the indices are still used together to index a 3D array.

Commit Summary

  • Update documentation
  • Add missing if __name__ == '__main__' on old documentation examples.
  • Remove static_rank and static_order properties from pyccel.ast.basic. These properties are now contained in static_type.
  • Remove _rank and _order properties from TypedAstNodes.
  • Change PyccelOperator.set_shape_rank functions to PyccelOperator.set_shape functions (the rank must now be determined at the same time as the type).
  • Remove PyccelOperator._set_order functions.
  • Simplify homogeneity checks for tuples, lists and sets
  • Ensure the shape of an empty list/tuple literal is set correctly.
  • Correct the class type of CmathPolar
  • Remove the order argument from the Allocate constructor (the information is now retrieved from the order of the allocated variable).
  • Add the properties rank and order to pyccel.ast.datatypes.PyccelType.
  • Add a switch_rank function to pyccel.ast.datatypes.HomogeneousContainerType.
  • Add a container_rank property to pyccel.ast.datatypes.HomogeneousContainerType.
  • Parameterise NumpyNDArrayType by the rank and the order
  • Change the argument of NumpyNewArray from dtype to class_type (to set the rank/order in the subclass).
  • Allow NumpyArray to handle mixed rank objects (e.g. list of F-ordered arrays).
  • Rename NumpyUfuncBase._set_shape_rank to NumpyUfuncBase._get_shape_rank and return the shape and rank to avoid saving the rank unnecessarily.
  • Rename NumpyUfuncBase._set_order to NumpyUfuncBase._get_order and pass the rank and return the order to avoid saving unnecessarily.
  • Add docstrings.
  • Add a NumpyNDArrayType.__new__ function which redirects rank 0 arrays to scalar types.
  • Augment NumpyNDArrayType.__add__ to handle rank and ordering.
  • Add a NumpyNDArrayType.swap_order function to change between C and F ordered equivalent types.
  • Add __str__ or __repr__ functions to PyccelTypes such that printing them gives a valid type annotation.
  • Add a NumpyInt object to easily find the NumPy integer which has the same precision as Python integers.
  • Remove rank and order arguments from pyccel.ast.type_annotations.VariableTypeAnnotation constructor (information now available in class_type).
  • Remove order property from pyccel.ast.type_annotations.VariableTypeAnnotation (rank is retained for now).
  • Update vector expression unravelling functions to handle multiple levels of IndexedElements.
  • Improve docstrings in pyccel.ast.utilities.
  • Remove rank and order arguments from pyccel.ast.variable.Variable constructor (the information is now retrieved from the class_type).
  • Remove unused property is_stack_scalar.
  • Simplify is_ndarray method.
  • Add _is_slice attribute to IndexedElement to indicate if an element or a slice of the base is represented.
  • Add allows_negative_indexes property to IndexedElement.
  • Update _print_IndexedElement to handle multi-level IndexedElements.
  • Correct abs call in _print_NumpyNorm
  • Improve error message for wrong arguments.
  • Allocate strings on the stack to avoid calling Allocate (to be improved with Strings #459 ).
  • Remove get_type_description (this is now handled by PyccelType.__str__).
  • Provide a traceback to errors.report to allow the location of a TypeError raised during a FunctionCall to be more easily located.
  • Stop collapsing ast.Subscript into one IndexedElement object.
  • Add some mixed ordered tests.
  • Disable tests with ambiguous interfaces (see Ambiguous interface doesn't raise an error #1821).

@pyccel-bot
Copy link

pyccel-bot bot commented Mar 27, 2024

Hello again! Thank you for this new pull request 🤩.

Please begin by requesting your checklist using the command /bot checklist

@EmilyBourne EmilyBourne linked an issue Apr 2, 2024 that may be closed by this pull request
@EmilyBourne
Copy link
Member Author

/bot run pr_tests

pyccel/ast/operators.py Outdated Show resolved Hide resolved
Co-authored-by: Yaman Güçlü <yaman.guclu@gmail.com>
Copy link

@pyccel-bot pyccel-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be lines in this PR which aren't tested. Please take a look at my comments and add tests which cover the new code.

If this is modified code which cannot be easily tested in this PR please open an issue to request that this code be either removed or tested. Once you have done that please leave a message on the relevant conversation beginning with the line /bot accept and referencing the issue.

Similarly if the new code cannot be tested for some reason, please leave a comment beginning with the line /bot accept on the relevant conversation explaining why the code can't be tested.

pyccel/ast/operators.py Outdated Show resolved Hide resolved
@github-actions github-actions bot marked this pull request as draft April 16, 2024 09:05
@pyccel-bot
Copy link

pyccel-bot bot commented Apr 16, 2024

Unfortunately your PR is not passing the tests so it is not quite ready for review yet. Let me know when it is fixed with /bot mark as ready.

@EmilyBourne
Copy link
Member Author

/bot run pr_tests

@EmilyBourne EmilyBourne marked this pull request as ready for review April 16, 2024 09:17
Copy link

@pyccel-bot pyccel-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be lines in this PR which aren't tested. Please take a look at my comments and add tests which cover the new code.

If this is modified code which cannot be easily tested in this PR please open an issue to request that this code be either removed or tested. Once you have done that please leave a message on the relevant conversation beginning with the line /bot accept and referencing the issue.

Similarly if the new code cannot be tested for some reason, please leave a comment beginning with the line /bot accept on the relevant conversation explaining why the code can't be tested.

@@ -972,7 +938,7 @@
elif len(possible_class_types) == 1:
class_type = possible_class_types.pop().switch_basic_type(dtype)
else:
description = f"({arg1} {cls.op} {arg2})" # pylint: disable=no-member
description = f"({arg1!r} {cls.op} {arg2!r})" # pylint: disable=no-member
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code isn't tested. Please can you take a look

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/bot accept
Error testing

@pyccel-bot
Copy link

pyccel-bot bot commented Apr 16, 2024

Unfortunately your PR is not passing the tests so it is not quite ready for review yet. Let me know when it is fixed with /bot mark as ready.

@EmilyBourne EmilyBourne marked this pull request as ready for review April 16, 2024 09:36
Copy link

@pyccel-bot pyccel-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job ! Your PR is using all the code it added/changed.

@yguclu yguclu merged commit e36726f into devel Apr 16, 2024
16 checks passed
@yguclu yguclu deleted the devel-issue1794 branch April 16, 2024 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
urgent Affects all PRs and should be merged as soon as possible
Projects
None yet
2 participants