Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean PythonCodePrinter and add dtype property #1260

Merged
merged 58 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
d2ec6a8
Correct duplicate function with missing contents
EmilyBourne Dec 14, 2022
ca1c047
Merge branch 'devel' into devel-issue1258
EmilyBourne Jul 28, 2023
06023c5
Add NumpyResultType class. Improve docstrings. Save init_dtype
EmilyBourne Aug 1, 2023
731c093
Add array.dtype
EmilyBourne Aug 1, 2023
53c1ff1
Improve python printing
EmilyBourne Aug 1, 2023
c9c0d42
Merge branch 'devel' into devel-issue1258
EmilyBourne Aug 1, 2023
dd69bb5
Class docstring
EmilyBourne Aug 1, 2023
9aea2df
Add dtype property test
EmilyBourne Aug 1, 2023
d19e291
Handle NumpyResultType. Correct warning. Simplify _print_NumpyAutoFill
EmilyBourne Aug 2, 2023
59a2c7d
Add missing name
EmilyBourne Aug 2, 2023
35ad1be
Print more familiar code
EmilyBourne Aug 2, 2023
517a64d
Correct doc format
EmilyBourne Aug 2, 2023
665298b
Reduce duplication and ensure types are imported
EmilyBourne Aug 2, 2023
b64ca1b
Reduce code duplication. Add support for multiple arguments to result…
EmilyBourne Aug 2, 2023
e7af045
Add _print_PyccelFunctionDef
EmilyBourne Aug 2, 2023
01211ef
Whitespace. Add test for np.result_type
EmilyBourne Aug 2, 2023
2503c72
Update CHANGELOG and documentation
EmilyBourne Aug 2, 2023
45da89d
Remove unnecessary import
EmilyBourne Aug 2, 2023
2783be3
Docstrings
EmilyBourne Aug 2, 2023
f81d219
Put back error. Correct type
EmilyBourne Aug 2, 2023
65d0eb1
Correct linspace dtype
EmilyBourne Aug 2, 2023
a29f50c
Correct NumpyMod precision
EmilyBourne Aug 2, 2023
86e8765
Fix docstring. Fix type name
EmilyBourne Aug 2, 2023
32a211c
Fix typo
EmilyBourne Aug 2, 2023
29d11a6
put back old code
EmilyBourne Aug 2, 2023
c56347a
Revert change
EmilyBourne Aug 3, 2023
9022c46
Add support for short type names
EmilyBourne Aug 3, 2023
4656fdb
Correct type printing
EmilyBourne Aug 3, 2023
881c56f
NumPy's bool is deprecated. Default to Python bool
EmilyBourne Aug 3, 2023
4792938
Ensure import
EmilyBourne Aug 3, 2023
a60674c
Avoid failure in developer mode. Fix language
EmilyBourne Aug 3, 2023
ce59a4a
Handle default bool
EmilyBourne Aug 3, 2023
57915ee
Preserve import order
EmilyBourne Aug 3, 2023
4eef7f7
Remove old skips
EmilyBourne Aug 3, 2023
e37e2e8
Handle default bool
EmilyBourne Aug 3, 2023
7143cce
Use default bool
EmilyBourne Aug 3, 2023
d069b9a
Update CHANGELOG
EmilyBourne Aug 3, 2023
a4616c9
Use default bool
EmilyBourne Aug 3, 2023
07783f0
Raise error if saving NumpyResultType to variable
EmilyBourne Aug 3, 2023
b6cbe28
Simplify output file now C tests are not verbose
EmilyBourne Aug 3, 2023
dff03eb
Correct variable
EmilyBourne Aug 3, 2023
9b509e5
Try to trigger code
EmilyBourne Aug 3, 2023
1acdf42
Remove dead code
EmilyBourne Aug 3, 2023
c91fb10
Extra docstring. Fix import
EmilyBourne Aug 3, 2023
2a73013
Trailing whitespace
EmilyBourne Aug 3, 2023
fc8a642
Use f-strings
EmilyBourne Aug 3, 2023
ee37f98
Empty commit to trigger Codacy
EmilyBourne Aug 4, 2023
ef957ad
Merge branch 'devel' into devel-issue1258
EmilyBourne Aug 7, 2023
64f5df7
Merge branch 'devel' into devel-issue1258
EmilyBourne Aug 7, 2023
9553f65
Merge branch 'devel' into devel-issue1258
EmilyBourne Aug 7, 2023
9e2e6e8
Merge branch 'devel' into devel-issue1258
EmilyBourne Aug 14, 2023
f92383e
Merge branch 'devel' into devel-issue1258
yguclu Aug 14, 2023
e770a5d
Fix alignment
EmilyBourne Aug 14, 2023
a8886da
Typo
EmilyBourne Aug 14, 2023
43cb294
Typo
EmilyBourne Aug 14, 2023
7eca256
Reduce code duplication
EmilyBourne Aug 14, 2023
31160ae
Fix capitalisation
EmilyBourne Aug 14, 2023
38086f9
Clarify function
EmilyBourne Aug 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ All notable changes to this project will be documented in this file.
- Allow interfaces in classes.
- Python support for a simple class.
- #1430 : Added conjugate support to booleans.
- #1260 : Add support for NumPy `dtype` property: `a.dtype`.
- #1260 : Add support for NumPy `result_type` function.

### Fixed

Expand All @@ -30,6 +32,7 @@ All notable changes to this project will be documented in this file.

### Deprecated

- Stop generating `numpy.bool` (deprecated from NumPy) in code.
- \[INTERNALS\] Removed `obsolete` folder.
- \[INTERNALS\] Removed out of date `samples` folder.
- \[INTERNALS\] Removed out of date `doc` folder.
Expand Down
59 changes: 20 additions & 39 deletions ci_tools/json_pytest_output.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import re
import sys

def mini_md_summary(title, outcome, c, f, py):
def mini_md_summary(title, outcome, failed_tests):
"""
Generate Markdown.

Expand All @@ -30,12 +30,8 @@ def mini_md_summary(title, outcome, c, f, py):
The title of the summary.
outcome : str
The result of a completed step, Possible values are success, failure, cancelled, or skipped.
c : list
A list of C test summaries, (Failed tests).
f : list
A list of Fortran test summaries, (Failed tests).
py : list
A list of Python test summaries, (Failed tests).
failed_tests : dict
A dictionary whose keys are languages and whose values are lists of failed test summaries.

Returns
-------
Expand All @@ -44,21 +40,12 @@ def mini_md_summary(title, outcome, c, f, py):
"""
md = f"## {title} - {outcome} "
if outcome == "failure":
if len(c) != 0:
md = md + '\n' + "### C Test summary: "
md = md + '\n'
for i in c:
md = md + i + "\n"
if len(f) != 0:
md = md + '\n' + "### Fortran Test summary: "
md = md + '\n'
for i in f:
md = md + i + "\n"
if len(py) != 0:
md = md + '\n' + "### Python Test summary: "
md = md + '\n'
for i in py:
md = md + i + "\n"
for lang, errs in failed_tests.items():
if len(errs) != 0:
md = md + '\n' + f"### {lang.capitalize()} Test summary: "
md = md + '\n'
for i in errs:
md = md + i + "\n"
md = md + "\n"
return(md)

Expand All @@ -73,6 +60,10 @@ def mini_md_summary(title, outcome, c, f, py):
output_file = 'test_json_result.json'
summary = ""

failed_pattern = re.compile(r".*FAILED.*")
languages = ('c', 'fortran', 'python')
pattern = {lang: re.compile(r".*\["+lang+r"\]\ \_.*") for lang in languages}

for i in p_args.tests:
values = i.split(':')
mini_title = values[0] if len(values) >= 1 else None
Expand All @@ -86,28 +77,18 @@ def mini_md_summary(title, outcome, c, f, py):
c_tests = []
f_tests = []
py_tests = []
failed_pattern = r".*FAILED.*"
c_pattern = r".*\[c\].*"
f_pattern = r".*\[fortran\]\ \_.*"
py_pattern = r".*\[python\]\ \_.*"

failed_matches = re.findall(failed_pattern, outfile, re.MULTILINE)
failed_matches = failed_pattern.findall(outfile, re.MULTILINE)
failed_matches = [re.sub(r'.*FAILED ', "- ``", string) for string in failed_matches]

r = re.compile(c_pattern)
c_failed = list(filter(r.match, failed_matches))
c_failed = [re.sub(r'\[c\]', "`` :heavy_multiplication_x:", string) for string in c_failed]

failed_matches = re.findall(f_pattern, outfile, re.MULTILINE)
failed_matches = ["- ``" + string.strip('_') for string in failed_matches]
f_failed = [re.sub(r'\[fortran\]', "`` :heavy_multiplication_x:", string) for string in failed_matches]

failed_matches = re.findall(py_pattern, outfile, re.MULTILINE)
failed_matches = ["- ``" + string.strip('_') for string in failed_matches]
py_failed = [re.sub(r'\[python\]', "`` :heavy_multiplication_x:", string) for string in failed_matches]
fails = {}

for lang in languages:
failed_matches = pattern[lang].findall(outfile, re.MULTILINE)
failed_matches = ["- ``" + string.strip('_') for string in failed_matches]
fails[lang] = [re.sub(r'\['+lang+r'\]', "`` :heavy_multiplication_x:", string) for string in failed_matches]

summary = summary + mini_md_summary(mini_title, outcome, c_failed, f_failed, py_failed)
summary = summary + mini_md_summary(mini_title, outcome, fails)

print(summary)
json_ouput = {
Expand Down
2 changes: 1 addition & 1 deletion docs/numpy-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -570,6 +570,6 @@ In Pyccel we try to support the NumPy functions which developers use the most..

- others:

- `amax`, `amin`, `sum`, `shape`, `size`, `floor`, `sign`
- `amax`, `amin`, `sum`, `shape`, `size`, `floor`, `sign`, `result_type`

If discrepancies beyond round-off error are found between [NumPy](https://numpy.org/doc/stable/reference/)'s and [Pyccel](https://github.com/pyccel/pyccel)'s results, please create an issue at <https://github.com/pyccel/pyccel/issues> and provide a small example of your problem. Do not forget to specify your target language.
6 changes: 4 additions & 2 deletions pyccel/ast/class_defs.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@
NativeComplex, NativeString, NativeNumeric)
from .numpyext import (NumpyShape, NumpySum, NumpyAmin, NumpyAmax,
NumpyImag, NumpyReal, NumpyTranspose,
NumpyConjugate, NumpySize)
NumpyConjugate, NumpySize, NumpyResultType)

__all__ = ('BooleanClass',
'IntegerClass',
'RealClass',
'ComplexClass',
'StringClass',
'NumpyArrayClass',
'TupleClass',
'literal_classes',
'get_cls_base')

Check warning on line 24 in pyccel/ast/class_defs.py

View check run for this annotation

Pyccel Bot / Pyccel best practices (pyccel_lint, 3.8)

pyccel/ast/class_defs.py#L16-L24

Sort the __all__ attribute of `class_defs`

#=======================================================================================

Expand Down Expand Up @@ -158,7 +158,9 @@
PyccelFunctionDef('conj', func_class = NumpyConjugate,
decorators = {'numpy_wrapper': 'numpy_wrapper'}),
PyccelFunctionDef('conjugate', func_class = NumpyConjugate,
decorators = {'numpy_wrapper': 'numpy_wrapper'})
decorators = {'numpy_wrapper': 'numpy_wrapper'}),
PyccelFunctionDef('dtype', func_class = NumpyResultType,
decorators = {'property': 'property', 'numpy_wrapper': 'numpy_wrapper'}),
]
)

Expand Down
6 changes: 2 additions & 4 deletions pyccel/ast/cwrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,27 +29,27 @@

errors = Errors()

__all__ = (
#
# --------- CLASSES -----------
#
'PyccelPyObject',
'PyArgKeywords',
'PyArg_ParseTupleNode',
'PyBuildValueNode',
'PyModule_AddObject',
#--------- CONSTANTS ----------
'Py_True',
'Py_False',
'Py_None',
'flags_registry',
#----- C / PYTHON FUNCTIONS ---
'Py_DECREF',
'set_python_error_message',
#----- CHECK FUNCTIONS ---
'generate_datatype_error',
'scalar_object_check',
)

Check warning on line 52 in pyccel/ast/cwrapper.py

View check run for this annotation

Pyccel Bot / Pyccel best practices (pyccel_lint, 3.8)

pyccel/ast/cwrapper.py#L32-L52

Sort the __all__ attribute of `cwrapper`

#-------------------------------------------------------------------
# Python DataTypes
Expand Down Expand Up @@ -108,7 +108,7 @@
(NativeFloat(), 4) : 'f',
(NativeComplex(), 4) : 'O',
(NativeComplex(), 8) : 'O',
(NativeBool(), 4) : 'p',
(NativeBool(), -1) : 'p',
(NativeString(), 0) : 's',
(PyccelPyObject(), 0) : 'O',
}
Expand Down Expand Up @@ -366,7 +366,7 @@

# Functions definitions are defined in pyccel/stdlib/cwrapper/cwrapper.c
py_to_c_registry = {
(NativeBool(), 4) : 'PyBool_to_Bool',
(NativeBool(), -1) : 'PyBool_to_Bool',
(NativeInteger(), 1) : 'PyInt8_to_Int8',
(NativeInteger(), 2) : 'PyInt16_to_Int16',
(NativeInteger(), 4) : 'PyInt32_to_Int32',
Expand Down Expand Up @@ -412,7 +412,6 @@
# Functions definitions are defined in pyccel/stdlib/cwrapper/cwrapper.c
c_to_py_registry = {
(NativeBool(), -1) : 'Bool_to_PyBool',
(NativeBool(), 4) : 'Bool_to_PyBool',
(NativeInteger(), -1) : 'Int'+str(default_precision['int']*8)+'_to_PyLong',
(NativeInteger(), 1) : 'Int8_to_NumpyLong',
(NativeInteger(), 2) : 'Int16_to_NumpyLong',
Expand Down Expand Up @@ -510,7 +509,6 @@
# Functions definitions are defined in pyccel/stdlib/cwrapper/cwrapper.c
check_type_registry = {
(NativeBool(), -1) : 'PyIs_Bool',
(NativeBool(), 4) : 'PyIs_Bool',
(NativeInteger(), -1) : 'PyIs_NativeInt',
(NativeInteger(), 1) : 'PyIs_Int8',
(NativeInteger(), 2) : 'PyIs_Int16',
Expand Down
15 changes: 12 additions & 3 deletions pyccel/ast/datatypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,50 +15,50 @@

# TODO [YG, 12.03.2020] verify why we need all these types
# NOTE: symbols not used in pyccel are commented out
__all__ = (
#
# --------- CLASSES -----------
#
'CustomDataType',
'DataType',
'NativeBool',
'NativeComplex',
'NativeGeneric',
'NativeInteger',
'NativeTuple',
'NativeNil',
'NativeRange',
'NativeFloat',
'NativeString',
'NativeSymbol',
'NativeVoid',
'UnionType',
'DataTypeFactory',
#
# --------- FUNCTIONS -----------
#
'datatype',
'is_iterable_datatype',
'is_pyccel_datatype',
'is_with_construct_datatype',
'str_dtype',
#
# --------- VARIABLES -----------
#
'Bool',
'Cmplx',
'Generic',
'Int',
'Nil',
'Float',
'String',
'Void',
'NativeNumeric',
# '_Symbol',
'default_precision',
'dtype_and_precision_registry',
'dtype_registry'
)

Check warning on line 61 in pyccel/ast/datatypes.py

View check run for this annotation

Pyccel Bot / Pyccel best practices (pyccel_lint, 3.8)

pyccel/ast/datatypes.py#L18-L61

Sort the __all__ attribute of `datatypes`

#==============================================================================
iso_c_binding = {
Expand All @@ -77,7 +77,7 @@
8 : 'C_DOUBLE_COMPLEX',
16 : 'C_LONG_DOUBLE_COMPLEX'},
"logical" : {
4 : "C_BOOL"}
-1 : "C_BOOL"}
}
iso_c_binding_shortcut_mapping = {
'C_INT8_T' : 'i8',
Expand All @@ -91,31 +91,40 @@
'C_FLOAT_COMPLEX' : 'c32',
'C_DOUBLE_COMPLEX' : 'c64',
'C_LONG_DOUBLE_COMPLEX' : 'c128',
'C_BOOL' : 'b4'
'C_BOOL' : 'b1'
}
default_precision = {'float': 8,
'int': numpy.dtype(int).alignment,
'integer': numpy.dtype(int).alignment,
'complex': 8,
'bool':4}
'bool':-1}
dtype_and_precision_registry = { 'float':('float', -1),
'double':('float', -1),
'real':('float', -1),
'pythonfloat':('float', -1), # built-in float
'float32':('float',4),
'float64':('float',8),
'f4':('float',4),
'f8':('float',8),
'pythoncomplex':('complex', -1),
'complex':('complex', -1), # to create numpy array with dtype='complex'
'complex64':('complex',4),
'complex128':('complex',8),
'c8':('complex',4),
'c16':('complex',8),
'int8' :('int',1),
'int16':('int',2),
'int32':('int',4),
'int64':('int',8),
'i1' :('int',1),
'i2':('int',2),
'i4':('int',4),
'i8':('int',8),
'int' :('int', -1),
'pythonint' :('int', -1),
'integer':('int',-1),
'bool' :('bool',-1),
'b1' :('bool',-1),
'pythonbool' :('bool',-1)}


Expand Down
2 changes: 1 addition & 1 deletion pyccel/ast/numpy_wrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,20 +28,20 @@

errors = Errors()

__all__ = (
#------- CAST FUNCTIONS ------
'pyarray_to_ndarray',
#-------CHECK FUNCTIONS ------
'array_type_check',
'scalar_type_check',
#-------HELPERS ------
'array_get_dim',
'array_get_data',
'array_get_c_step',
'array_get_f_step',
#-------OTHERS--------
'get_numpy_max_acceptable_version_file'
)

Check warning on line 44 in pyccel/ast/numpy_wrapper.py

View check run for this annotation

Pyccel Bot / Pyccel best practices (pyccel_lint, 3.8)

pyccel/ast/numpy_wrapper.py#L31-L44

Sort the __all__ attribute of `numpy_wrapper`

#-------------------------------------------------------------------
# Numpy functions
Expand Down Expand Up @@ -181,7 +181,7 @@
4 : np.dtype(np.int32).num,
8 : np.dtype(np.int64).num}

numpy_dtype_registry = {('bool',4) : numpy_bool_type,
numpy_dtype_registry = {('bool',-1) : numpy_bool_type,
('int',1) : numpy_num_to_type[numpy_int_type_precision_map[1]],
('int',2) : numpy_num_to_type[numpy_int_type_precision_map[2]],
('int',4) : numpy_num_to_type[numpy_int_type_precision_map[4]],
Expand Down
Loading
Loading