Segmentation fault when doing a "for reduction" with openmp #1143
-
When pyccelizing a function that does a As an example take a file
which is pyccelized with the command When calling these functions in another python file with e.g.
the output reads
I am using the newest pyccel version 1.5.2 |
Beta Was this translation helpful? Give feedback.
Replies: 21 comments 1 reply
-
Please can you share the generated code (see the |
Beta Was this translation helpful? Give feedback.
-
The fortran code from the
|
Beta Was this translation helpful? Give feedback.
-
I don't see anything obviously wrong in the translation. But your example doesn't implement a reduction. Given that you ask for one could this be causing the problem? |
Beta Was this translation helpful? Give feedback.
-
Can you please show us where you call the functions, I assumed you are not using the provided example because |
Beta Was this translation helpful? Give feedback.
-
I don't think the file with If it is working for you then it may be a compiler issue. What compiler are you both using? |
Beta Was this translation helpful? Give feedback.
-
I would expect the implemented code to behave differently with different compilers as putting a reduction clause on a loop that doesn't reduce is non-standard behaviour |
Beta Was this translation helpful? Give feedback.
-
As far as
In this example there is not reduction, you are right, but the application in which I encountered this problem is much more complex (in which a reduction is necessary). Even if I change the code above into something where reduction is needed, e.g.
the segmentation fault persists. |
Beta Was this translation helpful? Give feedback.
-
Yes, the file from which I call the file with the I am running a Ubuntu VM on a MacOS host. How can I check which compiler is used? (I would guess its the gcc compiler installed on the Mac host). |
Beta Was this translation helpful? Give feedback.
-
It should be the default gcc compiler in your environment. |
Beta Was this translation helpful? Give feedback.
-
When I run
When I run it in my host Mac machine I get
In order to compile Psydac kernel with pyccel, I had to change the flags to
so take out the |
Beta Was this translation helpful? Give feedback.
-
Could this be related to this: https://stackoverflow.com/questions/45885660/fortran-openmp-with-multiple-simultaneous-reductions-results-in-seg-fault ? I wonder if it has more success if you specify the actual reduced variable:
But I'm not sure if pyccel handles this |
Beta Was this translation helpful? Give feedback.
-
The good news is, I can reproduce this on my computer (Ubuntu gcc9.4) so I can play around and see if I find anything |
Beta Was this translation helpful? Give feedback.
-
This doesn't work; when trying something like
the following error pops up when compiling:
Also, this would have been a highly unpractical solution :) |
Beta Was this translation helpful? Give feedback.
-
I think it is the memory problem described in the stack overflow link. def mult_reduct1(arr : 'float[:,:]', arr1 : 'float[:,:]'):
"""
Parameters:
2 arrays of the same size
"""
nx = arr.shape[1]
ny = arr.shape[0]
#$ omp parallel private (i, j)
#$ omp for reduction(+: arr1)
for i in range(nx):
j = i * i
arr1[0,0] += arr[i,0]**2
arr1[1,1] += arr[i,1]**2
#$ omp end parallel
def mult_reduct2(arr : 'float[:,:]', arr1 : 'float[:,:]', arr2 : 'float[:,:]'):
"""
Parameters:
3 arrays of the same size
"""
nx = arr.shape[1]
ny = arr.shape[0]
#$ omp parallel private (i, j)
#$ omp for reduction(+: arr1, arr2)
for i in range(nx):
j = i * i
arr1[0,0] += arr[i,0]**2
arr1[1,1] += arr[i,1]**2
arr2[0,0] += arr[i,0]**2
arr2[1,1] += arr[i,1]**2
#$ omp end parallel
if __name__ == '__main__':
import numpy as np
nx = 1000
ny = 1000
array = np.empty((nx,ny))
for i in range(nx):
for j in range(ny):
array[i,j] = np.random.rand()
array1 = np.zeros_like(array)
array2 = np.zeros_like(array)
mult_reduct1(array, array1)
print('\n results \n')
print(np.sum(array1))
mult_reduct2(array, array1, array2)
print('\n results \n')
print(np.sum(array1))
print(np.sum(array2)) and run the executable I get:
but if I change: nx = 1000
ny = 1000 to nx = 100
ny = 100 then I get:
|
Beta Was this translation helpful? Give feedback.
-
I'm also on a VM, so @rmahjoubi presumably just has more memory available than we do |
Beta Was this translation helpful? Give feedback.
-
My VM has 4GB of RAM, so if I set
then each process (I run with 4 openmp processes) has 1GB of memory. Does a 1000x1000 array already exceed that? When I run a python via command line
the size of a 1000x1000 array is just 8MB; so even if two arrays are stored, it comes nowhere near 1GB. |
Beta Was this translation helpful? Give feedback.
-
Python can be quite greedy. That's why I ran with a program rather than via python. Can you confirm the behaviour? (i.e. does it start working for you if you reduce the size?) It is possible that python isn't grabbing all your environment variables. Does setting |
Beta Was this translation helpful? Give feedback.
-
I can confirm that when I reduce to a 100x100 array, the program runs without any issues. It seems to me that python is recognizing the environment variable; when I set I'm just amazed by the fact that 4GB of memory might not be enough for handling these arrays ... |
Beta Was this translation helpful? Give feedback.
-
I am very confused by this. Even for |
Beta Was this translation helpful? Give feedback.
-
I'm not sure if massif handles openmp properly though |
Beta Was this translation helpful? Give feedback.
-
So I tried the |
Beta Was this translation helpful? Give feedback.
I think it is the memory problem described in the stack overflow link.
If I translate: