openpiv-python refactoring

[alexlib]

This is a big milestone with a lot of refactoring:

• there is a single correlation function that is vectorized, based on the windef.py
• the correlation function is extended to work with the extended_search_area
• windef is now only wraps around the extended_search_area_piv function
• all tests pass but probably all notebooks fail.

[ErichZimmer]

Dear Prof. Liberzon,

Maybe, you should add a window deformation function to pyprocess.py. Something like;

def deform_windows(frame, x, y, u, v, interpolation_order = 1, kx = 3, ky = 3):
    # this part needs some rewording since I am not good at this type of stuff yet
    """Deform an image by window deformation where a new grid is defined based
    on the grid and displacements of the previous pass and pixel values are interpolated 
    onto the new grid.

    Parameters
    ----------
    frame : 2d np.ndarray, dtype=np.int32
        an two dimensions array of integers containing grey levels of 
        the first frame.

    x : 2d np.ndarray
        a two dimensional array containing the x coordinates of the
        interrogation window centers, in pixels.

    y : 2d np.ndarray
        a two dimensional array containing the y coordinates of the
        interrogation window centers, in pixels.

    u : 2d np.ndarray
        a two dimensional array containing the u velocity component,
        in pixels/seconds.

    v : 2d np.ndarray
        a two dimensional array containing the v velocity component,
        in pixels/seconds.

    interpolation_order: scalar
        the degree of the frame interpolation (deformation) of the mesh

    kx : scalar
         the degree of the interpolation of the B-splines over the x-axis of a rectangular mesh 

    ky : scalar
         the degree of the interpolation of the B-splines over the y-axis of a rectangular mesh

    Returns
    -------
    frame_def: 
        a deformed image based on the meshgrid and displacements of the previous pass
    """
    frame = frame.astype(np.float32)
    y1 = y[:, 0] # extract first coloumn from meshgrid
    y1 = y1[::-1] #flip 
    x1 = x[0, :] #extract first row from meshgrid
    side_x = np.arange(0, np.size(frame[0, :]), 1) #extract the image grid
    side_y = np.arange(0, np.size(frame[:, 0]), 1)

    # interpolating displacements onto a new meshgrid
    ip = RectBivariateSpline(y1, x1, u, kx = kx, ky = ky)
    ut = ip(side_y, side_x)# the way how to use the interpolation functions differs
                            #from matlab 
    ip2 = RectBivariateSpline(y1, x1, v, kx = kx, ky = ky)
    vt = ip2(side_y, side_x)

    #deform the image by using the map coordinates function (old grid + displacement)
    x, y = np.meshgrid(side_x, side_y)#create a meshgrid 
    frame_def = scn.map_coordinates(
        frame, ((y-vt, x+ut,)), order=interpolation_order,mode='nearest')

    return frame_def

[alexlib]

Great idea. Please think of a test function that checks the essential parts of this function.

[ErichZimmer]

Dear Prof. Liberzon,

I am not sure how to proceed with making a test function. Furthermore, I am still puzzled on how the functions in OpenPIV test folder work. Maybe, @TKaeufer can help? The function presented above is nearly identical to the deformation function in windef.py except it allows the user to control the degree of the B-Spline interpolation. Theoretically, this should allow the user to pick a faster, less precise deformation algorithm for large data sets. I haven't personally tested it with large data sets, but I also don't feel like waiting a few days again :)

Regards,
Erich Zimmer

[alexlib]

Dear Prof. Liberzon,

I am not sure how to proceed with making a test function. Furthermore, I am still puzzled on how the functions in OpenPIV test folder work. Maybe, @TKaeufer can help? The function presented above is nearly identical to the deformation function in windef.py except it allows the user to control the degree of the B-Spline interpolation. Theoretically, this should allow the user to pick a faster, less precise deformation algorithm for large data sets. I haven't personally tested it with large data sets, but I also don't feel like waiting a few days again :)

Regards,
Erich Zimmer

what you could do is just to write a simple function that takes a known input, uses deform_windows() and returns an output that you believe in. for instance it can generate an image and then the test will be every time we change something in the future versions is to compare it with the image you created. I can help with converting it into a proper test function, under /openpiv/tests

[TKaeufer]

Hi everbody,

I have a look at the new windef and test it on some images.
Concerning deform_windows function: As Erich said this is almost identical to the former frame_interpolation. Back then, when I wrote the function I decided to hide the interpolation order from the user to not further confuse the user. I think by default it used the first order spline interpolation.
Edit: I missed it at first glance. You made kx and ky adjustable, right?

I also think that we should either create a new .py file that contains the windef related function (first pass, multipass, deform_windows) or add those functions to the pyprocess.py to make the windef.py more clearly arranged. Since the windef.py now wraps around the extended search array PIV we might even consider renaming it with a more general name, e.g. PIV_interrogation_setup.py

Maybe we can also talk about this on Monday.

Best Regards,

Theo

[alexlib]

@AndreasBauerGit please test this branch, it might destroy few things. I'm here if needed.
@ErichZimmer and @eguvep please test your GUI with this version. I left first_pass and multi_pass untouched, but other things changed, e.g. get_field_shape and get_coordinates and save in tools.py

[TKaeufer]

Hey @alexlib,

I had a look on the the new windef wrap arround the extended search area PIV and it worked fine for me. I tested it with some syntheic images and a PIV challange case and the vector fields seem reasonable. I also added a new method for the window deformation that splits the deformation symetrically on both frames. I modified the deform windows function according to my needs and also created another one that just calculates the deformation field. I tested it on syntheic images and the result was fine. However, before commiting i would like to know if you use autopep and if so wich settings you use. Then i can formate the code like you do.

Best regards,

Theo

Best regards

Theo

[alexlib]

Hey @alexlib,

I had a look on the the new windef wrap arround the extended search area PIV and it worked fine for me. I tested it with some syntheic images and a PIV challange case and the vector fields seem reasonable. I also added a new method for the window deformation that splits the deformation symetrically on both frames. I modified the deform windows function according to my needs and also created another one that just calculates the deformation field. I tested it on syntheic images and the result was fine. However, before commiting i would like to know if you use autopep and if so wich settings you use. Then i can formate the code like you do.

Best regards,

Theo

Best regards

Theo

Yes, I use autopep8 and black and check myself with flake8 to manage the code organisation

[ErichZimmer]

Dear Alex,

I tested the GUI and only one line needed to be changed (import openpiv.process to import openpiv.pyprocess). Other than that, it worked very nicely. I did a quick percent error calculation (failed correlations and outliers) over some PIV data I had laying around and both the refactored and original performed within 0.0001% between each other. On my computer, both performed reasonably fast at four milliseconds per vector (nearly the same speed as PIVlab on the same computer).

Edit:
I got a strange error in the subpixel peak algorithm that I did not get on the refactored version. The error occured on the final pass.
Error:
IndexError: index 15 is out of bounds for axis 1 with size 15

Steps to reproduce:
The image data set used was PIV challenge 2001 case B. The windowing was set to 128>64>32>16>8 with a 50% overlap and no image preprocessing. The correlation algorithm used was the refactored windef script with circular correlation and large values set to the validation algorithms to "disable" them.

Operating system and specs:
Windows 10 Professional
Intel Pentium N3710
4 GB of RAM
Clocked at 1.6 GHz
64 bit operating system
5 GB free hard drive space (mainly caused by Matlab, Maple, and LOTS is PIV images)

PS; it is verry scary to use a mobile phone to comment on GitHub. The close with comment button occupies nearly of the space of the "keyboard". This created some harrowing moments and a frantic mom 😬, so sorry if something are left out, too vague, or confusing.

[ErichZimmer]

For clarification, the error occured on the refactored version and not the original. The error occured on line 282 in pyprocess.py.

[alexlib]

Hi everbody,

I have a look at the new windef and test it on some images.
Concerning deform_windows function: As Erich said this is almost identical to the former frame_interpolation. Back then, when I wrote the function I decided to hide the interpolation order from the user to not further confuse the user. I think by default it used the first order spline interpolation.
Edit: I missed it at first glance. You made kx and ky adjustable, right?

I also think that we should either create a new .py file that contains the windef related function (first pass, multipass, deform_windows) or add those functions to the pyprocess.py to make the windef.py more clearly arranged. Since the windef.py now wraps around the extended search array PIV we might even consider renaming it with a more general name, e.g. PIV_interrogation_setup.py

Maybe we can also talk about this on Monday.

Best Regards,

Theo

I'm not sure about the name - probably somebody has to suggest a single naming convention so we rename all functions in the same spirit - the name should say by itself what the function does

[alexlib]

For clarification, the error occured on the refactored version and not the original. The error occured on line 282 in pyprocess.py.

@ErichZimmer please test again with the new commit d542287

[alexlib]

Other than that, it worked very nicely. I did a quick percent error calculation (failed correlations and outliers) over some PIV data I had laying around and both the refactored and original performed within 0.0001% between each other. On my computer, both performed reasonably fast at four milliseconds per vector (nearly the same speed as PIVlab on the same computer).

@ErichZimmer - is it possible to have a screencast of your test? I'd like to make it a Jupyter notebook test where I can see differences of .0001% between every new version of the code. I don't know if it's possible to do it with GUI. Thanks in advance.

[alexlib]

Steps to reproduce:
The image data set used was PIV challenge 2001 case B. The windowing was set to 128>64>32>16>8 with a 50% overlap and no image preprocessing. The correlation algorithm used was the refactored windef script with circular correlation and large values set to the validation algorithms to "disable" them.

@ErichZimmer - I repeated this test and created a new Jupyter notebook that does these 5 step deformation analysis. However, I cannot find the raw data to compare the results to. Do you have a synthetic case with the raw data, so instead of running comparisons we'll run the comparison with the ground truth?

[TKaeufer]

I checked the windef.py combined with the example_windef_run.py and both worked for me.

[alexlib]

Hi everbody,

I have a look at the new windef and test it on some images.
Concerning deform_windows function: As Erich said this is almost identical to the former frame_interpolation. Back then, when I wrote the function I decided to hide the interpolation order from the user to not further confuse the user. I think by default it used the first order spline interpolation.
Edit: I missed it at first glance. You made kx and ky adjustable, right?

I also think that we should either create a new .py file that contains the windef related function (first pass, multipass, deform_windows) or add those functions to the pyprocess.py to make the windef.py more clearly arranged. Since the windef.py now wraps around the extended search array PIV we might even consider renaming it with a more general name, e.g. PIV_interrogation_setup.py

Maybe we can also talk about this on Monday.

Best Regards,

Theo

I see it's implemented. Thanks. a86a602