Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add image_fft and image_rfft #1953

Merged
merged 69 commits into from Jun 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
65ca093
:sparkles: Add fft and rfft filters
tkoyama010 Dec 9, 2021
d3da172
:sparkles: Add image_fft and image_rfft filters
tkoyama010 Dec 18, 2021
3963030
Merge branch 'main' of github.com:pyvista/pyvista into feat/image-fft
tkoyama010 Dec 19, 2021
4541538
Fix isort error
tkoyama010 Dec 19, 2021
e53ebde
Merge branch 'main' into feat/image-fft
tkoyama010 Jan 6, 2022
bf4b10f
Merge branch 'main' into feat/image-fft
tkoyama010 Feb 9, 2022
65e7e97
Merge branch 'main' into feat/image-fft
tkoyama010 Feb 25, 2022
3ea7491
Update pyvista/core/filters/uniform_grid.py
tkoyama010 Feb 25, 2022
f5d4a94
Merge branch 'main' into feat/image-fft
tkoyama010 Mar 19, 2022
1f45f35
Merge branch 'main' into feat/image-fft
tkoyama010 Mar 29, 2022
8b831f8
Merge branch 'main' into feat/image-fft
tkoyama010 Mar 31, 2022
ed7a8ec
Fix isort
tkoyama010 Mar 31, 2022
e3b3f6c
Merge branch 'main' into feat/image-fft
tkoyama010 Apr 6, 2022
f37e6c2
Merge branch 'main' into feat/image-fft
akaszynski Jun 2, 2022
baf04d3
improve examples
akaszynski Jun 3, 2022
069bad2
fix docs
akaszynski Jun 3, 2022
c45d00e
Merge branch 'main' into feat/image-fft
akaszynski Jun 3, 2022
b2c71b2
add in text for plots
akaszynski Jun 3, 2022
937b929
add license info
akaszynski Jun 3, 2022
6225e57
Apply suggestions from code review
akaszynski Jun 4, 2022
8eda23e
make isort happy
akaszynski Jun 4, 2022
703a664
Apply obvious changes from code review
adeak Jun 4, 2022
a69ea28
Apply suggestions from code review
akaszynski Jun 5, 2022
e35997e
Minor fixes in FFT examples
adeak Jun 5, 2022
2a04d0c
Fix Butterworth order formulae and indentation
adeak Jun 5, 2022
09593b0
lots of excitement
akaszynski Jun 5, 2022
34c2b02
fix docstring
akaszynski Jun 5, 2022
cb4ee61
cleanup complex
akaszynski Jun 6, 2022
4348154
Merge branch 'main' into feat/image-fft
akaszynski Jun 6, 2022
6a8a1e3
add image cache file
akaszynski Jun 6, 2022
3c9a374
fixture: noise --> noise_2d
akaszynski Jun 6, 2022
4b7fbe1
cast to float like matplotlib
akaszynski Jun 6, 2022
bb85f9f
improve example
akaszynski Jun 6, 2022
2f6af78
copy set
akaszynski Jun 6, 2022
66567ae
add header titles
akaszynski Jun 6, 2022
e7e949d
remove complex and return float features
akaszynski Jun 12, 2022
5fa3522
Merge branch 'main' into feat/image-fft
akaszynski Jun 12, 2022
997b3ce
remove remaining complex/float changes
akaszynski Jun 12, 2022
23e6f38
Merge branch 'main' into feat/image-fft
akaszynski Jun 20, 2022
421c833
fix axes bounds fontsize
akaszynski Jun 22, 2022
4788559
allow active scalars
akaszynski Jun 22, 2022
2b26ccc
improve doc
akaszynski Jun 22, 2022
09fc04c
improve docs; fix font_size scaling
akaszynski Jun 22, 2022
ceec4ca
Apply suggestions from code review
adeak Jun 22, 2022
1f244ba
Merge branch 'main' into feat/image-fft
akaszynski Jun 23, 2022
b4c47c3
Merge branch 'main' into feat/image-fft
tkoyama010 Jun 24, 2022
d4107b6
Fix by black
tkoyama010 Jun 24, 2022
758ffd9
Fix by black
tkoyama010 Jun 24, 2022
2a5572b
fix formatting
akaszynski Jun 24, 2022
d5aabc8
Fix by pre-commit
tkoyama010 Jun 24, 2022
6ce17e5
fix various issues
akaszynski Jun 24, 2022
45b2075
Merge branch 'feat/image-fft' of github.com:/pyvista/pyvista into fea…
akaszynski Jun 24, 2022
5ab9ace
Apply suggestions from code review
akaszynski Jun 24, 2022
2304731
fix docstring
akaszynski Jun 24, 2022
a7f0d60
Merge branch 'feat/image-fft' of github.com:/pyvista/pyvista into fea…
akaszynski Jun 24, 2022
51d7e37
cleanup another docstring
akaszynski Jun 24, 2022
c590d09
Merge branch 'main' into feat/image-fft
tkoyama010 Jun 25, 2022
1eb7f81
Fix typos and intersphinx links in uniformgrid filters
adeak Jun 25, 2022
c1ae207
Add note about standard FFT frequency layout
adeak Jun 25, 2022
e81b3c6
Cross-link FFT methods in See Also sections
adeak Jun 25, 2022
5db6f6e
Specify data-related error subclasses in pytest.raises
adeak Jun 25, 2022
a473c05
Apply suggestions from code review
akaszynski Jun 26, 2022
a1a2e94
Add 'inactive scalars' RFFT test for coverage
adeak Jun 26, 2022
d8766ea
Add scaled version of Perlin noise FFT example animation
adeak Jun 27, 2022
160e4d5
Update untested scalars doctest output
adeak Jun 27, 2022
cff7bc6
Apply suggestions from code review
tkoyama010 Jun 27, 2022
8ca30df
Fix Perlin FFT animation scaling
adeak Jun 27, 2022
01a1029
Update pyvista/examples/downloads.py
adeak Jun 27, 2022
6f0e62f
Apply suggestions from code review
akaszynski Jun 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
206 changes: 206 additions & 0 deletions examples/01-filter/image-fft-perlin-noise.py
@@ -0,0 +1,206 @@
"""
.. _image_fft_perlin_example:

Fast Fourier Transform with Perlin Noise
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This example shows how to apply a Fast Fourier Transform (FFT) to a
:class:`pyvista.UniformGrid` using :func:`pyvista.UniformGridFilters.fft`
filter.

Here, we demonstrate FFT usage by first generating Perlin noise using
:func:`pyvista.sample_function() <pyvista.core.imaging.sample_function>` to
sample :func:`pyvista.perlin_noise <pyvista.core.common_data.perlin_noise>`,
and then performing FFT of the sampled noise to show the frequency content of
that noise.
"""

import numpy as np

import pyvista as pv

###############################################################################
# Generate Perlin Noise
# ~~~~~~~~~~~~~~~~~~~~~
# Start by generating some `Perlin Noise
# <https://en.wikipedia.org/wiki/Perlin_noise>`_ as in
# :ref:`perlin_noise_2d_example` example.
#
# Note that we are generating it in a flat plane and using a frequency of 10 in
# the x direction and 5 in the y direction. The unit of frequency is
# ``1/pixel``.
#
# Also note that the dimensions of the image are powers of 2. This is because
# the FFT is much more efficient for arrays sized as a power of 2.

freq = [10, 5, 0]
noise = pv.perlin_noise(1, freq, (0, 0, 0))
xdim, ydim = (2**9, 2**9)
sampled = pv.sample_function(noise, bounds=(0, 10, 0, 10, 0, 10), dim=(xdim, ydim, 1))

# warp and plot the sampled noise
warped_noise = sampled.warp_by_scalar()
warped_noise.plot(show_scalar_bar=False, text='Perlin Noise', lighting=False)


###############################################################################
# Perform FFT of Perlin Noise
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Next, perform an FFT of the noise and plot the frequency content.
# For the sake of simplicity, we will only plot the content in the first
# quadrant.
#
# Note the usage of :func:`numpy.fft.fftfreq` to get the frequencies.

sampled_fft = sampled.fft()
freq = np.fft.fftfreq(sampled.dimensions[0], sampled.spacing[0])
max_freq = freq.max()

# only show the first quadrant
subset = sampled_fft.extract_subset((0, xdim // 2, 0, ydim // 2, 0, 0))


###############################################################################
# Plot the Frequency Domain
# ~~~~~~~~~~~~~~~~~~~~~~~~~
# Now, plot the noise in the frequency domain. Note how there is more high
# frequency content in the x direction and this matches the frequencies given
# to :func:`pyvista.perlin_noise <pyvista.core.common_data.perlin_noise>`.

# scale to make the plot viewable
subset['scalars'] = np.abs(subset.active_scalars)
warped_subset = subset.warp_by_scalar(factor=0.0001)

pl = pv.Plotter(lighting='three lights')
pl.add_mesh(warped_subset, cmap='blues', show_scalar_bar=False)
pl.show_bounds(
axes_ranges=(0, max_freq, 0, max_freq, 0, warped_subset.bounds[-1]),
xlabel='X Frequency',
ylabel='Y Frequency',
zlabel='Amplitude',
show_zlabels=False,
color='k',
font_size=26,
)
pl.add_text('Frequency Domain of the Perlin Noise')
pl.show()


###############################################################################
# Low Pass Filter
# ~~~~~~~~~~~~~~~
# Let's perform a low pass filter on the frequency content and then convert it
# back into the space (pixel) domain by immediately applying a reverse FFT.
#
# When converting back, keep only the real content. The imaginary content has
# no physical meaning in the physical domain. PyVista will drop the imaginary
# content, but will warn you of it.
akaszynski marked this conversation as resolved.
Show resolved Hide resolved
#
# As expected, we only see low frequency noise.

low_pass = sampled_fft.low_pass(1.0, 1.0, 1.0).rfft()
low_pass['scalars'] = np.real(low_pass.active_scalars)
warped_low_pass = low_pass.warp_by_scalar()
warped_low_pass.plot(show_scalar_bar=False, text='Low Pass of the Perlin Noise', lighting=False)


###############################################################################
# High Pass Filter
# ~~~~~~~~~~~~~~~~
# This time, let's perform a high pass filter on the frequency content and then
# convert it back into the space (pixel) domain by immediately applying a
# reverse FFT.
#
# When converting back, keep only the real content. The imaginary content has no
# physical meaning in the pixel domain.
#
# As expected, we only see the high frequency noise content as the low
# frequency noise has been attenuated.

high_pass = sampled_fft.high_pass(1.0, 1.0, 1.0).rfft()
high_pass['scalars'] = np.real(high_pass.active_scalars)
warped_high_pass = high_pass.warp_by_scalar()
warped_high_pass.plot(show_scalar_bar=False, text='High Pass of the Perlin Noise', lighting=False)


###############################################################################
# Sum Low and High Pass
# ~~~~~~~~~~~~~~~~~~~~~
# Show that the sum of the low and high passes equals the original noise.

grid = pv.UniformGrid(dims=sampled.dimensions, spacing=sampled.spacing)
grid['scalars'] = high_pass['scalars'] + low_pass['scalars']

print(
'Low and High Pass identical to the original:', np.allclose(grid['scalars'], sampled['scalars'])
)

pl = pv.Plotter(shape=(1, 2))
pl.add_mesh(sampled.warp_by_scalar(), show_scalar_bar=False, lighting=False)
pl.add_text('Original Dataset')
pl.subplot(0, 1)
pl.add_mesh(grid.warp_by_scalar(), show_scalar_bar=False, lighting=False)
pl.add_text('Sum of the Low and High Passes')
pl.show()


###############################################################################
# Animate
# ~~~~~~~
# Animate the variation of the cutoff frequency.


def warp_low_pass_noise(cfreq, scalar_ptp=sampled['scalars'].ptp()):
"""Process the sampled FFT and warp by scalars."""
output = sampled_fft.low_pass(cfreq, cfreq, cfreq).rfft()

# on the left: raw FFT magnitude
output['scalars'] = output.active_scalars.real
warped_raw = output.warp_by_scalar()

# on the right: scale to fixed warped height
output_scaled = output.translate((-11, 11, 0), inplace=False)
output_scaled['scalars_warp'] = output['scalars'] / output['scalars'].ptp() * scalar_ptp
warped_scaled = output_scaled.warp_by_scalar('scalars_warp')
warped_scaled.active_scalars_name = 'scalars'
# push center back to xy plane due to peaks near 0 frequency
warped_scaled.translate((0, 0, -warped_scaled.center[-1]), inplace=True)

return warped_raw + warped_scaled


# Initialize the plotter and plot off-screen to save the animation as a GIF.
plotter = pv.Plotter(notebook=False, off_screen=True)
plotter.open_gif("low_pass.gif", fps=8)
akaszynski marked this conversation as resolved.
Show resolved Hide resolved

# add the initial mesh
init_mesh = warp_low_pass_noise(1e-2)
plotter.add_mesh(init_mesh, show_scalar_bar=False, lighting=False, n_colors=128)
plotter.camera.zoom(1.3)

for freq in np.geomspace(1e-2, 10, 25):
plotter.clear()
mesh = warp_low_pass_noise(freq)
plotter.add_mesh(mesh, show_scalar_bar=False, lighting=False, n_colors=128)
plotter.add_text(f"Cutoff Frequency: {freq:.2f}", color="black")
plotter.write_frame()

# write the last frame a few times to "pause" the gif
for _ in range(10):
plotter.write_frame()

plotter.close()


###############################################################################
# The left mesh in the above animation warps based on the raw values of the FFT
# amplitude. This emphasizes how taking into account more and more frequencies
# as the animation progresses, we recover a gradually larger proportion of the
# full noise sample. This is why the mesh starts "flat" and grows larger as the
# frequency cutoff is increased.
#
# In contrast, the right mesh is always warped to the same visible height,
# irrespective of the cutoff frequency. This highlights how the typical
# wavelength (size of the features) of the Perlin noise decreases as the
# frequency cutoff is increased since wavelength and frequency are inversely
# proportional.
109 changes: 109 additions & 0 deletions examples/01-filter/image-fft.py
@@ -0,0 +1,109 @@
"""
.. _image_fft_example:

Fast Fourier Transform
~~~~~~~~~~~~~~~~~~~~~~

This example shows how to apply a Fast Fourier Transform (FFT) to a
:class:`pyvista.UniformGrid` using :func:`pyvista.UniformGridFilters.fft`
filter.

Here, we demonstrate FFT usage by denoising an image, effectively removing any
"high frequency" content by performing a `low pass filter
<https://en.wikipedia.org/wiki/Low-pass_filter>`_.

This example was inspired by `Image denoising by FFT
<https://scipy-lectures.org/intro/scipy/auto_examples/solutions/plot_fft_image_denoise.html>`_.

"""

import numpy as np

import pyvista as pv
from pyvista import examples

###############################################################################
# Load the example Moon landing image and plot it.

image = examples.download_moonlanding_image()
print(image.point_data)

# Create a theme that we can reuse when plotting the image
grey_theme = pv.themes.DocumentTheme()
grey_theme.cmap = 'gray'
grey_theme.show_scalar_bar = False
grey_theme.axes.show = False
image.plot(theme=grey_theme, cpos='xy', text='Unprocessed Moon Landing Image')


###############################################################################
# Apply FFT to the image
# ~~~~~~~~~~~~~~~~~~~~~~
# FFT will be applied to the active scalars, ``'PNGImage'``, the default
# scalars name when loading a PNG image.
#
# The output from the filter is a complex array stored by the same name unless
# specified using ``output_scalars_name``.

fft_image = image.fft()
fft_image.point_data


###############################################################################
# Plot the FFT of the image
# ~~~~~~~~~~~~~~~~~~~~~~~~~
# Plot the absolute value of the FFT of the image.
#
# Note that we are effectively viewing the "frequency" of the data in this
# image, where the four corners contain the low frequency content of the image,
# and the middle is the high frequency content of the image.
akaszynski marked this conversation as resolved.
Show resolved Hide resolved

fft_image.plot(
scalars=np.abs(fft_image.point_data['PNGImage']),
cpos="xy",
theme=grey_theme,
log_scale=True,
text='Moon Landing Image FFT',
)


###############################################################################
# Remove the noise from the ``fft_image``
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Effectively, we want to remove high frequency (noisy) data from our image.
# First, let's reshape by the size of the image.
tkoyama010 marked this conversation as resolved.
Show resolved Hide resolved
#
# Next, perform a low pass filter by removing the middle 80% of the content of
# the image. Note that the high frequency content is in the middle of the array.
#
# .. note::
# It is easier and more efficient to use the existing
# :func:`pyvista.UniformGridFilters.low_pass` filter. This section is here
# for demonstration purposes.

ratio_to_keep = 0.10

# modify the fft_image data
width, height, _ = fft_image.dimensions
data = fft_image['PNGImage'].reshape(height, width) # note: axes flipped
data[int(height * ratio_to_keep) : -int(height * ratio_to_keep)] = 0
data[:, int(width * ratio_to_keep) : -int(width * ratio_to_keep)] = 0

fft_image.plot(
scalars=np.abs(data),
cpos="xy",
theme=grey_theme,
log_scale=True,
text='Moon Landing Image FFT with Noise Removed',
)


###############################################################################
# Convert to the spatial domain using reverse FFT
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Finally, convert the image data back to the "spatial" domain and plot it.


rfft = fft_image.rfft()
rfft['PNGImage'] = np.real(rfft['PNGImage'])
rfft.plot(cpos="xy", theme=grey_theme, text='Processed Moon Landing Image')
5 changes: 3 additions & 2 deletions examples/01-filter/sampling_functions_2d.py
@@ -1,13 +1,14 @@
"""
.. _perlin_noise_2d_example:

Sample Function: Perlin Noise in 2D
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Here we use :func:`pyvista.core.imaging.sample_function` to sample
Perlin noise over a region to generate random terrain.

Perlin noise is atype of gradient noise often used by visual effects
artists to increase the appearance of realism in computer graphics.
Source:
https://en.wikipedia.org/wiki/Perlin_noise
Source: `Perlin Noise Wikipedia <https://en.wikipedia.org/wiki/Perlin_noise>`_

The development of Perlin Noise has allowed computer graphics artists
to better represent the complexity of natural phenomena in visual
Expand Down
6 changes: 6 additions & 0 deletions pyvista/_vtk.py
Expand Up @@ -399,6 +399,12 @@
except ImportError: # pragma: no cover
pass

from vtkmodules.vtkImagingFourier import (
vtkImageButterworthHighPass,
vtkImageButterworthLowPass,
vtkImageFFT,
vtkImageRFFT,
)
from vtkmodules.vtkRenderingCore import (
vtkActor,
vtkActor2D,
Expand Down
18 changes: 17 additions & 1 deletion pyvista/core/datasetattributes.py
Expand Up @@ -476,11 +476,15 @@ def active_t_coords_name(self) -> Optional[str]:

@active_t_coords_name.setter
def active_t_coords_name(self, name: str) -> None:
if name is None:
self.SetActiveTCoords(None)
return

self._raise_no_t_coords()
dtype = self[name].dtype
# only vtkDataArray subclasses can be set as active attributes
if np.issubdtype(dtype, np.number) or dtype == bool:
self.SetActiveScalars(name)
self.SetActiveTCoords(name)

def get_array(self, key: Union[str, int]) -> pyvista_ndarray:
"""Get an array in this object.
Expand Down Expand Up @@ -1150,6 +1154,10 @@ def active_scalars_name(self) -> Optional[str]:

@active_scalars_name.setter
def active_scalars_name(self, name: str) -> None:
# permit setting no active scalars
if name is None:
self.SetActiveScalars(None)
return
self._raise_field_data_no_scalars_vectors()
dtype = self[name].dtype
# only vtkDataArray subclasses can be set as active attributes
Expand Down Expand Up @@ -1177,6 +1185,10 @@ def active_vectors_name(self) -> Optional[str]:

@active_vectors_name.setter
def active_vectors_name(self, name: str) -> None:
# permit setting no active
if name is None:
self.SetActiveVectors(None)
return
self._raise_field_data_no_scalars_vectors()
if name not in self:
raise KeyError(f'DataSetAttribute does not contain "{name}"')
Expand Down Expand Up @@ -1313,6 +1325,10 @@ def active_normals_name(self) -> Optional[str]:

@active_normals_name.setter
def active_normals_name(self, name: str) -> None:
# permit setting no active
if name is None:
self.SetActiveNormals(None)
return
self._raise_no_normals()
self.SetActiveNormals(name)

Expand Down