Spectrum analysis of datasets. #193
Conversation
There was a problem hiding this comment.
Thank you @Robertboy18, great to have! Added a few comments.
examples/plot_darcy_flow_spectrum.py
Outdated
| dataset_name = "Darcy Flow" | ||
|
|
||
| # %% | ||
| HOME_PATH = '/home/user/' |
There was a problem hiding this comment.
@Robertboy18 this will not work for users - better to just use the builtin dataset for this, as in https://github.com/neuraloperator/neuraloperator/blob/main/examples/plot_FNO_darcy.py
There was a problem hiding this comment.
That makes sense, will load the dataset directly then!
There was a problem hiding this comment.
Actually, the issue is that we need to load the dataset in its raw format (so .pt, .mat, .zarr etc) to get the full spectrum; we cannot use the data loaders! I can leave a comment asking the user to change the HOME_PATH as 'neuraloperator/neuralop/datasets/data/darcy_test_16.pt' this path is the direct link to the raw format dataset.
neuralop/utils.py
Outdated
| """ | ||
| T = u.shape[0] | ||
| u = u.reshape(T, s, s) | ||
| # u = torch.rfft(u, 2, normalized=False, onesided=False) - depending on your choice of normalization and such |
There was a problem hiding this comment.
Can we make these options in the function rather than a comment here?
|
|
||
| # 2d wavenumbers following Pytorch fft convention | ||
| k_max = s // 2 | ||
| wavenumers = torch.cat((torch.arange(start=0, end=k_max, step=1), \ |
There was a problem hiding this comment.
Can we do everything in pytorch instead of going back and forth pytorch <-> numpy?
There was a problem hiding this comment.
Sure, that is a good suggestion.
|
Thank you @Robertboy18, looks great, merging! |
Adds Spectrum analysis function and example.
I have pushed a script for analyzing the spectrum of a given dataset. This script performs a comprehensive analysis of the frequency components present in the data, providing valuable insights into understanding the difficulty of the dataset by analyzing its frequency spectrum.
For example, the script below calculates the spectrum of the Darcy flow dataset. Comments have been added for a more general setup as well.
Thank you!