BNSE initialization is fragile

[jjramsey]

Here's a test code that I used to find one problem:

#!/usr/bin/env python3

import numpy as np
import mogptk

Q = 2
num_channels = 3
seed = 3428943
num_inp_comps = 3

min_channel_size = 10
max_channel_size = 70

print(f"""Q = {Q}
num_channels = {num_channels}
seed = {seed}
num_inp_comps = {num_inp_comps}

min_channel_size = {min_channel_size}
max_channel_size = {max_channel_size}
""")

# Using old-style NumPy RNG usage, since mogptk uses it under the
# hood.
np.random.seed(seed)

dataset = mogptk.DataSet()

for channel_num in range(num_channels):
    channel_size = np.random.randint(min_channel_size,
                                     max_channel_size + 1)

    x = np.pi*np.random.random_sample((channel_size, num_inp_comps))
    
    y = (np.sin(x).prod(axis = 1) +
         np.random.standard_normal(channel_size))
    
    curr_data = mogptk.Data(x, y, name = str(channel_num))

    dataset.append(curr_data)

model = mogptk.MOSM(dataset, Q = Q)
model.init_parameters(method = "BNSE")

The results of running this are as follows:

Traceback (most recent call last):
  File "./test-bnse-w-multi-inp-dataset.py", line 43, in <module>
    model.init_parameters(method = "BNSE")
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/mosm.py", line 107, in init_parameters
    self.set_parameter(q, 'magnitude', magnitude[:, q])
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/model.py", line 308, in set_parameter
    kern[key].assign(val)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/gpflow/base.py", line 228, in assign
    unconstrained_value = self.validate_unconstrained_value(value, self.dtype)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/gpflow/base.py", line 199, in validate_unconstrained_value
    return tf.debugging.assert_all_finite(unconstrained_value, message=message)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/tensorflow_core/python/ops/numerics.py", line 67, in verify_tensor_all_finite_v2
    verify_input = array_ops.check_numerics(x, message=message)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py", line 902, in check_numerics
    _ops.raise_from_not_ok_status(e, name)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 6606, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: gpflow.Parameter: the value to be assigned is incompatible with this parameter's transform (the corresponding unconstrained value has NaN or Inf) and hence cannot be assigned. : Tensor had NaN values [Op:CheckNumerics]

Unfortunately, reducing num_inp_comps to 1 doesn't fix this problem.

I've also had different errors with another dataset, which look like this:

Traceback (most recent call last):
  File "./mk_gp_mogptk.py", line 98, in <module>
    model.init_parameters(method = args.init)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/mosm.py", line 88, in init_parameters
    amplitudes, means, variances = self.dataset.get_bnse_estimation(self.Q)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/dataset.py", line 419, in get_bnse_estimation
    channel_amplitudes, channel_means, channel_variances = channel.get_bnse_estimation(Q, n)
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/data.py", line 1052, in get_bnse_estimation
    amplitudes, positions, variances = bnse.get_freq_peaks()
  File "/Users/jjramsey/venvs/gpflow2/lib/python3.7/site-packages/mogptk/bnse.py", line 111, in get_freq_peaks
    dx = x[1]-x[0]
IndexError: index 1 is out of bounds for axis 0 with size 1

Unfortunately, I'm not at liberty to release that other dataset, and I haven't been able to find a test dataset that reproduces the above error.

[tdewolff]

Thanks for the bug report. Could you check what the scale is of your x-axis? In particular, what is the range of the x values, and what is the (smallest) difference between the x values? We should be hardening the BNSE code to make sure it works for all types of scales.

You should try to rescale your x-axis so that it is roughly in the 0 -- 1000 range. We're working on a solution for x-axis scaling in the develop branch.

[jjramsey]

My inputs, both in my sample code and in the private dataset that I mentioned, are already well within the 0--1000 range.

[Nickel1997]

I also encountered this problem. Is there a better solution? In my case, this situation often occurs when the dimension of each channel is greater than 2 and the number of samples for each channel is not large (about 200).

[tdewolff]

Thanks for the feedback. We're in the process of moving to PyTorch and improving the library overall. Part of that will be rewriting the BNSE initialization and testing its stability. In particular, we'll explicitly try and test more than 2 input dimensions and prevent it from returning NaN, Inf, or have an out of bounds error. We're well on our way for the next version, but it will still take a few weeks to iron out all the details.

[tdewolff]

BNSE has been rewritten in PyTorch in 0c4f80b, this might fix this issue.

[tdewolff]

I'm going to close this issue, since the first error does not get thrown running the code you provided, and the second error involving x[1]-x[0] can never happen in the new code. If you run into problems again, please re-open!