posts/2023/aiml_essentials/part3/aiml-essentials-part3/

[giscus[bot]]

posts/2023/aiml_essentials/part3/aiml-essentials-part3/

https://lemonfold.io/posts/2023/aiml_essentials/part3/aiml-essentials-part3/

[Xemin0]

The code snippet for plotting at the end of the 'Implementation' section seems missing ?

[Xemin0]

Actually I will post my code when I worked through this tutorial

import matplotlib.pyplot as plt

'''
Plotting Util
'''
def plot_gpr(data_train, data_test, fn, gpr_results):
    X_train, Y_train = data_train
    X_test, _ = data_test
    mean, var, upper, lower = gpr_results

    plt.figure(figsize = (10, 6))
    # Ground truth
    plt.plot(X_test, fn(X_test), 'r', label = 'Ground Truth')

    # Training data
    plt.scatter(X_train, Y_train, c = 'r', s = 50, zorder = 10, edgecolors = (0,0,0))

    # Mean Prediction
    plt.plot(X_test, mean, 'k', lw = 1, zorder = 9, label = 'Mean Prediction')   

    # Shade between the upper and lower bounds of the variance
    plt.fill_between(X_test, lower, upper, alpha = 0.2, color = 'b')

    plt.tight_layout()
    plt.legend()
    plt.show()  

Then the main driver part of the regression model I tried is as follows

# range of points to be sampled from 
lb, ub = [-8, 8]
num_train = 4 ### Can't be larger than 5 because np.linalg.inv will yield redicuolous result
num_test = 100 

# Generate data
np.random.seed(42)
# Train
X_train = np.sort(np.random.uniform(lb, ub, num_train)) # num_train points between lb and ub

Y_train = np.sin(X_train) + np.random.normal(0, 0.1, len(X_train)) #  noise

# Test
X_test = np.linspace(lb, ub, num_test)


# Train the model
gpr_results = gpr(choice_of_kernel, [X_train, Y_train], [X_test, 0]) 


# Plot 
plot_gpr([X_train, Y_train], [X_test, 0], np.sin, gpr_results)

[Xemin0]

I realized in this example the number of training points can't be larger than 5 (at least on my machine), as it will yield ridiculously large mean predictions. I think it's because np.linalg.inv is struggling when calculating the inverse of cov_X. (where capital X is the training points)

One way to stabilize the covariance matrix is to add a small number (as noise) to the diagonal entries of the cov_X