Regarding Deep kernel learning.

[utkarshp1161]

Does it make sense to train on entire spectrum instead of using a single value(scalarizer)?

If yes, how do I extend the code when target is a n-dimensional(vector)?

Side thought: How about having a Neural-Network [weights optimized by dkl training]which converts the high dimensional spectrum to low dimension?

[ziatdinovmax]

You can. I experimented with predicting a full spectra with DKL 3 years ago, although that was done in AtomAI. It provided pretty decent results. The problem is that for active learning, you will still need to reduce it to a scalar value. So it's not clear what's the use of the predicted spectra.

[ziatdinovmax]

@utkarshp1161 to expand on it a bit, you will need N Gaussian processes running in parallel where N is a number of energy bands in your spectra. They can be connected to a single latent space or we can create N latent spaces using N neural nets. You can also treat each GP as a separate "task" and add a multi-task kernel to find potential correlations.

[ziatdinovmax]

AtomAI implementation based on gpytorch: https://github.com/pycroscopy/atomai/blob/8db3e944cd9ece68c33c8e3fcca3ef3ce9a111ea/atomai/trainers/gptrainer.py#L181

[utkarshp1161]

Thanks a lot Max. I will go through them and try to make this work. Will get back if any questions.

[utkarshp1161]

dklgp = aoi.models.dklGPR(data_dim, embedim=2, precision="single", shared_embedding_space = False)
dklgp.compile_multi_model_trainer(X_train, y_train, training_cycles=num_cycles )
mean, var = dklgp.predict(X, batch_size=len(X))

Information on data:
X_train.shape ~ (16, 25)
y_train.shape ~ (16, 439)

I am getting error when : mean, var = dklgp.predict(X, batch_size=len(X))

Full error:

mean, var = dklgp.predict(X, batch_size=len(X))

File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/atomai/models/dklgp/dklgpr.py", line 213, in predict
mean, var = self._predict(x)
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/atomai/models/dklgp/dklgpr.py", line 195, in _predict
posterior = self.compute_posterior(x_new)
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/atomai/models/dklgp/dklgpr.py", line 155, in compute_posterior
posterior = self.gp_model(*X.to(self.device).split(1))
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/model_list.py", line 79, in call
return [
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/model_list.py", line 80, in
model.call(*args, **kwargs) for model, args in length_safe_zip(self.models, _get_tensor_args(*args))
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/exact_gp.py", line 294, in call
self.prediction_strategy = prediction_strategy(
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/exact_prediction_strategies.py", line 37, in prediction_strategy
return cls(train_inputs, train_prior_dist, train_labels, likelihood)
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/kernels/grid_interpolation_kernel.py", line 213, in prediction_strategy
return InterpolatedPredictionStrategy(train_inputs, train_prior_dist, train_labels, likelihood)
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/exact_prediction_strategies.py", line 375, in init
super().init(train_inputs, train_prior_dist, train_labels, likelihood)
File "/nfs/home/upratius/.conda/envs/gpax_hae/lib/python3.10/site-packages/gpytorch/models/exact_prediction_strategies.py", line 51, in init
raise RuntimeError(
RuntimeError: Flattening the training labels failed. The most common cause of this error is that the shapes of the prior mean and the training labels are mismatched. The shape of the train targets is torch.Size([1, 439]), while the reported shape of the mean is torch.Size([1, 16]).

[utkarshp1161]

I think there is shape issue with prior distribution on mean.