You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
However, to make use of the efficiency of the neural network, it would be nice if I could feed a batch of spectra at a time. Would this capability be possible?
Thanks!
The text was updated successfully, but these errors were encountered:
Of course we can think of such a new feature, but let me clarify a bit and think out loud. :-)
What nn.normalize(wave_grid, spectrum) does underneath is splitting the spectrum into partially overlapping fragments (8192 samples by default). Then these fragments are collected in a batch and thrown into the network, so hardware acceleration is used here. Due to this approach, each sample in the spectrum has many corresponding pseudo-conitinuum predictions, from which the weighted mean (which is used as a final pseudo-continuum prediction) and the standard deviation (considered some proxy of uncertainty) is computed.
You can take a closer look on how these details are handled in the class ProcessSpectrum, but this is nothing more but just "sliding window":
Now to the point. What you are suggesting could be really beneficial if we want to apply the method to more than, let's say, 1000 spectra. For this purpose it should be relatively simple to modify this class that it would accept a list of spectra (wavelengths+fluxes, or maybe just filenames, etc.) as an input, apply this preprocessing procedure ("sliding window") to form batches of size fitted into your GPU and probably get decent speedup.
So this can be done and might be useful for such a case. It would take two steps:
Adding a proper GPU support (Here I mean preparing docker image for SUPPNet. I have it in mind for a long time but I didn't have time to work on that).
Adopting a ProcessSpectrum class to make proper batching and processing of arbitrary number of spectra.
Maybe it is right time for that developments. What are your thoughts?
Hi there,
Currently I am estimating the continuum for each spectrum separately using
cont, cont_err, seg, seg_err = nn.normalize(wave_grid, spectrum)
However, to make use of the efficiency of the neural network, it would be nice if I could feed a batch of spectra at a time. Would this capability be possible?
Thanks!
The text was updated successfully, but these errors were encountered: