Use batch rebin, transmission_Lyman, and calc_zchi2 operations.

Speed gains: without archetypes, redrock on 4 GPU / 4 CPU takes 14.8s
reported total run time, 7.3s of which is in the fine redshift scan.
Comparatively with 64 CPU and 0 GPU the base redrock runs in 40.0s
reported total run time with 6.7s spent in fine redshift scan.

Adding the base archetypes option (without per-camera or nearest
neighbor) raises CPU times to 63.8s overall and 28.1s in fine z scan so
about 60% increase overall and about 4x slower in fine z scan.

With the new code the "batch" CPU mode slightly improves this to 60.0s
and 24.2s.

With the new GPU code, it runs on 4 GPU / 4 CPU in 22.8s total and 14.3s
for fine z scan, a 50% overall increase but only 2x speed increase in
the fine z scan, a big improvement from CPU times.

Also updated transmission_Lyman to return None when given scalar z to
match behavior when given an array of redshifts, in the case where the
wavelength range is not affected by Lyman transmission.  There is no
need in this case to calculate an array of all ones and then
additionally multiply the rebinned data by it.

Have yet to update --per-camera and -n_nearest options so right now
there are placeholders so that it does not crash that simply loop over
the existing CPU-mode logic.  These will be updated shortly.

…ijeet_per_camera

branch and this broke a couple unit tests.

Restored.

…have concluded is unrelated to the modification in this PR.

Steps to reproduce:
srun -n 4 -c 2 --gpu-bind=map_gpu:3,2,1,0 rrdesi_mpi --gpu --max-gpuprocs 4 -n_nearest 4 --archetypes new-archetypes/ -i $CFS/desi/spectro/redux/fuji/tiles/cumulative/100/20210505/coadd-0-100-thru20210505.fits     -o $SCRATCH/abhijeet.fits
srun -n 64 -c 2 rrdesi_mpi -n_nearest 4 --archetypes new-archetypes/ -i $CFS/desi/spectro/redux/fuji/tiles/cumulative/100/20210505/coadd-0-100-thru20210505.fits     -o $SCRATCH/abhijeet2.fits
These two results will be np.allclose() but not equal which is as expected.  The CPU version will be equal to Abhijeet's original code, also as expected.

Now rerun with -n_nearest 5 and the CPU version is still equal to Abhijeet's original code, but there are a small ~10 number of zzchi2 values and zzcoeff values that are different more than np.allclose (as much as 3.0 difference) between GPU and CPU.  Going back to Abhijeet's original nearest_neighbor_model code, and running on the CPU for that method with GPU get_best_archetype and there is still the same small differences, despite the fact that the output is np.allclose when not doing -n_nearest (or even for doing -n_nearest 4).

It does look like the differences seem to be most? all? in QSO templates.  But the trans array shows no difference - as far as I've been able to tell, we are sending the same input tdata to calc_zchi2_one on the CPU and getting a slightly different chi2.  But only for -n_nearest == 5.

However since this is independent of the code changes in the current PR (I get the exact same differences in this version versus rolling back to the previous one), it makes sense to proceed with this PR.

----------------------

- Added legendre function to Target class so that legendre of a certain degree
can be calculated (and optionally copied to GPU) once without additional
overhead.

- Added default value of 15 for fitz()

- In fitz() use Target.legendre to calculate legendre.  Pass target object instead of spectra to get_best_archetype, which allows for simplification as spectra, gpuweights, gpuflux, etc are all members of Target class.  Store trans dict and pass that to get_best_archetype to eliminate need to re-calculate transmission for the same wavelength regime.

- In archetypes, added properties gpuwave and gpuflux to copy and cache data on the GPU.  These are then used in rebin_template_batch.

- In get_best_archetype, pass target instead of spectra, which allows for elimination of dedges and legendre as args.  Get spectra, gpuweights, gpuflux, gpuwflux, dedges, and legendre directly from the target object.  Copying trans and using Target.legendre reduce runtime by about 1s on 4 GPUs.

- Vectorized and GPUized nearest_neighbor_model.  Instead of just passing trans to it, get_best_archetype now passes the binned dict, which already has rebinned flux multiplied by trans.  Using the Target.legendre also saves time.  Since the size of the tdata arrays is small (nbasis of a few), it is faster to keep operations on CPU for calc_zchi2_batch similar to in fitz().

Timing notes: adding nearest_neighbor_model is a bigger hit to the GPU than CPU but since get_best_archetypes is so much faster on the GPU, the combined time is still a good speed-up.

it should be 15 as a default arg now but adding a line if nz is None, nz
= 15 solves issue.

…redrock into abhijeet_per_camera_gpu

Modified solve_matrices algorithm to accept PCA or BVLS as a method.
Re-orgianized logic for GPU to take advantage of existing
calc_zchi2_batch algorithm.
For some strange reason refactoring tdata to add rows for color data in
batch in 3d array on CPU takes way longer than a python loop over 2d
arrays so I left two methods for per_camera - one for GPU and one for
CPU - for now while this gets resolved.

This is not the cleanest but it is the fastest and we can revisit if we
change our mind on that.

cleanly without a speed loss in runtime.

Removed old methods Tbs_for_archetypes,
per_camera_coeff_with_least_square, and
per_camera_coeff_with_least_square_cpu and renamed the remaining method
to per_camera_coeff_with_least_square_batch.

Updated 1->n_nbh where Abhijeet pointed out errors had been made.

Merging gpu version "abhijeet_per_camera_gpu" TO abhijeet_per_camera

Rolled back changes by Abhijeet done while attempting to merge to
restore per camera batch method.

…et_per_camera_prior_gpu. Added prior as optional arg to calc_zchi2_batch to do this.

Added always_return_array arg to transmission_Lyman that is True by default.
Changed calls to transmission_Lyman throughout redrock in archetypes, fitz,
templates to call with always_return_array=False to optimize because no need
to generate additional arrays of all 1 and multiply by them.
If always_return_array is True, an array of all ones will be returned in this
case instead of None.