Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-38745: Resync with upstream #35

Merged
merged 110 commits into from
Aug 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
a5c479f
remove psf2d
Mar 7, 2023
862e6ff
add overwrite=True in writeto for temp fits file
Mar 29, 2023
c9bdd1c
add OBS_SECONDARY_MIRROR parameter to compute the effective telescope…
Mar 29, 2023
f422297
add value for the secondary mirror
Mar 30, 2023
bc90d03
add angle for better wavelength array in fit_spectrum.py
Mar 31, 2023
e48ba23
debug plot for order 2
Apr 10, 2023
40f5ab0
plot all colorbars in ffm plot_fit
Apr 10, 2023
0a5b129
update photutils and background sextractor method (and debug plot)
Apr 10, 2023
608080c
Merge branch 'stardice' of https://github.com/LSSTDESC/Spectractor in…
Apr 10, 2023
07ed347
Create FitParameters new class to store and manipulates fitted parame…
Apr 13, 2023
d4e480d
Create FitParameters new class to store and manipulates fitted parame…
Apr 13, 2023
bf88cab
Use new FitParameters class in fit workspaces
Apr 13, 2023
eba35f2
cleaning
Apr 13, 2023
d9ebb71
use new FitParameters class in FFM
Apr 13, 2023
50efc0e
update version number to 2.5
Apr 13, 2023
adfb786
less verbosity
Apr 13, 2023
3d2780a
repair tests
Apr 13, 2023
4f2975b
repair latex labels for order 2 parameters
Apr 13, 2023
d32cd9c
mask signal regions in extractor for complexe backgrounds
Apr 13, 2023
452ac5b
Use new FitParameters class
Apr 13, 2023
b233aea
Use new photutils method for background and source masking
Apr 13, 2023
2b98b2c
getcalspec 2.0.0
Apr 13, 2023
5a679bb
getcalspec 2.0.0
Apr 13, 2023
798364d
dilate detect source mask
Apr 13, 2023
ff9e7d0
remove rebuild calspec tables
Apr 13, 2023
812ea44
Merge branch 'master' into fitparams
jeremyneveu Apr 13, 2023
0408a10
clean sextractor remove background function
Apr 13, 2023
9ae2478
repair tests, upgrade scikit-image to 0.20 minimum for future depreca…
Apr 17, 2023
9d33fdc
upgrade scikit-image to 0.20 minimum for future deprecation of hessia…
Apr 17, 2023
2f23801
verbosity to repair github action tests
Apr 17, 2023
c3c1641
verbosity to repair github action tests
Apr 17, 2023
90c8d3d
verbosity to repair github action tests
Apr 17, 2023
b4c42d6
verbosity to repair github action tests
Apr 17, 2023
281250d
repair rotation functions with new scikit-image>=0.20
Apr 17, 2023
be8dede
write/read json files for fitparameters
Apr 17, 2023
645c13d
cleaning
Apr 17, 2023
8e75e10
use FitParameters.write_text()
Apr 17, 2023
3a0bfdb
remove deprecated and unused SpectrumSimGrid class and related functions
Apr 17, 2023
f9339e8
remove deprecated and unused SpectrumSimulator functions
Apr 17, 2023
6b2baab
simplify simulator functions
Apr 17, 2023
b6535d4
mandatory arugment for spectrumfitworkspace and spectrogramfitworkspa…
Apr 17, 2023
b3a492d
cleaning
Apr 17, 2023
29d0fbf
debug
Apr 17, 2023
e6a2af4
debug
Apr 17, 2023
42925f1
use write_json
Apr 17, 2023
f80db7a
debug
Apr 17, 2023
c63eb17
add test for order3 simulation
Apr 17, 2023
e6a24e8
add doctest for write_text
Apr 17, 2023
9fe815e
remove warning
Apr 17, 2023
04a5ce7
remove warning
Apr 17, 2023
f96071d
remove warning
Apr 17, 2023
009301f
remove warning
Apr 17, 2023
640b14a
remove warning
Apr 17, 2023
9e09313
remove warning
Apr 18, 2023
b55c3f1
debug to work at USDF
jeremyneveu Apr 18, 2023
754645f
refactor write_fitparameter_json function
Apr 20, 2023
554bcb2
Merge branch 'fitparams' of https://github.com/LSSTDESC/Spectractor i…
Apr 20, 2023
3549ae5
Merge pull request #127 from lsst/lsst-dev
jeremyneveu Apr 21, 2023
f988d84
merge with master
Apr 21, 2023
bf1f42b
re-enable test_atmosphere
Apr 21, 2023
0cbdae8
faster fit_spectrogram doctest
Apr 21, 2023
f931227
fitparameters.err of same size as value array (0 for fixed values)
Apr 22, 2023
cb864bb
move coverage script
May 2, 2023
8e9ab51
update fitworkspace classes
May 2, 2023
c5b3cd5
update coverage script
May 2, 2023
5325f7a
cleaning
May 2, 2023
f139a97
adapt for CCD_REBIN != 1
May 2, 2023
78adc65
cleaning
May 2, 2023
6f8ddab
SpectrogramModel working with CCD_REBIN > 1
May 2, 2023
d9705ea
un-rebinned testèctio_full_chain
May 2, 2023
d00446e
debug .err in fitparameters
May 3, 2023
03a5992
debug .err in fitparameters
May 3, 2023
e3159c9
debug fit_spectrogram with no crop is unnecessary
May 3, 2023
56edb30
photutils>=1.7.0; clean .travis.yml
May 4, 2023
1c6391e
version 2.5
May 4, 2023
5931e19
lower out of band values (1e-5 -> 1e-6) for proper resetting of lambd…
May 5, 2023
29225fe
add uncertainties on vendor curve
May 5, 2023
7ebe054
clean config load and fut_angstrom-exponent True
May 5, 2023
79e1b14
labels with units
May 5, 2023
c867824
labels with units
May 5, 2023
d9d6484
FitParameters class indexable
May 6, 2023
460ce77
FitParameters class indexable
May 6, 2023
2f269d7
harmonisation of attributes names between PSF, ChromaticPSF and FitPa…
May 6, 2023
4d12fe9
remove mpi4py from docs
May 6, 2023
f47e5d2
use FitParameters in PSF classes
May 6, 2023
fe99e3d
repair docs
May 6, 2023
1c0237d
repair image_simulation with new FitParameters
May 6, 2023
88e2930
repair image_simulation with new FitParameters
May 6, 2023
7aafd98
rename input_labels into labels
May 6, 2023
66218c0
repair chromaticpsf due to usage of FitParameters in PSF classes
May 6, 2023
ccbadea
repair docs
May 6, 2023
30fd0e8
reduce verbosity of gradient descent
May 6, 2023
78cb096
use FitParameters in ChromaticPSF class
May 9, 2023
9e81fcf
improve verbosity of fit
May 9, 2023
411110b
debug one doctest
May 9, 2023
dad02fd
repair doctests
May 9, 2023
df1425e
raise Implemented error if target is not a CALSPEC
May 12, 2023
9446f29
raise Implemented error if target is not a CALSPEC
May 12, 2023
0225766
raise ValueError if target is not a CALSPEC
May 12, 2023
f978760
gaia file name as argument in Astrometry()
May 16, 2023
c7bc6cd
Merge pull request #125 from LSSTDESC/fitparams
jeremyneveu May 16, 2023
a07cb1e
repair astrometry.net using .xyls extension
May 17, 2023
f827169
raise error if empty table
May 17, 2023
e321784
repair astrometry doctests
May 18, 2023
6332fa0
repair astrometry related doctests
May 18, 2023
950575a
round float arrays for doctests
May 18, 2023
d227d0f
Merge pull request #129 from LSSTDESC/stardice
jeremyneveu May 18, 2023
c456f79
Merge branch 'master' into tickets/DM-38745
mfisherlevine Aug 23, 2023
4f3f944
Fix mistake in merge conflict resolution
mfisherlevine Aug 24, 2023
e689fe0
Fix typo
mfisherlevine Aug 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
# Might be quicker to install rubin-env plus any necessary additions.
shell: bash -l {0}
run: |
mamba install -y "numpy>1.15" scipy "matplotlib>3.1" pandas llvmlite numba "astropy>=3.2" "photutils>=0.7" astroquery coloredlogs scikit-image h5py emcee tqdm mpi4py schwimmbad "iminuit>=2" "coverage>=3.6" coveralls deprecated pyyaml nose rubin-libradtran "getCalspec>=2.0.0"
mamba install -y "numpy>1.15" scipy "matplotlib>3.1" pandas llvmlite numba "astropy>=3.2" "photutils>=1.7" astroquery coloredlogs scikit-image>=0.20 h5py emcee tqdm mpi4py schwimmbad "iminuit>=2" "coverage>=3.6" configparser coveralls deprecated pyyaml nose rubin-libradtran "getCalspec>=2.0.0"
# python -c "from getCalspec.rebuild import rebuild_tables; rebuild_tables()"
mamba install astrometry

Expand Down Expand Up @@ -62,7 +62,7 @@ jobs:
- name: Run doctests and coverage
shell: bash -l {0}
run: |
./coverage_and_test.sh
.github/workflows/coverage_and_test.sh

- name: Coveralls
env:
Expand Down
File renamed without changes.
136 changes: 0 additions & 136 deletions .travis.yml

This file was deleted.

6 changes: 4 additions & 2 deletions config/stardice.ini
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ SPECTRACTOR_COMPUTE_ROTATION_ANGLE = disperser
# parameter to prevent from background subtraction
SPECTRACTOR_BACKGROUND_SUBTRACTION = True
# deconvolve spectrogram with simple 2D PSF analysis: False, True
SPECTRACTOR_DECONVOLUTION_PSF2D = True
SPECTRACTOR_DECONVOLUTION_PSF2D = False
# deconvolve spectrogram with full forward model: False, True
SPECTRACTOR_DECONVOLUTION_FFM = False
# value of sigma clip parameter for the spectractor deconvolution process PSF2D and FFM
SPECTRACTOR_DECONVOLUTION_SIGMA_CLIP = 8
SPECTRACTOR_DECONVOLUTION_SIGMA_CLIP = 100


[instrument]
Expand All @@ -28,6 +28,8 @@ OBS_ALTITUDE = 0.650
OBS_LATITUDE = 43.930832
# diameter of the telescope in meter
OBS_DIAMETER = 0.4
#diameter of the telescope secondary mirror in meter
OBS_SECONDARY_DIAMETER = 0.11
# observation epoch
OBS_EPOCH = J2000.0
# systematics on the instrument transmission
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ def __getattr__(cls, name):
return MagicMock()


MOCK_MODULES = ['mpi4py']
sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES)
# MOCK_MODULES = ['mpi4py']
# sys.modules.update((mod_name, Mock()) for mod_name in MOCK_MODULES)

# -- General configuration ------------------------------------------------

Expand Down
3 changes: 1 addition & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,10 @@ astropy>=3.2
photutils>=1.7.0
astroquery
coloredlogs
scikit-image
scikit-image>=0.20
h5py #==2.8.0
emcee #==3.0rc2
tqdm
#mpi4py
schwimmbad
iminuit>=2
coverage>=3.6 # <5
Expand Down
19 changes: 9 additions & 10 deletions runFitter.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from spectractor import parameters
from spectractor.extractor.spectrum import Spectrum
from spectractor.fit.fit_spectrogram import SpectrogramFitWorkspace, run_spectrogram_minimisation
from spectractor.fit.fit_spectrum import SpectrumFitWorkspace, run_spectrum_minimisation
from spectractor.config import load_config
Expand All @@ -16,10 +17,8 @@
help="Enter verbose (print more stuff).", default=False)
parser.add_argument("-o", "--output_directory", dest="output_directory", default="outputs/",
help="Write results in given output directory (default: ./outputs/).")
parser.add_argument("-l", "--logbook", dest="logbook", default="ctiofulllogbook_jun2017_v5.csv",
help="CSV logbook file. (default: ctiofulllogbook_jun2017_v5.csv).")
parser.add_argument("-c", "--config", dest="config", default="config/ctio.ini",
help="INI config file. (default: config/ctio.ini).")
parser.add_argument("-c", "--config", dest="config", default="",
help="config file to be given for spectra extracted with Spectractor<2.4. (default: ''.")
args = parser.parse_args()

parameters.VERBOSE = args.verbose
Expand All @@ -29,13 +28,13 @@

file_names = args.input

load_config(args.config)
if args.config != '':
load_config(args.config)

for file_name in file_names:
atmgrid_filename = '' #file_name.replace('sim', 'reduc').replace('spectrum', 'atmsim')
w = SpectrumFitWorkspace(file_name, atmgrid_file_name=atmgrid_filename, nsteps=1000,
burnin=200, nbins=10, verbose=1, plot=True, live_fit=False)
atmgrid_filename = '' # file_name.replace('sim', 'reduc').replace('spectrum', 'atmsim')
spec = Spectrum(file_name)
w = SpectrumFitWorkspace(spec, atmgrid_file_name=atmgrid_filename, verbose=True, plot=True, live_fit=False, fit_angstrom_exponent=True)
run_spectrum_minimisation(w, method="newton")
w = SpectrogramFitWorkspace(file_name, atmgrid_file_name=atmgrid_filename, nsteps=2000,
burnin=1000, nbins=10, verbose=1, plot=True, live_fit=False)
w = SpectrogramFitWorkspace(spec, atmgrid_file_name=atmgrid_filename, verbose=True, plot=True, live_fit=False, fit_angstrom_exponent=True)
run_spectrogram_minimisation(w, method="newton")
1 change: 0 additions & 1 deletion runSimulator.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
continue
spectrum_file_name = args.output_directory + '/' + tag.replace('.fits', '_spectrum.fits')
atmgrid = AtmosphereGrid(file_name)
SpectrumSimulatorSimGrid(spectrum_file_name, args.output_directory)
image = ImageSim(file_name, spectrum_file_name, args.output_directory, A1=1, A2=1,
pwv=5, ozone=300, aerosols=0.03,
psf_poly_params=None, with_stars=True)
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
[nosetests]
exclude=test_(extractor_ctio_planetary_nebula|atmosphere|astrometry)
exclude=test_(extractor_ctio_planetary_nebula|astrometry)
2 changes: 1 addition & 1 deletion spectractor/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

__version__ = '2.4'
__version__ = '2.5'
__version_info__ = tuple(map(int, __version__.split('.')))