-
Notifications
You must be signed in to change notification settings - Fork 49
/
RELEASE_NOTES
1135 lines (876 loc) · 53.9 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release Notes
Sherpa 4.12.1
=============
This release of Sherpa supports a patch for CIAO 4.12. This release is driven by fixes that were deemed necessary to warrant a patch to sherpa in the CIAO 4.12 release. The changes include issues which had the potential of impacting users by either stopping an analysis session or providing incorrect or potentially confusing results, particularly in the area of grating analysis and multiple datasets. Additionally, this patch includes several fixes and improvements such as initial support for Python 3.8.
This version of Sherpa has been tested with Python 3.5, 3.6, and 3.7. There has been limited python 3.8 testing as well.
Details
-------
Testing and infrastructure fixes have been omitted
#832 - Support building with NumPy 1.19
Compatibility updates for numpy v1.19 with regards to numpy.distutils and tostring deprecation
#781 - Docs: fix typo in docstring for calc_kcorr
Fixed a typo in docstring for calc_kcorrFix a typo in the documentation
for calc_kcorr: change "0." to "0.5" in one of the examples.
#759 - revert PR #444 (caching)
Revert the ARF cache added in #444, as well as some of the related code
changes, as they caused problems with Analysis in wavelength space
(e.g. #746)
#756 - calculate_photon_flux/calculate_energy_flux fix and improvement
Bug fixes and improvements for calculate_photon_flux and calculate_energy_flux:
- address flux and flux-density calculation issues discussed (fix #619)
- fix documentation of the lo and hi arguments (fix #308)
- add model parameter to calc_photon/energy_flux which allows users to
easily calculate "unabsorbed" fluxes
#747 - reworked regrid to eval usr grid, but 0 every where else
Modifies the 1D grid implementation to evaluate over the user-supplied
range, anything outside that is set to 0
#745 - ensure that min/max limits are applied to linked parameters before use
Enforce parameter restrictions (they must lie within their min/max range)
even when the parameter is linked to another (which has a different range).
#739 - Post 4.12.0 Updates
Update readme to include DOI for 4.12.0
#738 - Release 4.12.0
Merge the 4.12 Release into master
#737 - Update copyright year for the documentation
Read-The-Docs: update copyright year
#735 - Remove ChIPS support
Remove the ChIPS plotting code and configuration options from the code base
#734 - Change datastack query_by_header_keyword to not error if keyword is missing
Change the sherpa.astro.datastack query_by_header_keyword routine so that
it skips data sets which do not have the keyword, rather than raising a
KeyError. This also affects query_by_obsid.
#733 - fix a bug with fit(cache=False) passing the runtime option while fitting
#732 - Remove unused Python 2.7 compatibility code
Remove unused code from the removal of Python 2.7 support.
#696 - Support python 3.8
Updates to the support python 3.8 including
- remove syntax warnings from use of is with a literal
- remove deprecation warnings about PY_SSIZE_T_CLEAN
- fix index error (use scalar indexes)
- update the documentation to highlight initial python 3.8 support
Sherpa 4.12.0
=============
This release of Sherpa is based on the CIAO 4.12 release and includes additional
bug fixes and improvements.
This version of Sherpa has been tested with Python 3.5, 3.6, and 3.7.
This release also provides support for XSPEC 12.10.1 (patch 'a' or later) in
addition to versions 12.10.0 (included in CIAO) and version 12.9.1.
Details
-------
Documentation and infrastructure fixes are not shown.
#388 - Address indexing DeprecationWarning/IndexError from NumPy for PHA data with filter + ignore_bad (fix #361)
#602 - Reduce integration tol
Changed integration routine tolerance from double epsilon to float epsilon
since opt routines hove tolerance of ~ 1.0e-7
#606 - stop using numpy.typeNA which is now deprecated
A numpy deprecation warning was fixed by removing the usage of typeNA, which
was not documented and will be removed in a future release of numpy. The
print_fields function has been changed to include a default mapping using
the current typeNA implementation.
#608 - update ciao default plotter to matplotlib
The default Sherpa plotting package has been changed to matplotlib for
CIAO users as well (it had been the default for standalone users for
several years). The update will be applied to new users when the
`sherpa` command is run. Users of previous versions would need to
edit/regenerate the file.
#616 - enable run-time-option to test cache (default=True)
Capability to allow the user to turn off caching at runtime for testing
purpose.
#622 - Do not create warnings about error bars in plots if no error bars are to be shown (fix #621)
Remove the The displayed errorbars have been supplied with the data or
calculated using chi2xspecvar... warnings that appear for data,
residual/ratio, and fit plots when the user has explicitly turned off
the display of errorbars (by setting the yerrorbars plot-preference
setting to False).
#626 - add plot_fit_ratio and plot_bkg_fit_ratio functions
Add the plot_fit_ratio function to the sherpa.ui layer (to match
plot_fit_resid and plot_fit_delchi) and plot_bkg_fit_ratio to the
sherpa.astro.ui layer. Added tests for a number of plot types.
#640 - fix 'tuple index out of range' error in the fit.est_errors method
Sherpa's Confidence class is confused if a parameter is frozen then
thawed after a fit resulting with an exception of the form: IndexError:
tuple index out of range. The error does not show up if conf is called
from the ui level
#647 - fix numpy hist warnings
When plotting a PDF Sherpa now calls numpy.histogram with density=True |
False rather than normed=True|False. There is no change visible to the
user other than a warning that will not be issued anymore. normed=True
was broken for non uniform bins, but our code always produces uniform
bins, so we should never hit the problematic case.
#658 - Handle files with MJD_OBS or MJD-OBS keywords (datastack module)
The sherpa.astro.datastack module has been updated to deal with files
that use the MJD-OBS keyword instead of MJD_OBS. This is only used by
the show_stack routine, which prints out several keywords from each
file in the stack.
#665 - Remove python 2.7 support (fix #498)
Remove support for Python 2.7 from Sherpa. The metadata used by pip now
requires python 3.5 or higher (but not version 4),
#667 - improve matplotlib plot_fit plots (order of data and model)
Changes to Zorder of plot objects drawn by matplotlib to be more readible
#674 - Add link/unlink parameter tests
This PR adds three tests for sherpa's un/link commands.
#677 - XSPEC: build against 12.10.1m by removing support for models
Remove support for XSrelline, XSrelline_lp, and XSrelline_lp_ext models.
These were added in 12.10.1 but removed in 12.10.1m. It does not seem
worth supporting just those versions where they were available.
#680 - Remove the unused parameter in sherpa.astro.optical.LogEmission model (fix #219)
Remove the unused parameter in the sherpa.astro.optical.LogEmission
model. The LogEmission model only has four parameters (fwhm, pos, flux
and skew) consistent within the calc method. The hidden parameter limit
and it doc were removed.
#682 - jointplot now respects the ratio argument (top plot is larger) with matplotlib backend
The matplotlib back end now makes the main (top) plot taller than the
secondary (bottom) plot when using the sherpa.ui.plot_fit_resid and
sherpa.ui.plot_fit_delchi routines (this also holds for the
sherpa.astro.ui variants).
#683 - Remove chips from the documentation
Remove mention of ChIPS from the documentation (both on read the docs
and in the sherpa.astro.ui and sherpa.ui modules), replacing with
matplotlib where appropriate.
#684 - Add default config options
Whenever an option is read from the configuration file (~/.sherpa.rc)
provide a default option in case the configuration file is missing
(or is missing this option).
#685 - raise ModelErr when model using Data1DInt have overlapping bins
Adds a check to regrid make sure the integrated bins do not overlap.
The PR is a fix for issue #569
#688 - Improve documentation and testing for get_source_plot when sent lo/hi arguments
Improve the documentation for the sherpa.astro.ui.get_source_plot routine,
describing how touse the result when the lo or hi arguments are sent.
#691 - add warning when chips is selected but cannot be imported (rebase of #648)
Sherpa now warns the user if Chips is selected as a backend but it is
not available in the installation
#692 - Allow users to override plot preferences when creating a plot
Change to allow kwargs to be specified to change plot preferences at
creation time
#698 - Add warnings in documentation about masked arrays
Updates docstring to indicate that sherpa doesn't support numpy masked
arrays
#701 - a fix for /data/lenin2/Test/CIAO4.11/SherpaRegressionBeta1/45
Enables the testers to turn on (or off) caching at runtime to test #444
#704 - change calc_ftest from delta dof to dof (similar to XSPEC) - fix for #641
Update to make sherpa's calc_ftest provide same results as XSPEC's
ftest (fix to #641)
#705 - modelCacher1d needs to have a couple of deep instead of shallow copy()
Update to make deep copy of cached function values (fix to #673)
#710 - Add warning about masked array to sherpa.astro.ui.load_arrays
Updates docstring to contain warning consistent with #698
#712 Minor typo in docs
Corrects a typo "is is" in the documentation.
#716 Fix for plot_cdf: plot() got an unexpected keyword argument 'clearwindpw'
Recent changes broke plot_cdf()due to a typo 'clearwindpw'. This update
corrects the typo.
#721 Docs: update installation notes for conda environment
Add a warning about the need to set PYTHON_LDFLAGS to ' ' on macOS when
building within an anaconda environment. This is a documentation-only
change.
#725 convert a list to np.array to avoid warning messages
Fixes issue #723 (a lot of warnings while running 'resample_data()' in
CIAO4.12 Sherpa)
#728 a fix cause using load_multi_arfs causes caching error
Fixes issue #717 (unable to fit if script uses load_multi_arfs/rmfs)
Sherpa 4.11.1
=============
This release of Sherpa introduces several functional improvements and bug fixes,
in particular Sherpa now has support for:
* asymmetric error bars
* PSFs with better pixel resolution than the data
* running optimization in parallel
Details
-------
#630 Fix "get_int_proj does not work when recalc=True" (#543)
get_int_proj did not work when recalc=True on Python 3. This has now been fixed.
#615 Asymmetric Errors
Sherpa now supports asymmetric error bars. Errors can be read through a new
`load_ascii_with_errors` high level function, or through the new
`Data1DAsymmetricErrs` class. Sherpa uses bootstrap for estimating the uncertainties.
#585 plot pvalue
Updates to utilize the appropriate response files (ARF and RMF) for X-ray spectra
and changes to the p_value output to 1/(number of simulations) when p_value is 0
and the number of simulations in not large enough.
#596 Run optimization algorithms over multiple cores
This PR enables the user to run the optimization algorithms (DifEvo, LevMar,
and NelderMead) on multi-cores.
#607 PSF rebinning (fix #43)
Sherpa now supports using a PSF with a finer resolution than 2D images. If Sherpa
detects that the PSF has a smaller pixel size than the data, it will evaluate the
model on a "PSF Space" that has the same resolution as the PSF and the same footprint as
the data, then rebin the evaluated model back to the data space for calculating the statistic.
#614 Refactor data classes (fix #563, #627, #628)
Sherpa's basic data classes have been refactored and cleaned up to help
facilitating fixing bugs and implementing new features. New tests were added to reduce
the chances of introducing regressions in the future.
#612 Fix #609 User Model: unable to change parameter values
A regression introduced in Sherpa 4.10.0 presented users from change
user-model parameter values through direct access. This issue has been
fixed. Several tests were added to reduce the chance of regressions in
the future.
Fix #514 - the command line tool `sherpa_smoke` in the conda package now correctly honors
command line arguments.
Sherpa 4.11.0
=============
This release of Sherpa is based on the CIAO 4.11 release and includes
additional bug fixes and improvements.
Optimization routines and statistic methods were made more robust with several
bug fixes and improvements.
This version of Sherpa has been tested with Python 2.7, 3.5, 3.6, and 3.7.
Support for Python 2.7 is being deprecated and may be dropped in future
releases.
This release also provides support for XSPEC 12.10.1 (patch 'a' or later) in
addition to versions 12.10.0 (included in CIAO) and version 12.9.1.
Details
-------
Documentation and infrastructure fixes are not shown.
#444 Improve caching
When fitting multiple datasets simultaneously Sherpa now tries to cache
model evaluations to improve performance.
#465 Support XSPEC 12.10.0 (fix #436)
XSPEC 12.10.0e is now supported
#508 Ensure fields are initialized before use (pileup code)
Resolved an issue where some fields in the C++ extension could have been
uninitialized in a corner case.
#523 Ensure the ui modules have a unique list of exported symbols (fix #502)
Remove the `sherpa.utils.erf` symbol from the two `ui` modules as it is
"over-written" by the ModelWrapper-created symbol when applied to
`sherpa.models.basic.Erf`.
#524 Be explicit about 'invalid escape characters' in strings
Convert strings that contain "invalid" escape characters, which newer
versions of Python complain about, to raw strings by preceding them with a 'r'
character.
#525 Fixed based line radpro_dm test and changed double --> real (template)
#530 Ensure XSPEC 12.10.0 uses ATOMDB 3.0.9 by default
Ensure that the default AtomDB version is 3.0.9 when using XSPEC 12.10.0.
This is to fix an issue with the models-only XSPEC installation in XSPEC
12.10.0 which uses a default version of 3.0.7 but only provides data files for
3.0.9. Without this change models such as XSapec would return all 0's (unless
the AtomDB version was set manually by the user).
#534 XSPEC 12.10.1 support and mtable fix
Sherpa now supports XSPEC 12.10.1. Several issues were fixed to ensure
compatibility with this release. Note that there are known issues with version
12.10.1 (no patches), so at least v12.10.1a must be used.
#537 / #552 sherpa.rc changes
Sherpa does not read the verbosity.level setting from sherpa.rc anymore but
the option is still in the file for backward compatibility
#539 Fix XSPEC models: mtable table models and ismabs (fix #535, #538, #540)
Fix to address issues with XSpec- #535 XSPEC multiplicative table models
(mtable models loaded with load_xstable_model); #538 incorrect evaluation of
the ismabs model; #540 the kerrd model evaluated to 0 for XSPEC 12.10.0 and
later.
#546 Small code cleanup in LevMar
small code cleanup in the LevMar class to match the other classes in that
file.
#564 / #567 Harmonise the handling of dof is zero or negative in fit and calc_stat_info (fix #565)
The `fit` and `calc_stat_info` methods now use the same code to calculate
the reduced statistic (`rstat`) and "quality of fit" (`qval`) values. This
avoids a `TypeError` when the number of degrees of freedom is negative in
`calc_stat_info` and ensures that `NaN` is returned for both values when the
goodness-of-fit calculation has failed (e.g. the reduced statistic is zero or
less or the statistic is negative), whereas in previous versions the `qval`
value could be `None` or `1` depending on the code path.
#576 Fix regression with responses `get_x` method
Version 4.10.1 introduced a regression which made the `get_x()` method fail
when called on RMF and ARF objects. This has been resolved.
#583 Update pager code (fix #445 #561)
Replace the use of an external pager (such as `more` or `less`) with a
Python one. This means that Sherpa no-longer uses the PAGER environment
variable, and that the screen output from the `show_*` series of commands
should now appear in the correct location when using a Jupyter notebook or the
spyder application.
Sherpa 4.10.2
==========
This release fixes a regression introduced in Sherpa 4.10.1 related to PSF convolution. As part of the long term PSF rebinning improvements,
Sherpa 4.10.1 introduced a check to validate that the data pixel size and the PSF pixel size match. If they don't match, then a warning is issued.
The change did not account for an edge error case. Sherpa 4.10.2 solves this problem.
The regression introduced a downstream regression in the gammapy project.
Details
---------
#551 Fix gammapy/gammapy#1905, catch errors with psf pixel size.
Sherpa did not properly catch errors when checking the PSF pixel size matches the image pixel size. This has now been fixed.
Caveats
-----------
There is a potential issue for macOS conda users with python > 3.6 and matplotlib v3.0.1, when pyqt5 is not installed and selected as a matplotlib
backend. With this specific combination of platform and dependencies matplotlib can fail to work properly. The issue can be replicated without
importing Sherpa, so this is not a Sherpa bug.
Several workarounds exist, the simplest of which are:
* install pyqt5 (`conda install pyqt`).
* downgrade matplotlib to v2 or v3.0.0 (`conda install matplotlib=3.0.0`)
Sherpa 4.10.1
=============
This release fixes several bugs and introduces a few new features, notably the ability to evaluate model components on
arbitrary grids and generate user-defined ARFs and RMFs. Also, as of this release Sherpa will no longer rely on any
Fortran code. See the following section for details.
It is now possible to build the Sherpa documentation using Sphinx. Additionally, the Sphinx documentation is
automatically built and hosted on ReadTheDocs: https://sherpa.readthedocs.io/
Details
-------
#407 Ensure 0-length array is an error in filter_resp (fix #405)
Add an explicit check in the C++ filter_resp code to error out if the noticed channels array
is empty.
#422 Improve error message with wrong xspec version
Improve the handling of XSPEC versions mismatch.
#466 Fix bounding box out-of-bounds memory read
Avoid an out-of-bounds memory read when calling pad_bounding_box (when the data is not
matching the expected conditions for this call).
#469 Evaluate model on finer grid
Sherpa users can now define arbitrary grids, called evaluation spaces, on which to
evaluate individual model components, both in 1D and 2D. This can be useful in a number
of cases, for instance when it is desirable to evaluate models on a finer grid than the one
defined by the data, or in convolution models where information outside of the data range
can be used to reduce boundary effects or to inform the evaluation inside the data space.
Also, when plotting individual 1D model components (plot_source_component), if a specific
evaluation space was attached to the model then the plot will be of the model evaluated
over that evaluation space, not the data space. Other plotting commands should be
unaffected by this change.
#470 Order for np.ones() should be a one length string not a bool
Fixed a `DeprecationWarning` in the optimization module.
#471 Using overwrite rather than clobber of astropy.io.fits
`clobber` in astropy has been deprecated for a while now in favour of overwrite and thus
issues a `DeprecationWarning`. Sherpa now uses `overwrite` instead.
#475 Fix unecessary runtime warning (fix #402)
A condition check in the optimization code was modified so as not to produce a warning
when the value is a NaN.
#481 Remove Fortran code
Sherpa does not rely on any Fortran routines anymore, except those compiled in XSPEC.
The existing Fortran code, which was used mainly in some of the optimization routines, has
been replaced by equivalent C++ code. This means that gfortran is no longer required for
building Sherpa, although a version of libgfortran compatible with the one used to link the
XSPEC libraries would still be needed at runtime if the XSPEC extension is
built.
#482 ARF/RMF creation functions
The `create_rmf` and `create_arf` functions now allow users to easily generate user
defined response objects.
#486 `ZeroDivisionError` in `calc_stat_info` (fix #476)
Sherpa did not catch divisions by zero in the calc_stat_info command. That has been fixed.
#484 Equivalent Width errors
Several optional arguments `(params=None, error=False, otherids=(), niter=1000)`, were
added to `eqwidth`. If `error = True`, then get_draws shall be run if the fit stat is one of the
following: (Cash, CStat, WStat) otherwise a multivariate normal distribution shall be run to
generate the samples. The optional niter parameter is used to generate the number of
samples. The optional parameter otherids is only used if get_draws is used internally when
multiple data sets were used to fit. Alternatively, the user can enter the samples via the
`params` option where the samples must be a `numpy.ndarray` of dimension `(num, npar)`.
#487 PSF bin size warning
Sherpa assumes that the PSF image and the data have the same pixel size. When this is
not true Sherpa ignores the difference, which results in a larger PSF being applied. From
now on Sherpa will issue a warning when the PSF image
pixel size and the data image pixel size are different.
Sherpa 4.10.0
=============
This release of Standalone Sherpa corresponds to the Sherpa code released as part of CIAO 4.10.
Sherpa 4.10.0 fixes several bugs related to the support of instrumental responses, including improved support of XMM and
Swift responses.
Also, this release fixes a significant bug in the support of user statistics, improvements to the Python 3 compatibility,
more robust usage of the numpy API, as well as several other minor bug fixes and new tests.
Additionally, this release introduces support for XSPEC 12.9.1n models, as well as the ability to use aliases for
parameter names. Some parameter names have been deprecated and may be removed in a future release. We reviewed the
parameter limits for many models and updated them to reflect the latest XSPEC specification. Also, multiple versions of
XSPEC are now supported, through optional models and version-dependent low-level function calls. This feature is for
advanced users building Sherpa from source. Note that Sherpa has been tested against XSPEC 12.9.0i, 12.9.0o, and
12.9.1n. Note that XSPEC is not directly supported by the standalone binary builds, and users are expected to build
Sherpa from sources if they want to link it against their version of XSPEC. These changes make it easier for user to
link different versions of XSPEC with the same Sherpa code base. Also note, however, that XSPEC 12.10 is not currently
supported.
Sherpa now requires pytest-3.3 or later for running the tests.
Details
---------
#438 Change from error to warning for OGIP violations
Given that users can not easily change a response file, and previous versions of Sherpa would allow the responses to
be used, this commit changes some of the errors recently introduced (in PR #383) into warnings. The errors for the
first bin edge being <= 0 are still left in because users of the sherpa.astro.ui module will find that these files are
auto-corrected for them (by PR #383).
#430 Update XSPEC parameters
XSPEC model parameter default values, limits, and properties were reviewed and updated to reflect changes in or
mismatches with the model.dat file shipped with XSPEC 12.9.1n.
#428 handle function name changes in XSPEC 12.9.1
Sherpa now supports multiple versions of the XSPEC models from the 12.9.0 and 12.9.1 series. Some models recommend
using the C-style interface over the older FORTRAN one, which may also resolve some memory access issues. For CIAO
4.10 users this means the interfaces to XSPEC models have been updated to the 12.9.1n versions. For Standalone Sherpa
users this means they can build and run Sherpa against a larger range of XSPEC versions, and Sherpa will pick the
XSPEC low level model function accordingly. Note that Sherpa has been tested against XSPEC 12.9.0i, 12.9.0o, and
12.9.1n. The 14 models that have been changed are: apec, bapec, bvapec, bvvapec, gaussian, lorentz, meka, mekal,
raymond, vapec, vmeka, vmekal, vraymond, and vvapec.
#427 Add support for XSPEC models added in 12.9.1 (fix #331)
Add support for models added in XSPEC 12.9.1: bvtapec, bvvtapec, tapec, vtapec, vvtapec, carbatm, hatm, ismabs,
slimbh, snapec, TBfeo, TBgas, TBpcf, TBrel, voigt, xscat.
Version 12.9.0 of XSPEC can still be used, in which case the models can be generated - e.g. a user can say xstapec.mdl
to create a Python object - but it will error out when evaluated.
The Si and S elemental parameters for the ismabs model have been renamed from the XSPEC versions since they are not
unique using the caseinsensitive matching used by Sherpa: so SI, SII, SIII and SiI, SiII, SiIII have been renamed S_I,
S_II, S_III and Si_I, Si_II, and Si_III respectively.
Low level support for the following convolution models from XSPEC 12.9.1 has also been added, but there is no Python
model support for these: clumin, rfxconv, vashift, vmshift, xilconv.
#412 support ROSAT PSPC (and similar) RMF files with AstroPy
Add explicit tests for reading in, and using, ROSAT PSPC PHA and RMF files. Fix a bug in the AstroPy back-end handling
of the ROSAT RMF file.
#409 EmissionGaussian model when skew parameter is not unity (fix #403)
The `sherpa.astro.optical.EmissionGaussian` code has been fixed for the case when the skew parameter is not unity. The
documentation has also been updated.
#399 Python 3 fixes and new tests for pha2 datasets
Fixed several problems when using Sherpa with Python 3: failures when calling `list_bkg_ids` and `list_response_ids`
and the parameter querying after `paramprompt(True)` has been called.
There is also a change to avoid a `FutureWarning` being raised in the astropy backend when reading in PHA2 data files.
Tests have been added to test the reading of a PHA2 format dataset, and new files added to the `sherpa-test-data`
repository for this purpose.
#398 Check that list_samplers returns a list (fix #397)
The `list_samplers` function now returns a list in Python 3 as well.
#396 Fix `set_xlog` and related commands using Python 3 (fix #393)
Fix use of commands that set the plot state using the `sherpa.astro.ui` module, such as `set_xlog` and `set_xlinear`,
when using Python 3.
#395 Add xspec optional models
Sherpa has new infrastructure for supporting multiple versions of XSPEC at the same time, with models that are built
and enabled conditionally depending on the version of XSPEC being linked. Models are assumed to have the same
parameters and low level functions across supported versions.
#394 XSPEC build and functionality improvements
Add support for reading and changing the path to the XSPEC "manager" directory, and reading the current XSPEC "model"
directory (`get_xspath_manager`, `set_xspath_manager`, and `get_xspath_model`). These are intended for the power user
and so require an explicit import from `sherpa.astro.xspec`.
There are several improvements to the build and interface to the XSPEC model library: these have no user-visible
changes.
#390 Add nlapec model
The `nlapec` XSpec model has been added. Note that XSPEC 12.9.0 is now required.
#385 DS9 calls return string in Python 2 and 3 (fix #319)
Some low lever `xpa` calls were returning byte strings rather than strings in Python 3. This had particular impact on
the `image_getregion` function. This has now been fixed.
#383 Replace 0-energy bins in ARF and RMFs (fix #332)
Sherpa now performs some validation of the energy ranges of ARF and RMF files against the OGIP standard. If the ranges
are inconsistent then the code will return with an error. If one energy bound is 0 the bound is replaced with a small
number to avoid numerical issues like division by zero. A new configuration option `minimum_energy`, if present,
allows users to override Sherpa's default behavior.
#379 Use a line not span to draw horizontal line (fix #378)
Ensure that the line at y=0 (residual) or y=1 (ratio) is drawn for residual or ratio plots when using matplotlib 2.0.
#373 Make sure ds9 properly works with Python 2.7 (fix #368)
For Python versions prior to 3.2 add in explicit code to make _Popen work as a context manager.
#352 Add low-level support for the XSPEC rgsxsrc convolution model
Add low-level support for the rgsxsrc convolution model in XSPEC. This is intended for R&D into fully supporting XSPEC
convolution models in Sherpa.
#255 Add aliases for parameter names so to support new xspec names (fix #74)
The XSPEC models have been updated to use parameter names that match the new XSPEC naming scheme introduced in XSPEC
12.9.0. The old parameter names can still be used as aliases for the new name, but are deprecated and may be removed
in a next major release. Note this allows any models to define aliases for their parameters.
Sherpa 4.9.1
============
This version introduces full support for Python 3.6. It also fixes issues with non-Chandra response files, correctly
handles the AREASCAL column in PHA files, and fixes a significant regression that was preventing user statistics from
working in v4.8.2. It also introduces a number of smaller improvements and fixes. In particular, quite a few
improvements have been made to the documentation and to the testing framework, including several new tests to improve
stability.
Details
-------
Infrastructure and minor non-functional changes have been omitted.
#335 Fix `setup.py` install command
The `setup.py install` command was not enforcing the installation of the dependencies listed in `setup.py`. This has
been fixed.
#368 Remove ds9 warnings when run under Python 3.6
Update the DS9 code so that external processes are cleaned up properly, so to remove the potential ResourceWarning
warnings when running DS9 on Python 3.6.
#351 fix handling of AREASCAL column in PHA files (fix #350)
Add support for handling the AREASCAL value (either scalar of vector) for PHA data sets. This array is used in XMM
RGS data to handle missing chips.
#358 Properly handle Swift RMF when using Astropy backend (fix #357)
A Swift RMF could not be read in when the AstroPy back end was in use. The problem was that the code did not support
RMF matrices that were not stored as variable-length arrays. This has now been fixed, and new tests have been added
for this kind of files.
#343 Fix user statistics regression (fix #341)
A number of regressions were introduced in version 4.8.1 up to version 4.9.0, so user statistics that were properly
working in version 4.7 have not been working any more. This has been fixed, and a number of regression tests have
been added.
#381 Fix `bdist_wheel` command
We reviewed our customization of the setuptools commands as we realized that a recent bug fix (#335) introduced a
regression in the bdist_wheel command used by pip to build Sherpa from source distributions. Both commands should now
properly work.
Sherpa 4.9.0
============
This version fixes many bugs in the Python 3 support. Moreover, it includes a significant
refactoring of the `Fit` and `Stat` classes that made it possible to fix several bugs
related to the recent `wstat` implementation while making these classes more maintainable
and extensible.
Note that this version deprecates the use of `load_table_model` for XSPEC models.
Sherpa/XSPEC users should use the new `load_xstable_model` function instead.
Details
-------
Infrastructure and minor non-functional changes have been omitted.
#242 Avoid use of inspect.getargspec in Python3
Finish off the replacement of inspect.getargspec by inspect.signature.
#263 List_data_ids() fails on py3 with mixed id types (Fix #262).
Sherpa was sorting the list of dataset IDs in a non-python3 compliant fashion,
which resulted in issues when using strings and integers together as dataset
IDs. This has now been fixed.
#267 add wstat tests
Add several regression tests for wstat.
#282 Parallel_map not working on py3 with numcores=1 (Fix #277).
The `utils` function `parallel_map` failed on Python 3 when called with `numcores=1`,
i.e. on systems with only one processor/core. This has been fixed.
#283 Sample flux and numpy deprecations (Fix #273 and #276).
The `sample_flux` function was not working under Python 3 if the `scales` argument
was provided. This has been fixed. Also, a `DeprecationWarning` was issued by
`numpy` because during the `sample_flux` execution values were extracted from
an array with non-integer indices. This has also been fixed.
#284 String representation of data classes under py3 (Fix #275).
Data classes `DataPHA`, `DataARF`, `DataRMF`, `DataIMG`, and `DataIMGInt` in
`sherpa.astro.data` would throw an exception if users tried to print them as
strings, under Python 3. This has been fixed.
#287 Rewrite sherpa.stats.Stat.calc_stat and simplify sherpa.fit.Fit (fix #227 #248 #289 #292).
In order to fix several issues related to the WStat support, and in order to
make the code more maintainable, the `sherpa.stats.Stat.calc_stat` and
`sherpa.fit.Fit` classes have gone through a round of refactoring. This fixes
the following issues: #227 Issues using wstat when grouping/filtering data; #248
backscal column not treated properly for WStat; #289 calc_stat does not error out
if background subtracted data is used with Likelihood statistics; #292 stat info
does not include reduced stat/qval for wstat.
#295 Fix display of pileup model in Python 3.5 (Fix #294).
Fix display of instances of `sherpa.astro.models.JDPileup` so that, in Python 3.5,
they can be displayed after the model has been evaluated.
#304 replace file -> open (Fix #297).
The `save` and `restore` functions used to use the `file` function which is not
compatible with Python 3. This has now been fixed.
#305 Fix python 3 issues with some session commands (Fix #303).
The `set_xlog`, `set_ylog`, and `show_bkg_model` functions were not compatible
with Python 3. This has now been fixed (Issue #303).
#307 Move XSPEC table support to load_xstable_model and deprecate its support
in load_table_model (Fix #270).
Add the `load_xstable_model` routine to the `sherpa.astro.ui` module, which
supports loading XSPEC additive or multiplicative (atable and mtable) models.
The support for these models is still available via `load_table_model` in this
release, but it is deprecated. The `read_xstable_model` routine has been added
to the `sherpa.astro.xspec` module.
#312 Fix over-zealous code clean up in PR #287 affecting sigmarej.
Fits using the sigmarej iterated-fit method were broken if a filter had been
applied to the data before the fit and there are any bins that get ignored at
larger bin values than the filtered-out data. (This fixes a subtle regression
introduced by #287).
#313 Allow sequence=None when using gridsearch and Python 3.5 (Fix #309).
Allow the `gridsearch` optimiser to be used with the `sequence` option set
to `None` for Python 3.5.
Caveats
------------
The requirements for Sherpa are to build with Python 2.7 and 3.5. There has been limited
testing with Python 3.6, for which we distribute conda binaries. If in doubt, please install
Sherpa in 2.7 or 3.5 environments only. Support for Python versions 3.3 and 3.4 is possible
but would require community support.
It has been reported during testing that some versions of the matplotlib conda package do not
install properly because of a pyqt v5 dependency. If you encounter this issue, please pin down
pyqt to version 4, e.g. `conda install matplotlib pyqt=4`.
The `sherpatest` package is not distributed as a conda package anymore. This will probably be
true for the foreseeable future. The `sherpatest` package contains data and functional tests
that relies on external datasets, so it allows users and developers to run the entire regression
tests suite. If you want to install `sherpatest`, please use pip and github:
~~~~
$ pip install https://github.com/sherpa/sherpa-test-data/archive/4.9.0.tar.gz
~~~~
If you decide to run the full regression tests suite you should also have matplotlib installed.
If matplotlib is not installed a test will run and fail rather than being skipped. This issue
will be fixed in the next release.
Sherpa 4.8.2
============
This version of Sherpa is the first one to run under both Python 2.7
and Python 3.5. The python 3 conversion is considered a beta designed
to maintain backwards compatibility with python 2.7. As such, we
expect to have some iterations before the Python 3.5 support stabilizes.
The smoke test has been greatly simplified: rather that running all of
the unit and regression tests that do not require test data, the smoke
test now simply ensures that the basic installation works, i.e. that
basic commands can be run and that dependencies can be reached. This
allows the use of more advanced tools for actual unit and regression
tests without having to ship such tools to users.
Several bugs were fixed, some enhancements implemented, and some
deprecated calls to external API replaced. In particular:
* more documentation was migrated and is not part of the code base
as *docstrings*.
* the `calc_stat` API has been generalized to make statistic
functions more extensible.
* `calc_stat_info` now properly takes background into account when
`wstat` is selected.
Details
-------
Infrastructure and minor non-functional changes have been omitted.
#209: Fix docstrings for `group_snr()` and `group_adapt_snr()`. Docstrings
used to say "Combine the data so that each bin has a signal-to-noise
ratio of at least minimum" but the end of the groups are marked only
if the SNR exceeds the given SNR.
#210: The smoke test has been greatly simplified and it now simply ensures
that the basic installation works, i.e. that basic commands can be run
and that dependencies can be reached. Previously, the smoke test would
run all unit tests. Units and regression tests can now take advantage of
`py.test` and `mock` packages (the latter is part of the standard
library in Python3).
#211: Cleanup of documentation and code in `sherpa.astro.utils`. The
`calc_kcorr` function is now exported by sherpa.astro.utils. Minor
changes to the documentation in `sherpa.astro.utils` to conform to
Sphinx standards.
#221: Integrate existing model documentation (from external sources and
the CIAO ahelp documentation system) into the model classes.
#229: Code is both Python 2.7 and 3.5 compliant. This included updating
the CIAO dependencies: region, group, and stack libraries, which were
ported to Python 3 as well.
#242: Avoid use of `inspect.argspec`, which was deprecated in Python3.
#252: `plot_photon_flux` function was calling an internal function with
the wrong number of arguments and thus raising an error. This has been
fixed.
#253: `calc_stat_info` call failed when `wstat` was selected, as the
background was not taken into account. This issue has now been fixed.
#256: Fix docstring in `set_quality`, which were listing the wrong values
of the quality flag.
#257: Fix docstring for `levmar` tolerance, which was listing the wrong
function defaults.
Caveats/Known Issues
--------------------
The following are known issues with the standalone 4.8.2 release
**`show_all` does not work on Python3 with PHA**. E.g., the following
code throws an exception:
``` python
from sherpa.astro.ui.utils import Session
from sherpa.astro.data import DataPHA
session = Session()
session.load_arrays(1, [1, 2, 3], [1, 2, 3], DataPHA)
session.show_all()
```
**the `parallel_map` function throws an exception on Python 3** when
only one core is available on the system running Sherpa. This impacts
a number of functions taking advantage of tasks parallelization on
multi-core systems. This functions include `calc_flux`, `sample_flux`,
a number of functions for plotting parameter projections, and the
`grid_search` optimization algorithm.
**string representation of some (but not all) dataset classes is broken
on Python 3**. For instance, the following code will result in an exception:
```python
from sherpa.astro.ui import *
load_pha('3c273.pi')
print(get_data())
```
**`sample_flux` does not work on Python 3 when called with the `scales`
argument**, e.g.:
```python
from sherpa.astro import ui
ui.load_pha('sherpa-test-data/sherpatest/3c273.pi')
ui.set_model('polynom1d.p1')
ui.fit()
ui.covar()
scal = ui.get_covar_results().parmaxes
ui.sample_flux(p1, 0.5, 1, num=5, correlated=False, scales=scal)
```
Sherpa 4.8.1
============
Sherpa 4.8.1 is the standalone counterpart to the 4.8.0 release, which was
focused on supporting CIAO 4.8. In particular, this version introduces support
for newer versions of the dependency, along with some feature enhancements,
bug fixes and additional, more accurate tests.
The newly supported dependencies:
- matplotlib v1.5
- numpy 1.10 (with and without mkl support)
- xspec v12.9.0i (when building from source)
- astropy v1.1.2
- region library v4.8 (from CIAO 4.8)
Mode details below (infrastructure changes are not shown):
#102: fix issues when writing out FITS files using the `save_pha` and
`save_table` commands when using the `astropy`/`pyfits` backend (bug #46).
Fix for when the notice2d_id, notice2d_image, and the ignore version functions
are called with an invalid identifier (i.e. an identifier that is not an integer
or string value). The error is now an ArgumentTypeErr with the message "'ids'
must be an identifier or list of identifiers". It was a NameError with the
message "global name '_argument_type_error' is not defined".
#107: Normalize plot labels. There are two main changes for plots of PHA data
sets (and related quantities, such as the source, model, and ARF):
- plots with matplotlib now use the LaTeX support - so that 'cm^2' is
now displayed as a superscript; previously they were displayed
directly. This does not change the display with the ChIPS backend.
- plots created with `plot_source` used a different format to other
plots when analysis=wavelength, in that LaTeX symbols were used for
Angstrom and lambda (in other plots the string 'Angstrom' is used
instead). The source plots now match the other plots.
#109: fix #103 and #113 in order to support `matplotlib` v1.5.
#116: fix bug #27. The `astropy.io.fits`/`pyfits` interface used deprecated
functionality. The code was updated to use the replacement classes/methods when
available, falling back to the original code if not. The interfaces that were
changed were, when the new symbols are availble, to:
- use `astropy.io.fits.BinTableHDU.from_columns` rather than
`astropy.io.fits.new_table`
- use `astropy.io.fits.Header` rather than `astropy.io.CardList`
#137: upgrade CIAO region library to v4.8
#138: improve and fix issues in `save_all` function.
- added a new argument to `save_all`: if `outfile` is `None` then the `outfh`
argument is used
to define the output handle (the argument can be any file-like argument, such
as a file
handle like `sys.stdout` or the output of `open`, or a `StringIO` object)
- setting the `clobber` argument to `save_all` now means that the output file
(the `outfile`
argument, if not `None`) is deleted if it already exists; prior to this, the
file would be
appended to instead
- the source expression is now saved correctly for most cases (e.g. when not
using `set_full_model`); this is bug #97 but also affects non-PHA data sets
- the background model expression was not always written out correctly when
using PHA
data sets
- quality and grouping arrays of PHA data sets are now stored as 16-byte
integers rather
than a floating-point value (this has no affect on the results, but matches
the OGIP standard)
- fixed up saving the grouping and quality arrays of background PHA data sets
(this would only
be an issue if the background is being fit, rather than subtracted)
- basic data sets created with the `load_arrays` function are now written out by
`save_all`
as part of the script; this is intended for small datasets and may have
problems with
precision if used with floating-point arrays
- calls to `load_psf` are now correctly restored (they may not have been written
out correctly
if multiple data sets were loaded)
- user models are now written out to disk; this consists of two parts:
- writing out the function that defines the model, which may or may not be
possible (if
not, a place-holder function is added to the output and a warning
displayed).
- the necessary calls to `load_user_model` and `add_user_pars` are now
included in the output
- the Python code created by `save` all has undergone several minor changes:
- it now explicitly imports the `sherpa.astro.ui` module, so that it can be
run from the
IPython prompt using the `%run <filename>` command, or directly as `python <filename>`
- it uses the `create_model_component` function rather than `eval` to create
model
components (this is CXC bug 12146)
- many optional arguments to functions are now given as `name=value` rather
than
being a positional argument, to make it clearer what the script is doing.
- calls to `load_data` have been replaced by more-specific versions - e.g.
`load_pha`
and `load_image` - if appropriate
- there have seen several minor syntactic clean ups to better follow the
suggestions from PEP8
When writing out code that defines a user-model, there is no attempt to make
sure that
modules used by the function are available. These will need to be added, either
directly
or imported, manually to the output.
#151: Ensure AstroPy and Crates behave the same with gzipped files. Change the
behaviour of the AstroPy back end so that it matches that of Crates when given a
file name which does not exist, but a compressed version, with the suffix `.gz`
does. The Crates behavior is to read the file. This extends to PHA files whose
ancillary files - e.g. those stored in the `BACKFILE`, `ANCRFILE`, and
`RESPFILE` keywords - are given as unzipped names, but only the gzipped names
exist on disk.
As an example: if `pha.fits.gz` exists but `pha.fits` does not, then
load_pha('pha.fits')
will now load the file with either back end. If the response files are set to
`arf.fits` and `rmf.fits` (via the `ANCRFILE` and `RESPFILE` keywords), but only
the `.gz` versions exist, then they will now also be loaded by the AstroPy back
end.
#153: Make comparison test in `calc_chi2datavar_errors` less stringent, so to
include the case where `sqrt(x)=0`.
#155: The `get_draws` function now accepts a user-provided covariance matrix. If
no covariance matrix is provided, the covariance matrix computed by the default
implementation is used. Note that `covar()` must be invoked before invoking
`get_draws` if no covariance matrix is provided, otherwise `get_draws` will exit
with an error.
#158: Fix bug that prevented region ascii files to be read in standalone
Sherpa.
#165: Remove usage of deprecated numpy API.
#185: Fix the problem where if the working directory contained a file called
`x` or `y` then the `sherpa.astro.ui.image_data()` function would fail with the message