Astronomy Using Unevenly Sampled Data : GSoC 2023 #737

pupperemeritus · 2023-06-07T17:30:03Z

This is a work in progress. Feel free to provide critique and suggestions for any bugs that may be or incorrect behaviour. So far I have implemented the base Lomb-Scargle Cross Spectrum and Power Spectrum classes. Do mention any useful statistical functions that may be of interest to people dealing with unevenly sampled data. I have so far only completed the slow implementation. As I work on the fast implementation, tests, statistical functions and documentation, I will tack on commits to this PR as i progress.

matteobachetti · 2023-06-08T21:21:00Z

The enable_deprecations_as_warnings issue is partially solved in commit #cc2a561bbf770fd269a82b9aa3704c303a36dc2c, feel free to cherry-pick.

codecov · 2023-06-08T21:34:54Z

Codecov Report

Merging #737 (03606c8) into main (5fac201) will increase coverage by 0.71%.
Report is 5 commits behind head on main.
The diff coverage is 100.00%.

❗ Current head 03606c8 differs from pull request most recent head 01a1533. Consider uploading reports for the commit 01a1533 to get more accurate results

@@            Coverage Diff             @@
##             main     #737      +/-   ##
==========================================
+ Coverage   96.62%   97.34%   +0.71%     
==========================================
  Files          41       42       +1     
  Lines        7388     7649     +261     
==========================================
+ Hits         7139     7446     +307     
+ Misses        249      203      -46

Files	Coverage Δ
stingray/__init__.py	`100.00% <100.00%> (ø)`
stingray/fourier.py	`99.79% <100.00%> (+0.03%)`	⬆️
stingray/lightcurve.py	`97.89% <ø> (ø)`
stingray/lombscargle.py	`100.00% <100.00%> (ø)`

... and 6 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

stingray/fourier.py

matteobachetti · 2023-06-22T07:47:43Z

@pupperemeritus a small suggestion: when you incorporate recent additions to main, please rebase instead of merging/pulling. This will avoid all the merge commits polluting the history.

pupperemeritus · 2023-06-22T07:49:09Z

I will do that 👍

stingray/lombscargle.py

matteobachetti

@pupperemeritus thanks for your work!
It's overall good, I left a few comments about things that can be improved.
Phase lags should work! I added a test that shows that, at least when the time arrays are the same, phase lags should be measured correctly.

stingray/fourier.py

stingray/lombscargle.py

pupperemeritus · 2023-09-05T14:18:20Z

I will make those changes

matteobachetti

Thanks @pupperemeritus. Please note that the _make_crossspectrum is the legacy interface in Stingray, and that much of the checks on EventLists are currently wrong and can be simplified.
As a final change, I would like to ask you to please use the from_events, from_lightcurves, etc. functions, to uniformize to the most recent versions of AveragedCrossspectrum. When implementing from_events, you will need to create light curves internally using a given sample time dt.

matteobachetti · 2023-09-12T13:40:42Z

stingray/lombscargle.py

+        ):
+            raise TypeError("Both the events are not of type Eventlist or Lightcurve or None")
+
+        if type(data1) == type(data2):


Instead of raising ValueError, I would just say that the lags will not be correct. The process should actually work.

matteobachetti · 2023-09-12T13:42:24Z

stingray/lombscargle.py

+                if len(data1.time) != len(data2.time):
+                    raise ValueError("data1 and data2 must have the same length")
+        else:
+            if (isinstance(data1, EventList) or isinstance(data2, EventList)) and (


The logic here is wrong. If you have event lists, the time arrays will always be different.

matteobachetti · 2023-09-12T13:43:22Z

stingray/lombscargle.py

+        if not isinstance(fullspec, bool):
+            raise TypeError("fullspec must be a boolean")
+
+        if np.logical_xor(


In general, I'd say you are overcomplicating the data type selection here. You have various instances of isinstance filtering that should be simplified. What don't you like of the machinery in the standard Crossspectrum?

I think it was something along the lines of EventList, Lightcurve intermixing while accounting for mixed type with one none. I think that train of thought went quiet overcomplicated. I guess I will simplify and bring it in line with the regular crossspectrum

matteobachetti

@pupperemeritus great new additions! Some parts of the code are not tested yet. I added an example test that should pass if the EventList machinery works, there should be a few more, some making the code run correctly and testing its values, some making it break in predictable ways. Please try to get the patch coverage to 100%

stingray/tests/test_lombscargle.py

…ragedCrossspectrum

Co-authored-by: Matteo Bachetti <matteo@matteobachetti.it>

matteobachetti · 2023-09-27T09:22:24Z

@astrofrog @jakevdp would the Lomb-Scargle cross spectrum be interesting to be put in astropy timeseries? @pupperemeritus's work might be useful for a more general audience than ours.

matteobachetti · 2023-09-28T10:24:12Z

@pupperemeritus, thanks for your hard work! This makes an excellent addition to Stingray, and it will be one of the highlights of the next release.

pupperemeritus · 2023-09-28T10:25:08Z

Thank you @matteobachetti. It was a fun working on it and a great learning experience.

pupperemeritus changed the title ~~Astronomy Using Unevenly Sampled Data : GSoC 2023~~ Astronomy Using Unevenly Sampled Data : GSoC 2023 [WiP] Jun 8, 2023

matteobachetti reviewed Jun 8, 2023

View reviewed changes

stingray/fourier.py Outdated Show resolved Hide resolved

matteobachetti mentioned this pull request Jun 9, 2023

Implement Lomb-Scargle Versions of Fourier algorithms for various products #689

Closed

pupperemeritus changed the title ~~Astronomy Using Unevenly Sampled Data : GSoC 2023 [WiP]~~ Astronomy Using Unevenly Sampled Data : GSoC 2023 : Part 1 [WiP] Jun 21, 2023

pupperemeritus force-pushed the LombScargle branch 2 times, most recently from 58097e6 to b5998dc Compare June 25, 2023 14:32

pupperemeritus force-pushed the LombScargle branch from b3d9ed1 to 59b2aa5 Compare August 3, 2023 10:21

pupperemeritus force-pushed the LombScargle branch 2 times, most recently from f013cc8 to d2d7968 Compare August 21, 2023 08:33

pupperemeritus marked this pull request as ready for review August 21, 2023 08:34

pupperemeritus force-pushed the LombScargle branch from 42b8646 to ef71402 Compare August 23, 2023 14:54

matteobachetti reviewed Aug 24, 2023

View reviewed changes

stingray/lombscargle.py Outdated Show resolved Hide resolved

matteobachetti reviewed Aug 24, 2023

View reviewed changes

stingray/lombscargle.py Outdated Show resolved Hide resolved

pupperemeritus force-pushed the LombScargle branch 3 times, most recently from b227128 to a9d66a3 Compare August 31, 2023 21:05

matteobachetti mentioned this pull request Sep 1, 2023

Fix failure with external forks #752

Merged

pupperemeritus force-pushed the LombScargle branch from cdb9428 to 8428ecd Compare September 2, 2023 19:00

matteobachetti requested changes Sep 5, 2023

View reviewed changes

pupperemeritus force-pushed the LombScargle branch from bc23d38 to 8ceaeab Compare September 6, 2023 20:12

pupperemeritus requested a review from matteobachetti September 7, 2023 14:50

matteobachetti requested changes Sep 12, 2023

View reviewed changes

pupperemeritus changed the title ~~Astronomy Using Unevenly Sampled Data : GSoC 2023 : Part 1 [WiP]~~ Astronomy Using Unevenly Sampled Data : GSoC 2023 Sep 17, 2023

pupperemeritus force-pushed the LombScargle branch from dad252a to bf96713 Compare September 17, 2023 11:16

matteobachetti requested changes Sep 18, 2023

View reviewed changes

stingray/tests/test_lombscargle.py Show resolved Hide resolved

matteobachetti and others added 10 commits September 26, 2023 18:21

Test time lag

fd91bc4

Phase lag fixes, code cleanup and optimization

bc6541a

Test Coverage Improvement

ed96e46

Updated docs and rewrote the classes to use modern interface from Ave…

f5d47b1

…ragedCrossspectrum

Example eventlist test

0c37abc

Co-authored-by: Matteo Bachetti <matteo@matteobachetti.it>

Code Cleanup, Tests and lc warning changes

68fcf81

method test fix

d89f0f5

Reintroduced ls notebooks

ccf0af0

Removed largememory

53521ff

fix for python 3.11 tests

c55c0d6

matteobachetti force-pushed the LombScargle branch from 43474dc to c55c0d6 Compare September 26, 2023 16:21

matteobachetti added 2 commits September 26, 2023 20:18

Fix the nphots parameter

8d23144

Import all from lombscargle

1957239

matteobachetti added 5 commits September 27, 2023 12:24

Make the autofrequency calculation more consistent and controllable

da9e4b6

Test maxfreq <0

436db70

Update notebooks

03606c8

Add reference to Lomb-Scargle tutorial

42087dd

Fix missing whitespace [docs only]

8a162a5

Refresh front page [docs only]

01a1533

matteobachetti enabled auto-merge September 28, 2023 10:42

matteobachetti approved these changes Sep 28, 2023

View reviewed changes

matteobachetti added this pull request to the merge queue Sep 28, 2023

Merged via the queue into StingraySoftware:main with commit be88264 Sep 28, 2023
5 checks passed

This was referenced Jan 5, 2024

Suggestions for additional algorithms #368

Closed

Implement CrossSpectrum for unevenly sampled data? #533

Closed

pupperemeritus deleted the LombScargle branch February 12, 2024 13:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Astronomy Using Unevenly Sampled Data : GSoC 2023 #737

Astronomy Using Unevenly Sampled Data : GSoC 2023 #737

pupperemeritus commented Jun 7, 2023 •

edited

matteobachetti commented Jun 8, 2023

codecov bot commented Jun 8, 2023 •

edited

matteobachetti commented Jun 22, 2023

pupperemeritus commented Jun 22, 2023

matteobachetti left a comment

pupperemeritus commented Sep 5, 2023

matteobachetti left a comment

matteobachetti Sep 12, 2023

matteobachetti Sep 12, 2023

matteobachetti Sep 12, 2023

pupperemeritus Sep 12, 2023

matteobachetti left a comment

matteobachetti commented Sep 27, 2023

matteobachetti commented Sep 28, 2023

pupperemeritus commented Sep 28, 2023

Astronomy Using Unevenly Sampled Data : GSoC 2023 #737

Astronomy Using Unevenly Sampled Data : GSoC 2023 #737

Conversation

pupperemeritus commented Jun 7, 2023 • edited

matteobachetti commented Jun 8, 2023

codecov bot commented Jun 8, 2023 • edited

Codecov Report

matteobachetti commented Jun 22, 2023

pupperemeritus commented Jun 22, 2023

matteobachetti left a comment

Choose a reason for hiding this comment

pupperemeritus commented Sep 5, 2023

matteobachetti left a comment

Choose a reason for hiding this comment

matteobachetti Sep 12, 2023

Choose a reason for hiding this comment

matteobachetti Sep 12, 2023

Choose a reason for hiding this comment

matteobachetti Sep 12, 2023

Choose a reason for hiding this comment

pupperemeritus Sep 12, 2023

Choose a reason for hiding this comment

matteobachetti left a comment

Choose a reason for hiding this comment

matteobachetti commented Sep 27, 2023

matteobachetti commented Sep 28, 2023

pupperemeritus commented Sep 28, 2023

pupperemeritus commented Jun 7, 2023 •

edited

codecov bot commented Jun 8, 2023 •

edited