Skip to content
This repository has been archived by the owner on Jun 4, 2024. It is now read-only.

Integrate the conversion of timestamp to epoch into test #12

Closed
franciumxzf opened this issue Feb 17, 2023 · 1 comment
Closed

Integrate the conversion of timestamp to epoch into test #12

franciumxzf opened this issue Feb 17, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@franciumxzf
Copy link
Owner

franciumxzf commented Feb 17, 2023

image

@franciumxzf franciumxzf added the enhancement New feature or request label Feb 23, 2023
@franciumxzf
Copy link
Owner Author

The pfind epoch tests can be found in tests/test_pfind_epoch.py.

When we tried to convert epochs to timestamp arrays, we just used several epochs that matches the acquisition time interval instead of all epochs getting from chopper.c and chopper2.c. For example, if our acquisition time interval is 2^31ns (4 * 2^29), this is equal to the length of 4 epochs and we will read only the first 4 epochs and another 4 epochs after the separation time. However, this failed some of the test cases with longer frequency offset (specifically, 20221123165621, dataset13 and dataset22).

The reason why this happens is that the first epoch doesn't span 2^29ns as usual one epoch but shorter. For example, in dataset13, we have:

total time difference for one epoch: 276710385.75
epoch size: 6096
total time difference for timestamp of one epoch length: 536865102.53125
timestamp size of one epoch length: 11762

we can see that it is only about half compare with the raw timestamp file. As the function process_timestamp will take the timestamp from specific time interval and pass it to Fast Fourier Transform (FFT), some of the timestamp events read from epochs will be "filtered out" and the effective acquisition time interval is shorter. For longer frequency offset, this may not be enough for the algorithm to successfully identify a peak, hence fail the test. We verified this by running FFT with the raw timestamp files of same effective acquisition time interval as epochs, and it also cannot resolve a peak.

In order to solve this problem, when we convert epochs to timestamp arrays, instead of using exact same number of epochs as the acquisition time, we read one more epoch in each interval. This is to make sure that the after the process_timestamp function, we will still have enough events spanning enough time to run FFT and successfully resolve a peak. It then passed those tests that failed previously.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant