Persistence should be simulated

[hugobuddel]

It should be possible to simulate persistence on the raw data.

Currently the simulations are timeless. This would require some sort of observation time to be added.

A possible way to achieve persistence is to make the caller (e.g. MICADO-WISE) responsible for the bookkeeping:

• An observation time can be given to ScopeSIM (e.g. as TPL.START, which is normally output from the ICS).
• A previous set of exposures (e.g. previous simulations) with earlier observation times (MJD-OBS) can be given to ScopeSIM (as FITS files, or pyfits HDULists), as parameters to a Persistence effect (as well as some decay parameters).
• The Persistence effect calculates the residual persistence from the given exposures and adds this to the simulated data.

It should be determined where in the pipeline the persistence should be added. Perhaps just before the dark current?

[astronomyk]

I see two possible ways forward here

1. have scopesim save a file locally with the persistence pattern, and this pattern is updated each time a simulation is run.
2. implement a two-way street with the pipeline and the DFS (is there a "cool" name for the pipeline yet?), where scopesim can call the pipeline and ask for a persistance image. Maybe in the form of a microservice?

My preference is for the 2nd option, as the 1st has the disadvantage that if a user simulated the same object 100 times over, the persistance image will essentially be the same as the observed image, but will add unaccountable flux to the object.

That said, I probably won't add a persistence effect to the sci-team package

A further option would be just to have a static (or randomly generated) persistence image when needed

[hugobuddel]

About the options:

1. It is not clear whether it is possible to update a single image with each observation. It seems that the decay of the trapped electrons is not just a function of the current signal. That is, it seems to be an sum of exponential decays of all previous exposures. As in, that a high signal an hour ago would have a different decay then a medium signal a minute ago, even if the flux at the current time would be the same. If that is indeed the case, then you really need all previous observations up to a certain time in the past. But perhaps this is too much detail for our simulation goals.

2. Asking the DFS would be a bit cheating, because the persistence the DFS can estimate would be the same as the persistence it can correct, meaning that whatever algorithm it uses, it will always be perfectly capable of correcting the persistence.

The cleanest way would be to

3. Have the caller (the DFS) send all previous observations to ScopeSIM as input to an hypothetical Persistence effect. This effect would then calculate the persistence. But this is complicated and requires the concept of time to be added to ScopeSIM.

Simpler might be

4. Let the caller (the DFS) simply provide an persistence image. This would still be cheating as in 2, but seems to be the simplest thing to implement.

[hugobuddel]

From @oczoske

Here's a technical report from the Roman Space Telescope (WFIRST, right?), "Identification of an algorith to simulate image persistence". At first glance this seems doable, but is probably not top priority for Scopesim.

https://www.stsci.edu/files/live/sites/www/files/home/roman/_documents/Roman-STScI-000480-IdentificationAlgorithm.pdf

Roman-STScI-000480-IdentificationAlgorithm.pdf