Source_from_array functionality unclear

[kegardner]

It seems like source_from_array wants to take an n by m 2D array describing the fluxes at each pixel of an image and return an observable source. However, everything source_from_array returns seems to be something that can't be observed by OpticalTrains. Is this intentional? What is the purpose of source_from_array? Is there a way to take an arbitrary array of data and generate a source that can be observed by an OpticalTrain?
Additionally, source_from_file also appears to be non-working. Is there a particular setup that is required?

Reproduction (source_from_array issue):
from scopesim_templates.misc import source_from_array
import matplotlib.pyplot as plt
import numpy as np
import astropy.units as u
import yaml
import scopesim as sim

testarray = np.zeros((10,10))
testarray[0:10,0:10] = 5000

arraysource = source_from_array(arr=testarray,sed="sne/sn1a", amplitude = 15, pixel_scale=1 * u.arcsec, filter_curve="Ks")

source = arraysource

print(sim.rc.config)

exposure_time = 1000

file_name = f"arraysource_test_espt{exposure_time}"

sim.rc.config["!SIM.file.local_packages_path"] = ".\inst_pkgs\"

cmds = sim.UserCommands(use_instrument="MICADO",
properties={"!OBS.dit": exposure_time,
"!OBS.airmass": 2.25,
})

micado = sim.OpticalTrain(cmds)
micado.observe(source)
hdus = micado.readout(filename=f"fitsOutput/{file_name}.fits")

plt.imshow(hdus[0][1].data)
plt.show()

While this does generate an output image, it outputs to the console with the warning "WARNING - Empty image HDU."
Related, is_field_in_fov seems to be filtering out the above examples array. I'm having difficulty parsing exactly what the later half of the is_field_in_fov function is doing to calculate if a non-BG_SRC ImageHDU is in the fov. Is there a positional element which defaults on call to source_from_array and needs to be modified to align with the OpticalTrain?

[hugobuddel]

Thanks for the report @kegardner .

It seems that this bug has been fixed in #321. That is, the dev_master branch of ScopeSim does not print the warning and does produce an image. However, we have not yet released a new ScopeSim version with that fix.

If you wish, you can install the development version from ScopeSim with

pip install git+https://github.com/AstarVienna/ScopeSim.git

However, I'm not sure whether source_from_array() works as it should..

I've modified your code to include a small source in the center of your array (so the illumination is not uniform anymore):

from scopesim_templates.misc import source_from_array
import matplotlib.pyplot as plt
import numpy as np 
import astropy.units as u 
import yaml
import scopesim as sim

# Odd sized array, so it has a one-pixel center
testarray = np.zeros((9,9))
testarray[0:9,0:9] = 5000
# Add an actual source
testarray[4, 4] = 50000

source = source_from_array(arr=testarray,sed="sne/sn1a", amplitude = 15, pixel_scale=1 * u.arcsec, filter_curve="Ks")

exposure_time = 1000
# sim.rc.__config__["!SIM.file.local_packages_path"] = ".\inst_pkgs\\"

cmds = sim.UserCommands(use_instrument="MICADO",
                    properties={"!OBS.dit": 1000,
                                "!OBS.airmass": 2.25,
                                })

micado = sim.OpticalTrain(cmds)
micado.observe(source)
hdus = micado.readout(filename=f"sourcefromarraytest.fits")

plt.imshow(hdus[0][1].data)
plt.savefig("sourcefromarraytest.png")
plt.show()

And it shows up at the corner of the image.

It is possible to shift the source around, by changing where it is in the array, but it is not obvious to me how the function behaves.

@kegardner, could you try to install the development version of ScopeSim and see whether that behaves as you expect?

@teutoburg, do you perhaps have something insightful to say on why the source in this image is located at the edge, and not, e.g. in the center?

@teutoburg, should we release a new version of ScopeSim?

[teutoburg]

do you perhaps have something insightful to say on why the source in this image is located at the edge, and not, e.g. in the center?

There seems to be an issue on how ipu.rescale_imagehdu determines the scaled crpix fov_utils.extract_area_from_imagehdu matches the new header wcs values and the new data shape. But I don't understand why we didn't run into this earlier? I'll see if there's an easy-ish fix...

should we release a new version of ScopeSim?

After the coming Easter holidays perhaps. I'll be mostly off from tomorrow though Tuesday...

[teutoburg]

@hugobuddel I managed to get it in the center in fh/offbyone, but this caused other things to explode (who would have thought). As expected, this is not an issue in source_from_array, but again yet another incarnation of the off-by-one error.

[kegardner]

Going over the diff between dev_master and my current distribution (0.7.0, realizing now that I need to upgrade! I will also install the dev version and continue investigating there), it looks like the only substantive difference that relates to the current matter is an added call to xyp.sort(axis=0) in FieldOfView.extract_area_from_imagehdu; adding that presented me with the same results as @hugobuddel.
Still looking into what actually determines the placement of the source with respect to the bounds of the FOV, but it currently appears to be directly tied to the dimensions of the input ImageHDU.data (ergo the test array).