Aggregator for png splicing

[Jammy2211]

Use Case:

To model 500 strong lens from Euclid, I found the workflow of navigating the output folder to inspect results
slow,

Furthermore, it was common for the information I needed to be spread across multiple images, leading to more clicks:

I developed a workflow which went to .png files I had output in the pipeline, extracted cut-outs showing the parts of the
fits of each individual lens I was interested in and saved them to .png files in a single folder that I could inspect
quickly.

The following URL shows an example of how

https://github.com/Jammy2211/autolens_workspace_test/tree/main/agg_png_csv

If you run the file png_to_single_line.py, you'll make a folder png containing images of the two example images
that enable quick inspection:

This uses the PIL library to open .png subplots showing many images and extract the individual images of each subplot
and save them to a new folder with a new figure and file name.

The images used by the code above to extract subplots are in the dataset folder, because for Euclid modeling
this is where I chose to store the images I used to extract subplots. However, I think a better implementation
would revolve entirely around the standard output folder produced by PyAutoFit.

AggregatorPng

I am picturing an AggregatorPng class that mirrors the API of the Aggregator class, which will allow a user
to quickly extract images from the image subplots inside the output folder and save them to a new folder.

The GitHub above has an output folder which contains the example outputs for the two lens, which is what the
AggregatorPng class can be developed to work with.

Example Query 1:

The first query would be to produce the following .png showing the fit to a given lens:

All of the information needed to make this plot is contained in a single .png file of each lens fit, called
subplot_fit.png. For this image, the source pixelization is shown, so we need the results of source_pix[2]:

So the first query for AggregatorPng would be something like:

agg = AggregatorPng.from_directory(
    directory=path.join("output),
)

image = agg.extract_images(
    image_type="subplot_fit",
    new_order = [5, 6, 11], # These are actually correct and I have chosen them based on subplot_fit.png
    original_shape = (3, 4), # The shape of the subplots in subplot_fit.png
    new_shape = (1, 3), # The shape of the new image we want to make
)


image.output_to_png(
    output_path=path.join("output", 102018665_NEG570040238507752998)
)

Automating original_shape

The original_shape can probably be determined when the .png is loaded, as the shape is clear and some
clever library can probably determine this. I dont know of one, but never looked.#

Example Query 2:

The next image adds the dataset/*/rgb_0.png image to the subplot, which shows the data of the lens:

This means the AggregatorPng class needs to be able to load multiple images and combine them into a single image,
something like this:

agg = AggregatorPng.from_directory(
    directory=path.join("output"),
)

image = agg.extract_images(
    image_name="subplot_fit",
    new_order = [5, 6, 11], # These are actually correct and I have chosen them based on subplot_fit.png
    original_shape = (3, 4), # The shape of the subplots in subplot_fit.png
    new_shape = (1, 3), # The shape of the new image we want to make
)

# Use the functionality in `png_util.add_image_to_left` to add the rgb image, but also make the API support all
directions.

[INSERT CODE HERE]

image.output_to_png(
    output_path=path.join("output", 102018665_NEG570040238507752998)
)

Example Query 3:

The image we made above only accessed one .png, subplot_fit.png. However, the output folder contains many
subplots, which we may want to combine into a single image.

For example, we may want to combine the subplot_fit.png of the source_pix[2] with the source_lp[1] result, and
put their RGB images ont he left, as done in the original example:

This combines the following .png files:

At this point, I think you get the idea of the framework I am hoping we can develop.

Example Query 4:

The final example combines the same images (e.g. the same things extract from subplot_fit.png) as rows for each result folder (vis, nir_h, nir_j, nir_y:

I suspect the code you develop will already support the making of this image, via a clever combination of of AggregatorPng commands.

However, this image performs a specific pattern, finding all .png files of the same name in a folder and putting the extracted images as rows. This is a common pattern, and may warrant a short-cut API. However, I think just having examples which do this in a for loop, using the base AggregatorPng API, is probably better to teach the user how to use the tool in this way, so maybe just make sure this query is doable!

Missing .png Files

A user can disable the output of .png files, so the AggregatorPNG should raise an warning if the user tries to use
the API and the file is not present.

It should still try to produce a .png where possible, for example if a subset of the .png files needed to make it are avaialble. If none are available no .png should be output.

[Jammy2211]

Can we also make sure the zoom_png functionality in png_util is supported:

def zoom_image(image, zoom_factor):

    # Get image dimensions
    width, height = image.size

    # Calculate the size of the crop box
    crop_width = width // zoom_factor
    crop_height = height // zoom_factor

    # Calculate the coordinates for the crop box around the center
    left = (width - crop_width) // 2
    upper = (height - crop_height) // 2
    right = left + crop_width
    lower = upper + crop_height

    # Crop the image
    center_crop = image.crop((left, upper, right, lower))

    # Resize back to the original dimensions (optional, for zoom effect)
    return center_crop.resize((width, height), Image.Resampling.LANCZOS)

[Jammy2211]

Example subplot_fit.png:

[Jammy2211]

Extract Image with custom subplot shape

The extract_image function dertermine the subplot size intenrally, which is fine and should be default, but user input should also be possible:

image = agg_png.extract_image(
    ag.agg.subplot_fit.data,
    ag.agg.subplot_fit.model_image,
    ag.agg.subplot_fit.normalized_residual_map,
    # subplot_shape=(1, 3),
)

Image Folder Output

"""
__Output to Folder__

An alternative way to output the image is to output them as single .png files for each model-fit in a single folder,
which is done using the `output_to_folder` method.

It can sometimes be easier and quicker to inspect the results of many model-fits when they are output to individual
files in a folder, as using an IDE you can click load and flick through the images. This contrasts a single .png
file you scroll through, which may be slower to load and inspect.
"""
agg_png.output_to_folder(
    name="png_splice_single_subplot",
    path="output_folder",
)

Image Folder Named Output

"""
__Naming Convention__

By default, each subplot uses the input `name` with an integer index prefix to name the subplot, which is how
the output folder .png files above are named.

It is more informative to name the subplots after the dataset name. A list of the `name`'s of the datasets can be
input to the `output_to_folder` method, which will name the subplots after the dataset names. 

To use the names we use the `Aggregator` to loop over the `search` objects and extract their `unique_id`'s, which 
when we fitted the model above used the dataset names. This API can also be used to extract the `name` or `path_prefix`
of the search and build an informative list for the names of the subplots.
"""
name_list = [search.unique_tag for search in agg.values("search")]

agg_png.output_to_folder(
     name_list=name_list,
     path="output_folder",
 )

Add Extra png

Something to note about this is that actually, we want to add a list of images where the list size corresponds to the number of datasets we have fitted.

For example, the image that comes from extract_image is actually lots of images corresponding to each fit.

We therefore need pair it with image_rgb which is a list of rgb images for each dataset. Maybe this would require a generator?

"""
__Add Extra Png__

We can also add an extra .png image to the subplot, for example an RGB image of the dataset.

We create an image of shape (1, 2) and add the RGB image to the left of the subplot, so that the new subplot has
shape (1, 3).

When we add a single .png, we cannot extract or splice it, it simply gets added to the subplot.
"""
# image_rgb = Image.open(path.join(dataset_path, "rgb.png"))
#
# image = agg_png.extract_image(
#     ag.agg.subplot_dataset.data,
#     ag.agg.subplot_dataset.psf_log_10,
#     subplot_shape=(1, 2),
# )

# image = ag.add_image_to_left(image, additional_img)

# image.save("png_splice_with_rgb.png")