feature/aggregate summary

[rhayes777]

Introduces a summary CSV over chosen fields from search output using the aggregator as per #1087

Does not currently support numerical errors as the current output is not sufficient and changing the output results in failing tests.

Usage:

aggregator = Aggregator.from_directory(directory)
summary = AggregateSummary(aggregator)

# Simply add the median pdf value by its path
summary.add_column("galaxies.lens.bulge.centre.centre_0")

# Add a median pdf value but give it a specific name
summary.add_column(
    "galaxies.lens.bulge.centre.centre_1",
    name="centre_1",
)

# If a latent_summary.json file is present latent values can be used implicitly
summary.add_column(
    "latent.value",
)

# Computed column values can also be added
def compute(samples):
     return 1
summary.add_computed_column(
    "computed",
    compute,
)

# Save the CSV
summary.save(output_path)

[Jammy2211]

I would maybe suggest AggregateCSV rather than AggregateSummary as the latter seems generic, assuming that we'll also have an AggregatePNG soon too.

Unless you think the API you've chosen makes more sense for some reason? I

[rhayes777]

Yeah makes sense

[Jammy2211]

I began writing an example script using this for PyAutoGalaxy.

First, I wrote my own Subplot object in PyAutoGalaxy, specific to its subplot_fit.png:

class SubplotFit(Enum):
    """
    The subplots that can be extracted from the subplot_fit image.

    The values correspond to the position of the subplot in the 4x3 grid.
    """

    Data = (0, 0)
    SignalToNoiseMap = (1, 0)
    ModelImage = (2, 0)
    NormalizedResidualMap = (0, 1)
    NormalizedResidualMapOneSigma = (1, 1)
    ChiSquaredMap = (2, 1)

However, the SubplotFitImage in autofit also had coded into it the subplot shape:

class SubplotFitImage:
    def __init__(self, image: Image.Image):
        """
        The subplot_fit image associated with one fit.

        Parameters
        ----------
        image
            The subplot_fit image.
        """
        self._image = image

        self._single_image_width = self._image.width // 4
        self._single_image_height = self._image.height // 3

The API for how a user customizes the png splicing makes a lot of sense:

image = agg_png.extract_image(
    ag.agg.SubplotFit.Data,
    ag.agg.SubplotFit.ModelData,
    ag.agg.SubplotFit.NormalizedResidualMap,
)

But the SubplotFitImage is not updated to account for the fact its using autogalaxy.

Is it feasible to make it so SubplotFit(Enum) contains all information we need to set up this interface, so in PyAutoGalaxy my method would read:

class SubplotFit(Enum):
    """
    The subplots that can be extracted from the subplot_fit image.

    The values correspond to the position of the subplot in the 4x3 grid.
    """

    filename = "subplot_fit.png"

    subplot_shape = (3, 2)

    Data = (0, 0)
    SignalToNoiseMap = (1, 0)
    ModelImage = (2, 0)
    NormalizedResidualMap = (0, 1)
    NormalizedResidualMapOneSigma = (1, 1)
    ChiSquaredMap = (2, 1)

This would mean that SubplotFitImage in autofit, which looks like it contains general functionality to do the splicing, would need a subplot_shape input.

[rhayes777]

I was thinking about this the other day and maybe the way to handle it is to have a mapping in each place the enum is used?

Something like

class Subplot(Enum):
    Data = 1
    SignalToNoiseMap = 2
    ...


position_map = {
    Subplot.Data: (0, 0),
    SignalToNoiseMap: (1, 0),
    ...
}

Alternatively we could bake everything in to the enum values. You can even use class instances to provide the values giving a bit more structure and flexibility.