Documentation update

docs/.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 6b4bd43eeea172ae57af29bf18236b28
+tags: 645f666f9bcd5a90fca523b33c5a78b7

docs/Makefile

@@ -7,6 +7,7 @@ SPHINXOPTS    ?=
 SPHINXBUILD   ?= sphinx-build
 SOURCEDIR     = source
 BUILDDIR      = build
+HTMLCOPYDIR   = .
 
 # Put it first so that "make" without argument is like "make help".
 help:
@@ -18,3 +19,4 @@ help:
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	cp -rT $(BUILDDIR)/html $(HTMLCOPYDIR)

docs/_sources/args.rst.txt

@@ -0,0 +1,50 @@
+Command Line Arguments
+======================
+Command line arguments to be used with the files in the ``test/`` directory.
+
+.. autoclass:: src.args.CustomParser
+
+General Arguments
+-----------------
+Arguments used during both the generation and analysis of decision regions.
+
+* ``--save_loc/--save-loc``: directory in which to save program outputs.
+* ``--save_name/--save-name``: name to use for the created decision region hdf5 file.
+* ``--classes``: model output classes, in format category:option1,option2 (ex. COVID_positive:Yes,No).
+* ``--class_order``: used to designate which class corresponds with a model output of 0 or 1; default: 0,1.
+* ``--subgroup_attributes``: attributes to be used in addition to `--classes` to group samples and analysis.
+* ``--overwrite``: if passed, previously generated hdf5 or analysis files may be overwritten.
+
+Generation Arguments
+--------------------
+Arguments used exclusivly during the generation of decision regions.
+
+* ``--model_file/--model-file``: file path to the onnx model file.
+* ``--data_csv/--data-csv``: csv file which pairs sample ids with image file paths and class and attribute information.
+* ``--batch_size/--batch-size``: batch size to be used with the data loader.
+* ``--shape``: the shape of the generated vicinal distributions; default: triangle.
+* ``--steps``: the number of steps to take between samples in the triplet when creating the vicinal distribution. With --shape=rectangle, approximately step^2 virtual samples will be generated.
+* ``--n_triplets/--n-triplets``: the number of triplets to generate for each group.
+* ``--img_rel_path/--img-rel-path``: the common directory for the image file paths, include if the file paths in ``--data_csv`` are relative.
+* ``--random_seed/--random-seed/--random_state/--random-state``: the random state used during the selection of image triplets.
+
+Analysis Arguments
+------------------
+Arguments used exclusivly during the analysis and plotting of decision regions.
+
+* ``--out_function``: the function to be applied to model output scores; see `utilities <src/utils.py>` for options.
+* ``--aggregate/--agg``: how to aggregate composition analysis; options: class, group, all.
+	Note: decision region compositions are always calculated by triplet before being aggregating to ensure that each triplet has the same impact on the calculated compositiond despite slight variations in the number of virtual samples between triplets' decision regions.
+* ``--threshold``: the treshold applied during composition analysis; does not affect region plots.
+* ``--plot_only/--plot-only``: pass to not save composition analysis files.
+* ``--plot``: type of plot to generate; options: composition, performance, region; no plots will be generated if argument is not passed.
+* ``--show``: pass to show plots.
+* ``--display-only``: pass to not save plots.
+* ``--hide-percent``: pass to not include percent text on composition/performance plots.
+* ``--hide-errorbar``: pass to not include errorbars on composition/performance plots.
+* ``--save_dpi/--save-dpi``: dpi used when saving summary plots.
+* ``--plot_output_format/--plot-output-format``: Output formats in which the plots should be saved.
+* ``--plot_palette/--plot-palette``: color palette to be used during plotting, can be either a matplotlib colorpalette or a custom palette.
+* ``--plot_threshold/--plot-threshold``: threshold applied to ouput scores in 'region' plots; if None, no threshold is applied.
+* ``--n_per_group/--n-per-group``: number of decision regions to plot per group if plot = 'region'.
+*

docs/_sources/index.rst.txt

@@ -0,0 +1,45 @@
+DRAGen
+======
+Decsion Region Analysis for Generalizability (DRAGen) is a tool to analyze the decision space of an image classification model to increase understanding of the model's generalizability. A model's decision space maps a change in the input to a change in the model's output. DRAGen utilizes triplets of image samples to generate vicinal distributions of virtual images, created by linearly interpolating between the triplet images. These virtual images increase the density of available samples in the decision space, which allows for characterization of the decision space beyond the original finite data set. This insight into the decision space composition indicates how the model is likely to behave on data distributions upon which the model cannot generalize well.
+
+Getting Started
+===============
+Three inputs are required:
+
+1. **Trained Model:** the model must be saved in onnx format. Information on how to convert your model to onnx format can be found at the `onnx GitHub page`_.
+2. **Images:** Images can be saved in any format supported by PIL.
+3. **Input csv:** A csv file which can be used to map the image paths to subgroup attributes.
+
+Examples implementation can be found in the `examples folder`_.
+
+Scripts to generate and analyze decision regions can be found in the ``test`` folder, 
+all arguments used to run these scripts are located in ``src/args.py``.
+Example inputs are included in the ``examples`` folder.
+
+.. _onnx GitHub page: https://github.com/onnx/tutorials#converting-to-onnx-format
+.. _examples folder: https://github.com/DIDSR/RST_Decision_Region_Analysis/tree/main/examples
+
+Terminology
+===========
+* ``class``: An attribute by which the model classifies images. Only binary classification models are currently supported.
+* ``subgroup attribute``: An attribute by which the model *does not* classify images, but can be used to group samples into subgroups. 
+* ``decision region``: A portion of the decision space. The decision regions generated in this RST are the regions of the decision space near to a 'triplet' of sample images.
+* ``virtual image``: An image that was created by modifying existing image(s), rather than obtained through a typical image acquisition method.
+* ``vicinal distribution``: The collection of virtual images created by linearly interpolating between a 'triplet' of three images.
+
+
+Contents
+========
+.. toctree::
+   :maxdepth: 2
+
+   self
+   src
+   args
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/_sources/info.rst.txt

@@ -0,0 +1,30 @@
+DRAGen
+======
+Decsion Region Analysis for Generalizability (DRAGen) is a tool to analyze the decision space of an image classification model to increase understanding of the model's generalizability. A model's decision space maps a change in the input to a change in the model's output. DRAGen utilizes triplets of image samples to generate vicinal distributions of virtual images, created by linearly interpolating between the triplet images. These virtual images increase the density of available samples in the decision space, which allows for characterization of the decision space beyond the original finite data set. This insight into the decision space composition indicates how the model is likely to behave on data distributions upon which the model cannot generalize well.
+
+Getting Started
+===============
+Three inputs are required for this RST:
+
+1. **Trained Model:** the model must be saved in onnx format. Information on how to convert your model to onnx format can be found at the `onnx GitHub page`_.
+2. **Images:** Images can be saved in any format supported by PIL.
+3. **Input csv:** A csv file which can be used to map the image paths to subgroup attributes.
+
+Examples implementation can be found in the `examples folder`_.
+
+Scripts to generate and analyze decision regions can be found in the ``test`` folder, 
+all arguments used to run these scripts are located in ``src/args.py``.
+Example inputs are included in the ``examples`` folder.
+
+.. _onnx GitHub page: https://github.com/onnx/tutorials#converting-to-onnx-format
+.. _examples folder: https://github.com/DIDSR/RST_Decision_Region_Analysis/tree/main/examples
+
+Terminology
+===========
+* ``class``: An attribute by which the model classifies images. Only binary classification models are currently supported.
+* ``subgroup attribute``: An attribute by which the model *does not* classify images, but can be used to group samples into subgroups. 
+* ``decision region``: A portion of the decision space. The decision regions generated in this RST are the regions of the decision space near to a 'triplet' of sample images.
+* ``virtual image``: An image that was created by modifying existing image(s), rather than obtained through a typical image acquisition method.
+* ``vicinal distribution``: The collection of virtual images created by linearly interpolating between a 'triplet' of three images.
+
+

docs/_sources/src.rst.txt

@@ -0,0 +1,30 @@
+Data Input
+==========
+
+.. automodule:: src.data_input
+   :members:
+   :imported-members: 
+
+
+Decision Region Generation
+==========================
+
+.. autoclass:: src.decision_region_generation.triplet_manager.TripletManager
+   :members: 
+
+   .. automethod:: __getitem__
+
+.. automodule:: src.decision_region_generation.generate
+   :members:
+
+.. automodule:: src.decision_region_generation.vicinal_distribution
+   :members: plane_dataset, plane_dataloader, get_plane
+
+
+Composition Analysis
+====================
+
+.. automodule:: src.composition_analysis
+   :members:
+   :imported-members:
+