…ic into MAD

helper function for loading in images and making sure they're the
right shape

with tests

moves a bunch of code from Metamer to Synthesis. currently not much
change to those methods, but will do so as attempt to use the
abstraction for geodesics, eigendistortions, and MAD

…ic into MAD

…into abstractions

with this commit, we add an initial implementation of MAD
competition. it currently only minimizes model 1 while holding model 2
constant, and only works on models (not metrics), but the basic
framework is there. a fair amount of cleanup is required as well.

MAD competition (like Metamer) inherits the Synthesis class and so
makes use of much there. more functionality consolidation is needed
here, as there was a lot of copy-and-paste

 - Synthesis now uses the self.names dictionary to determine what to
   target / match etc. this doesn't change how Metamer works (or
   honestly, probably any of the other synthesis methods) but is
   necessary for MAD

 - adds a base gradient descent optimizer choice, GD

 - scheduler is now optional

 - optimizer_step's pbar arg is now optional. if None, will not update
   information. can also add additional information to the pbar as
   kwargs

originally, I was planning on adding an arg to objective_function to
turn normalizing on/off, but now we always do it if possible

with this, all targets are now supported, which requires an overhaul
of how we handle getting and setting the relevant attributes.

Synthesis goes back to knowing nothing about the attribute structure
of its children classes, assuming it can just directly grab
self.matched_representation (instead of going through self.names). it
is the responsibility of the child class to make sure this is the
correct attribute to modify, which we do here by modifying the getter
and setter (and continued use of a self._names attribute)

 - adds way more documentation to mad competition

 - moves representation_error to Synthesis

 - mad competition has own representation_error method, to handle the
   multiple synthesis targets / models

 - normalizing the loss is now an option, not always done

 - many MADCompetition attributes now have "all" variants, which store
   across targets

it's now in Synthesis, mad_competition has its own version (like
representation_error)

this moves the following functions from the Metamer class to
Synthesis, so it can be used by other synthesis methods:
plot_representation_error, plot_synthesis_status (was
plot_metamer_status), animate

other changes, to MADCompetition:

 - corrects name from saved_representation_gradient_{num} to
   saved_representation_{num}_gradient

 - now correctly handles _all attributes and copying information back
   and forth between them and the local version, does so with helper
   functions

 - nu, learning_rate, gradient now lists with _all versions that are
   dicts (like the other attributes)

 - adds _check_state() helper function which updates the synthesis
   target if necessary and returns the current state (used by the
   various plotting functions)

variety of updates necessary to make this happen:

 - mad competition has wrappers around many of the functions, which
   just make sure that the synthesis target is appropriately set. we
   make sure these have the args in the same order as the Synthesis
   class's version (with the new args tucked on the end)

 - mad_competition.representation_error's model arg can also be
   'both', which returns a dictionary containing both model's
   representation errors

 - correctly handle the copying attributes in *and out* of _all for
   tensors and lists

 - adds methods for MADCompetition.plot_synthesized_image,
   plot_synthesized_image_all (shows all synthesized images),
   plot_loss, plot_loss_all, plot_synthesis_status, and animate

 - in Synthesis, split out plot_synthesized_image and plot_loss from
   plot_synthesis_status

 - Synthesis.animate now takes two more args, plot_data_attr, a list
   which specifes the attributes to plot on the second subplot (so can
   be more than one; thus we grab all artists), and rep_error_kwargs,
   which specifies additional args to pass to the
   self.representation_error() call

 - display.rescale_ylim works with an array like or a dict. if a
   dictionary, we take the max over all of its values

for mad competition, can set whether you want to normalize the two
models' losses or keep them the same. unsure which is best right now

because it is likely to have positive and negative values

the MAD competition paper, as originally written, compared SSIM and
MSE, which are metrics, not models (they take two images and return a
scalar distance between them). as our code was written, it only worked
for models (which take an image and return a representation, and we
use the L2-norm of the difference between representations for the
distance between images).

with this commit, can now use metrics for MAD competition. to do that,
we construct a 'dummy model', that just returns a copy of the image
and modify the function called for our objective function

other changes

 - adds Identity() class in simulate/models/naive.py

 - adds MSE class to mse() function, since metrics should be functions
   now

 - we no longer do the whole randomizing of pixels before computing
   the loss in Synthesis._closure(). this really messes up those
   metrics that extract something meaningful from the image (i.e., all
   the ones we're interested in). so this is turned off by default,
   and only turned on when needed. right now, this is just when
   metamer has fraction_removed > 1 or loss_change_fraction <
   1. should try and generalize that tools

 - updates docstrings

 - adds loss_function attr, which is how we switch things around for
   the two models. objective_function calls this on x and y. (still
   use objective_function because it handles the normalizing)

 - adds _get_model_name, which first tries to grab the model's name
   attribute and, if it doesn't have one, then returns the class
   name. useful for the Identity class

 - raise a warning whenever the representation error is computed with
   a metric, since it will be meaningless

 - renames MSE to mse in import statement

 - replaces plot_metamer_status with plot_synthesis_status

realized there was more stuff that can be done by the superclass

two big changes, for Synthesis, MADCompetition, Metamer:

 - metric support added. this just moves the bits that handle this
   from mad_competition.py to Synthesis.py

 - can set loss_function to custom function. by default is L2-norm of
   difference

 - updates everyone's documentation and call signatures

this commit breaks the components of synthesize() into functions,
moves them to Synthesize abstract metaclass as possible, and thus
makes things more readable

as part of doing this, realized why matched_representation was
sometimes a Parameter after saving: because I was making it one when a
NaN was hit during loss. removed that and everything seems to still
work

still need to:

 - figure out how I want to handle coarse-to-fine and the randomizing
   stuff

 - standardize synthesize() call signature

accidentally had hard-coded (in _init_store_progress) that model_2 was
the stable model, by setting self.saved_representation =
list(self.saved_representation_2). fixes that, correctly sets
synthesis_target now, and adds some warning to docstring

several related commits, all aiming on finishing up the
standardization of synthesize():

 - adds _init_ctf_and_randomizer_() to Synthesis class (moving over
   the relevant part from the beginning of Metamer.synthesis()). this
   function sets up the coarse-to-fine optimization and possible
   randomizations (using some subset of representation when
   calculating gradients)

 - Synthesis._clamp_and_store() now returns a bool specifying if we
   stored or not this iteration.

 - Synthesis has new function, _checkfro_stabilization(), which checks
   whether loss has stabilized or not (and does so appropriately for
   coarse to fine optimization)

 - Synthesis.synthesize() now has args and documentation. it still
   raises an exception if you call it, but the idea is this provides a
   framework you can copy and you should keep the arguments in the
   order they are, putting new args in the beginning

 - MAD competition now works with coarse to fine. coarse_to_fine is
   one of those attributes that has values for each model. it is
   always false for the stable model, though it may be true for the
   target one. MAD competition has its own _init_ctf_and_randomizer
   which calls the super and sets the stable model's coarse_to_fine to
   False

 - bugfix: move MAD competition's first update_target within the
   synthesis loop

 - bugfix: MAD competition correctly updates other
   matched_representation when _check_nan_loss is True

 - standardizes synthesize() call signature for both

hadn't corrected it since the last update

to make it more clear they're global variables

this should enable it to handle GPUs successfully

actually the test files are not needed for test_mad

this requires allowing for the possibility of multiple models

includes file that tests all the important things (I think)

load call signature had changed

several changes here, all related to the merging of master into
abstractions on last commit

 - updates docstrings of Synthesis and Metamer to more accurately
   reflect current capabilities

 - changes Synthesis.synthesize() call signature and some of function
   for new abilities

 - MADCompetition now handles the new capabilities/changes correctly:
   better ability to resume synthesis, new synthesize() args
   clamp_each_iter and clip_grad_norm, default clamper now
   RangeClamper (instead of None)

 - adds tests for resuming synthesis (and the errors it should raise)
   for MADCompetition

gets it working

resume can only work if store_progress was not False

so I don't have to keep creating it

this new arg controls what to do if one of the synthesis targets has
been run before, with three choices

and adds tests

 - seed can now be None, in which case we don't set it. intended use
   case is to avoid resetting when resuming synthesis

 - store_progress can now be None in order to maintain setting from
   previous call (when resuming) and will raise an Exception if it
   changes between synthesis calls

 - updates documentation for optimizer to reflect full range of
   options

 - animation() now handles multiple calls to synthesize()

 - adds po.tools.data.convert_float_to_im to convert a float that lies
   between 0 and 1 to another dtype (intended: np.uint8 or np.uint16)

this might serve as our quick start page? demonstrates basics of how
to interact with Synthesis objects and the bare minimum of what a
model requries

Front_End had a problem with torch 1.5. previously, torch would
convert np.float64 to torch.float32; now, it converts them to
torch.float64. so need to explicitly set them to float32

also cleans up a bit

and those that are more relevant to it

attempts to be a tutorial for implementing a new synthesis method, but
having trouble with it

which will contain objective functions and the like

loss functions now work differently: they must all take at least the
keyword arguments synth_rep, ref_rep, synth_img, ref_img and return
some scalar. this allows us to compute loss with respect ot things in
the image, such as range or moments.

this required a fair amount of work

also:

 - adds tests

 - expands section of MADCompetition notebook explaining loss
   functions to include more details

previously, wasn't saving loss function and was not able to save if we
used a metric instead of a model because saving functions is not
supported by default pickle. we now use dill, which does support
this (added to setup.py as well)

additionally:

 - loss_kwargs renamed to loss_function_kwargs to be more consistent

 - save now has a model_attr_names arg just like load, so will
   properly save multiple models

 - copies load() to Metamer, makes call signature simpler

 - load() call signature simplified for MADCompetition, updates
   docstring to be more MAD-specific

 - adds better save/load tests for Metamer and MADCompetition

so we can figure out the weird error

to try and figure out why the one test is hanging

- the plotting in test_loss_func would cause a stall. it's not that
  important to test, so got rid of it

- also goes back to regular pytest command then

- removes pytest-timeout, because that wasn't actually helping

https://stackoverflow.com/a/35403128 suggests that I should enable
xvfb and close the open images, so let's try that.

with L1 and L2 norm, showing that it does the right thing, but that it
might not be easy -- in this case, because those two are hard to
separate

standard way of doing coarse-to-fine (ctf) optimization is *not* what
I had been doing before. before this commit, ctf computed the gradient
with respect to the target scale individually, ignoring the
others (not doing anything to hold them fixed). the standard way is to
compute the gradient with respect to the target scale *and all coarser
scales*. this commit allows you to toggle between those two choices,
by setting coarse_to_fine to either 'together' or 'separate'.

also updates MADCompetition.save to appropriately save the related
attributes, and makes the tests for metamer and MAD more complete

…lenoptic into abstractions

and notebook showing how it's done

since it's not a requirement and this is the only place we use it. in
the dataframe's place, we return a well-structured dictionary

adds docstrings for _gaussian (renamed from gaussian), create_windows,
ssim.

Updates SSIM to allow for weighting that matches the output from MAD
competition code

 - MSE now only averages across last two dimensions (height and width),
   as metrics should

 - SSIM:

   - removes unnecessary arguments (window and window_size no longer
     allowed, size_average because we always just average across last
     two dimensions)

   - Gaussian window is normalized in 2d, not 1d

   - Removes cs output

   - val_range renamed to dynamic_range and must always be explicitly
     set (None no longer allowed)

   - checks n_batches and handles them as you'd like

 - adds add_noise function, which adds noise such that user specifies
   the target MSE (approximately correct, not exact) and we can handle
   multiple noise values and images correctly (always with independent
   seeds and broadcasting as appropriate)

because the mapping between min/max and best/worst is not the same for
MSE and SSIM, this makes it clearer and sets the order for both to be
Best, Worst, matching the paper

need to average noise over the last two dimensions in order to get the
variances matched exactly

with this commit, we remove msssim (because there are some tricky things
that I don't have the time to figure out -- issue opened for it if
people want to check it out) and SSIM (caching window no longer makes
sense), and thus MSSSIM

also restructures ssim so that we have a helper function which computes
the ssim_map, contrast map, and weight. ssim just returns the mean over
the image. adds new function, ssim_map, which just returns the map.
contrast map will be used by msssim if we re-implement it

adds tests for the new SSIM functionality, both to make sure it works as
we'd like and that it matches MATLAB outputs. this involves downloading
some new files off the OSF, which means the downloading function is
updated to be a bit more general

also:

 - updates MAD notebooks

 - bugfix for add_noise to make sure the sizing works out as we'd like

makes sure to save all the _all attributes

…ic into abstractions

it doesn't actually make sense

renames:

 - matched_ -> synthesized_

 - _image -> _signal (except for plot_synthesized_image, which really is
   plotting a single image)

 - target_ -> base_