Skip to content

Commit

Permalink
add layer collection and metrics documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
tfjgeorge committed Aug 21, 2020
1 parent 048b73a commit 92ec660
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 3 deletions.
16 changes: 16 additions & 0 deletions docs/api/layercollection.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Layer collection
================

Layer collections describe the structure of parameters that will be differentiated. We need the LayerCollection object in order to be able to map components of different objects together. As an example, when performing a matrix-vector product using a block diagonal representation, we need to make sure that elements of the vector corresponding to parameters from layer 1 are multiplied with the diagonal block also corresponding to parameters from layer 1, and so on.

Typical use cases include:

- All parameters of your network: for this you can simply use the constructor :func:`nngeometry.layercollection.LayerCollection.from_model`
- Only parameters from some layers of your network. In this case you need to
1. instantiate a new LayerCollection object
2. add your layers one at a time using :func:`nngeometry.layercollection.LayerCollection.add_layer_from_model`


.. automodule:: nngeometry.layercollection
:members:
:undoc-members:
7 changes: 7 additions & 0 deletions docs/api/metrics.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Metrics
=======

Metrics introduction

.. automodule:: nngeometry.metrics
:members:
6 changes: 3 additions & 3 deletions nngeometry/metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ def FIM_MonteCarlo1(layer_collection,
representation,
variant='classif_logsoftmax'):
"""
Helper to create a matrix computing the Fisher Information
Matrix using a Monte-Carlo estimate with 1 sample per example
Helper that creates a matrix computing the Fisher Information
Matrix using a Monte-Carlo estimate of y|x with 1 sample per example
"""

if variant == 'classif_logsoftmax':
Expand Down Expand Up @@ -39,7 +39,7 @@ def FIM(layer_collection,
variant='classif_logits',
device='cpu'):
"""
Helper to create a matrix computing the Fisher Information
Helper that creates a matrix computing the Fisher Information
Matrix using closed form expressions for the expectation y|x
as described in (Pascanu and Bengio, 2013)
"""
Expand Down

0 comments on commit 92ec660

Please sign in to comment.