Learning System Widget learning

Obsolete documentation

The code explained below is no longer used (at least not much).

Widget learning

We have a learning system that can be though of as a system for learning to make and identify useful widgets, where widgets have certain properties which we desire. Thus, the set up is as follows.

• We have a (finite) probability distribution on widgets to start with.
• Widgets can be transformed and combined to make new ones.
• Widgets map to properties, on which we have a (typically infinitely supported) probability distribution given by weights.
• Each widget has a property (for the moment unique), but there are properties corresponding to which there are no widgets.
• Our goal is to find widgets, if they exists, with desired properties.
• Goals can be specified by giving a larger weight to associated properties.

What we learn

• Given a property as a goal and an initial set of widgets, we can generate new widgets and look for one with the given property.
• This process is better if we make transformations with appropriately chosen weights, and starting from an initial set of widgets containing many useful ones.
• We try to learn what are useful widgets and good weights.

The learning process.

• The evolution function gives a finite distribution on widget (element of FD[W]) given
  • an initial finite distribution
  • a finite distribution on transformations and combinations.
  • the number of steps.
• These give a function f from FD[W] to itself, which has a gradient (given in the code).
• There is also a function g from FD[W] to FD[P], the space of finite distributions on properties, induced by mapping widgets to their properties. This also has a gradient.
• Remark: this can be readily modified if we have multiple properties for a widget.
• The composition h of f and g gives a function FD[W] to FD[P]
• We have a feedback flow on FD[P], namely we move a finite distribution towards a given distribution on properties.
• The feedback flow on FD[P] conjugated by h gives a feedback flow on FD[W] (conjugation explained in the wiki page on differentiable functions).
• We evolve FD[W] by such a flow, along with some pruning etc.