Optimization interface

[josh146]

This issue reflects an interface change, away from a single Optimizer class which the user invokes with a keyword argument, towards something more akin to TensorFlow:

openqml.AdamOptimizer(args, kwargs, etc)

The Optimizer class would then wrap these individual methods/classes as a convenience to the user.

From @mariaschuld:

Nathan, Josh and I think that it would be best to define an optimizer as an object or method that defines to compute one step of updating the parameters, just like tensorflow does. We can then build the machine learning functionality around it in the next versions.

Issues:

1. Is there any overhead created by autograd that we would like to be shared between steps? Can we somehow store it in the optimizer class?
2. How can optimizers using past gradient information keep track of past steps?

Open questions:
Instead of relying on SciPy, shall we hand code the optimizers to reduce overheads? If yes, how do we deal with computational stability and such things?

[mariaschuld]

My idea is to do something like below, and I successfully tested autograd.grad to check if it can do this. This seems to solve part of the parameter issue by using weights and data as positional parameters and only deriving for weights. Or is there a reason we cannot do this with quantum nodes?

class MyGradbasedOptimizer:

    def _init_(self, cost, initial_weights ...):
        self.grad = autograd.grad(cost, 0) # Computes derivative for first argument, the weights
        self.past_gradient = 0 # Keeps track of gradients at past points, i.e. for momentum optimizer
        self.global_step = 0 # Keeps track of steps performed with this optimizer
        self.weights = initial_weights
        ...
    
    def step(self, current_weights, current_data):
        self.weights =  self.grad(current_weights, current_data)
        ....


def cost(weights, data):
    ''' weights and data can be any kind of (nested) np.array it seems...'''
    # Depends on quantum nodes and classical nodes...
    return scalar_cost
  

o = MyGradbasedOptimizer(cost, initial_weights,  ...)  
o.step(data)     
new_weights = o.weights() ```

[co9olguy]

Looks like it does all the basic things we want to do, while keeping things as simple as possible 👍 . I've asked @smite to take a crack at writing a couple slimmed-down optimizers based on the examples in autograd (which acts largely like these in terms of what they do)

[smite]

I separated the optimize_SGD method from the Optimizer class into a standalone function in 60c5756.

[josh146]

@smite I have a couple of questions regarding optimize_SGD.

1. Is it possible to slim it down even further? For instance, have a very basic function openqml.SGDOptimizer, that simply performs a single optimization step given a cost function and initial weights. The user could then use this to define their own optimizer class, or use our provided higher level openqml.Optimizer.

2. How come optimize_SGD now requires training data? Shouldn't this be able to be called without training data/classification data having to be provided?

[smite]

1. All nontrivial optimizers need to keep track of internal state variables that typically change at each iteration. In the simple SGD implementation we have now the state consists of the learning rate (which is a function of the current step number and some fixed parameters). More complex optimization algorithms like BFGS keep track of e.g. an approximate Hessian that is updated based on the gradient information over several iterations. The single step function would need to get all this state information as input arguments, and ideally also return every intermediate variable it computes (like the gradient) to the caller so the caller can update the state variables without having to recompute the intermediates. This doesn't seem very helpful, but maybe there's an use case I'm not thinking of?

1b. The current function can also be forced to run just one iteration by using optimize_SGD(max_steps=1).

2. As far as I understand SGD always takes a data set as input, otherwise it would be just normal gradient descent. The stochastic part is selecting a random subsample of the data set for each iteration, and passing that to the cost function.

[co9olguy]

Yep, the optimizers need to be classes, since they need to keep track of internal state. We want something only slightly more complex than they have in autograd (https://github.com/HIPS/autograd/blob/master/autograd/misc/optimizers.py). Those ones force you to either predict the full number of iterations you want from the start, or risk losing state between multilple calls.

No ML library that I have used has a SGD optimizer. They all have GD optimizers. It is up to the user to decide what to pass this. In future versions, we can build more complex classes, but for the initial release, our goal is to have the basic mathematical functions built-in with minimal further assumptions about what the user might want to do