New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flesh out support for stochastic systems #6374

Closed
RussTedrake opened this Issue Jun 17, 2017 · 6 comments

Comments

Projects
None yet
3 participants
@RussTedrake
Contributor

RussTedrake commented Jun 17, 2017

Based on numerous face to face conversations, I am trying to collect the current design idea here.

There is an important distinction between two sources of uncertainty during the lifetime of a simulation
a) randomness in the dynamics/output methods (e.g. process and measurement noise).
b) random initial conditions (including random parameters).

Sources of randomness in group (a) must be treated with some care -- these methods get called by many different types of algorithms, including simulations that may make repeated calls to a dynamics method in order to perform error correction or zero-crossing detection. Analysis methods also need a way to easily determine any sources of randomness in the system. Therefore, we have the following rule:

All system dynamics/output methods (called during the lifetime of a simulation) must be strictly deterministic. Any randomness in the model should be captured by adding an input to the system and wiring that input up to a RandomSource.

Note that this is the classical approach to modeling stochastic systems in the robust control literature.

Sources of randomness in group (b) are treated slightly differently. We will implement a set of SetRandomContext() methods at the System level, which are allowed to populate the state and parameters of the system with values computed from a random number generator. In order to make the details of these distributions available to our best design and analysis tools, we will provide a small set of entry points to the STL random number generators through wrapper methods like "UniformRandomVector()" that always has the interval [0,1] and "GaussianRandomVector()" that always has unit covariances. Then we will add support to SymbolicExpression to overload those operators and add them as an ExpressionKind.

An algorithm that wishes to understand that a system happens to be linear-Gaussian, for instance, would discover that by symbolically parsing the SetRandomContext() and CalcTimeDerivatives() etc methods and inspecting the resulting SymbolicExpressions.

@RussTedrake

This comment has been minimized.

Show comment
Hide comment
@RussTedrake

RussTedrake Sep 3, 2017

Contributor

My specific next goal for this is to get the (Extended)KalmanFilter classes extracting covariance matrices automatically using symbolic interrogation of systems that have random sources sprinkled through the diagram (this requires #6959).

More advanced stochastic system tools should follow pretty quickly after that milestone!

Contributor

RussTedrake commented Sep 3, 2017

My specific next goal for this is to get the (Extended)KalmanFilter classes extracting covariance matrices automatically using symbolic interrogation of systems that have random sources sprinkled through the diagram (this requires #6959).

More advanced stochastic system tools should follow pretty quickly after that milestone!

@RussTedrake

This comment has been minimized.

Show comment
Hide comment
@RussTedrake

RussTedrake Sep 3, 2017

Contributor

A few more thoughts: for Diagram::ToAutoDiffXd, i could imagine having a routine that examines the diagram and replaces all RandomSource (of a specified type) blocks with input blocks (returning a list of pointers to the random inputs) so that algorithms can initialize those random values with autodiff as necessary.

Contributor

RussTedrake commented Sep 3, 2017

A few more thoughts: for Diagram::ToAutoDiffXd, i could imagine having a routine that examines the diagram and replaces all RandomSource (of a specified type) blocks with input blocks (returning a list of pointers to the random inputs) so that algorithms can initialize those random values with autodiff as necessary.

@RussTedrake

This comment has been minimized.

Show comment
Hide comment
@RussTedrake

RussTedrake Sep 7, 2017

Contributor

Based on a discussion with @naveenoid about my plans for stochastic systems in drake, it seems that it would be good for me to document my overall goal here. I think it can be stated most succinctly as this: a (leaf) system's dynamics have the form
ẋ(t) = f(x(t),u(t),w(t))
x[n+1] = f(x[n],u[n],w[n])
etc
where w(t) is implemented as a random "disturbance" input. My goal is to be able to think of every system in this form, even if it is a Diagram composed of many complex subsystems, and the state is now the vector concatenation of the state, etc.

We will also be able to set the random initial context, via the SetRandomContext() methods described above, and we can simulate the stochastic system by wiring the w input port up to a RandomSource system (which can be e.g. a uniform or gaussian distributed random source).

My current plan is to reuse the wiring up of a RandomSource (for simulation) as a tool for other analysis/design -- any inputs wired to RandomSource blocks are effectively labeled as the random disturbances, and tagged with uniform/gaussain, etc. So an algorithm that wants to e.g. formulate a search over the (random) inputs w, could systematically replace the RandomSource subsystems with FixedInput blocks, etc. And, as I discussed above, I think we can make ToSymbolic work even more automatically.

cc @sherm1, @jwnimmer-tri

Contributor

RussTedrake commented Sep 7, 2017

Based on a discussion with @naveenoid about my plans for stochastic systems in drake, it seems that it would be good for me to document my overall goal here. I think it can be stated most succinctly as this: a (leaf) system's dynamics have the form
ẋ(t) = f(x(t),u(t),w(t))
x[n+1] = f(x[n],u[n],w[n])
etc
where w(t) is implemented as a random "disturbance" input. My goal is to be able to think of every system in this form, even if it is a Diagram composed of many complex subsystems, and the state is now the vector concatenation of the state, etc.

We will also be able to set the random initial context, via the SetRandomContext() methods described above, and we can simulate the stochastic system by wiring the w input port up to a RandomSource system (which can be e.g. a uniform or gaussian distributed random source).

My current plan is to reuse the wiring up of a RandomSource (for simulation) as a tool for other analysis/design -- any inputs wired to RandomSource blocks are effectively labeled as the random disturbances, and tagged with uniform/gaussain, etc. So an algorithm that wants to e.g. formulate a search over the (random) inputs w, could systematically replace the RandomSource subsystems with FixedInput blocks, etc. And, as I discussed above, I think we can make ToSymbolic work even more automatically.

cc @sherm1, @jwnimmer-tri

@RussTedrake

This comment has been minimized.

Show comment
Hide comment
@RussTedrake

RussTedrake Sep 7, 2017

Contributor

An alternative design, which we discussed before but could discuss again, is to add additional labels to the InputPort to be able to explicitly label random inputs (and their types: uniform/gaussian). Then Diagrams could special case them and algorithms like simulator method could add RandomSource or FixedInput blocks to them automatically as appropriate.

Some things I like about this approach (which weren't all available last time we discussed):

  • inside DeclareRandomInputPort( ... Uniform .. ) we could immediately register the input constraints that e.g. 0 <= u <= 1 in a way that would stay with the system. (by contrast, declaring this as a constraint on the output of the randomsource block would not help if we e.g. pop that source and replace it with a FixedInputPort)
  • it becomes easier to simulate a simple stochastic system (no diagram/wiring required by the user)
  • i dislike the current API that requires users to pass in the generator to each individual RandomSystem (and they could/should mostly pass the same generator to all), but the api suggests that each RandomSystem can set it's own random seed... but they are in fact coupled).
Contributor

RussTedrake commented Sep 7, 2017

An alternative design, which we discussed before but could discuss again, is to add additional labels to the InputPort to be able to explicitly label random inputs (and their types: uniform/gaussian). Then Diagrams could special case them and algorithms like simulator method could add RandomSource or FixedInput blocks to them automatically as appropriate.

Some things I like about this approach (which weren't all available last time we discussed):

  • inside DeclareRandomInputPort( ... Uniform .. ) we could immediately register the input constraints that e.g. 0 <= u <= 1 in a way that would stay with the system. (by contrast, declaring this as a constraint on the output of the randomsource block would not help if we e.g. pop that source and replace it with a FixedInputPort)
  • it becomes easier to simulate a simple stochastic system (no diagram/wiring required by the user)
  • i dislike the current API that requires users to pass in the generator to each individual RandomSystem (and they could/should mostly pass the same generator to all), but the api suggests that each RandomSystem can set it's own random seed... but they are in fact coupled).
@sherm1

This comment has been minimized.

Show comment
Hide comment
@sherm1

sherm1 Sep 7, 2017

Member

One thing I prefer about making special random inputs w is that the code will better-match the math. Currently we can talk about the input ports u, state x, parameters p, and output ports y. That matches the math nicely. It would be awkward to have to describe how some of the input ports become w in the math if they are plugged into certain kinds of output ports.

Member

sherm1 commented Sep 7, 2017

One thing I prefer about making special random inputs w is that the code will better-match the math. Currently we can talk about the input ports u, state x, parameters p, and output ports y. That matches the math nicely. It would be awkward to have to describe how some of the input ports become w in the math if they are plugged into certain kinds of output ports.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 10, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 10, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 10, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 11, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 21, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 22, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.

RussTedrake added a commit to RussTedrake/drake that referenced this issue Sep 22, 2017

adds annotation to InputPortDescriptor for inputs that are anticipate…
…d as random noise/disturbances, as discussed in #6374.  pipes optional input through System, LeafSystem, and DiagramBuilder/Diagram, and adds basic unit test coverage.
@RussTedrake

This comment has been minimized.

Show comment
Hide comment
@RussTedrake

RussTedrake Oct 17, 2017

Contributor

stochastic support is in.

Contributor

RussTedrake commented Oct 17, 2017

stochastic support is in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment