How should node/process seeds interact?

[tbekolay]

Currently, we have a few interacting mechanisms for seeding. In most cases, it's pretty obvious what should happen, and that is what happens. Here's a brief explanation of what I think happens now:

Builder seed hierarchy

1. If you set a seed for an object (specifically Ensemble, Connection and Probe) then that seed will be used for any random operation (which sometimes doesn't happen for Probe).
2. If you don't set a seed, the seed for the containing network is used.
3. If the containing network doesn't have a seed, it will be assigned one from its containing network.

This hierarchy ensures that the build process is deterministic if a top-level seed is set. Because seeds are assigned at the network level, changes within one subnetwork should not affect other subnetworks.

Simulator seed hierarchy

1. If you set a seed for a Process, then that seed will be used.
2. If you don't set a seed, the seed for the Simulator is used.

The question that came up in #958 is how should this hierarchy change when a Node seed is set. Since nodes can contain processes, it makes sense to affect the seed of the Process, but since the seed of the process is managed by the simulator, it's not a straightforward change.

So, the first issue is whether we should keep two separate seeds. The main reason for the simulator seed was for random processes; if it's easier to manage things with one seed hierarchy then I think we should go that way.

Then the second issue is how the Process seed hierarchy should go. Processes can be in both Ensemble.noise and Node at the moment. Should it be Process > Node/Ensemble > Network?

[jgosmann]

To me it seems conceptually easier to have only one hierarchy. Otherwise we have to be really careful about documenting which seed influences what and it is not necessarily obvious. I assume having only one hierarchy makes some use cases a bit more tedious to program, but I am not sure how common those use cases are.

[hunse]

For me, having one hierarchy is a good end-goal. But as part of that system, I would also like to be able to rebuild nodes without rebuilding the model, something that we've talked about for a long time and have a PR for, but has never actually been finalized.

I think it will be a bit unfortunate that under a one-seed system, it will be impossible to change the random number generation for non-node processes (like those injecting noise into an ensemble) without rebuilding the whole model. But as @jgosmann said, I feel like that use case isn't common enough to build around, and when people do want to do that it's probably with small models where rebuilding the model isn't very costly anyway.

Though one thing I just thought of: under a one-seed system, it's difficult to change the model noise without changing the model parameters (gains, biases, etc.), and vice versa. Even if you're rebuilding the model, it's annoying to have to somehow search through, find all random processes, and change the seed, if you just want to change the randomness and keep the same parameters. So maybe I'm less okay with a one-seed system than I thought.

[hunse]

Though looking at this for nengo_ocl, it's actually kind of annoying to have to support changing the simulation seed on reset. It's also going to be a bit of work to support seeds for processes, since I'll have to make individual RNGs rather than one central one, but that's not as bad.

So I think one seed is better. But can anyone think of a way to easily change it for all processes, if I want to change the noise in my model but keep the parameters the same when I rebuild? Maybe we'll need to add a helper function or something for this.

[tbekolay]

Hm, one possibility would be to have a global process seed that is combined with a node/process seed if one is set. That might be just reimplementing the simulator seed though, so maybe a helper function would be better. nengo.utils.processes.seed_processes(network, seed)?

[jgosmann]

Idea from the dev meeting: If not given, set the simulator seed based on the network seed. Anyone seeing any problems with that?

[bjkomer]

I like that solution, as long as there is still an option to have the network seeded but not the simulator (by explicitly giving the simulator a seed of None, or something along those lines).

[jgosmann]

You could seed the simulator with a random number (from an unseeded RNG) to have that effect.