[MXNET-262] Implement mx.random.seed_context to seed random number generators of a specific device context

[asitstands]

This PR is modified to add an optional argument ctx to mx.random.seed instead of introducing a new function seed_context. Seeding with ctx option produces random number sequence independent of device id. The below is the original description before the modification.

---

Description

This PR introduces a function mx.random.seed_context to seed random number generators of a specific device context. mx.random.seed_context(seed, ctx) seeds the parallel and non-parallel generators of ctx where ctx is optional and the default is the current context. The random number sequence generated on the device is completely determined by seed, differently from existing mx.random.seed which implicitly uses the device id of each context together with the given seed. Using device id is reasonable to seed all generators at once with a given seed, but to reproduce the same random number sequence we need to set the running context besides seeding with the same number. Sometimes setting a context would be inconvenient or the running context could be not deterministic. mx.random.seed_context would be helpful in such cases.

The implementation is simple. It just hands over the given seed to the underlying generators. The unit tests are an adaptation of the existing tests for mx.random.seed.

Here is an example.

# Seeding with `mx.random.seed`. Different results on gpu(0) and gpu(1).
>>> with mx.Context(mx.gpu(0)):
...     mx.random.seed(99)
...     print(mx.nd.random.uniform(0, 1, 3))
[0.29560053 0.07938761 0.29997164]
<NDArray 3 @gpu(0)>
>>> with mx.Context(mx.gpu(1)):
...     mx.random.seed(99)
...     print(mx.nd.random.uniform(0, 1, 3))
[0.8797334 0.8857584 0.3797555]
<NDArray 3 @gpu(1)>

# Seeding with `mx.random.seed_context`. Identical results on gpu(0) and gpu(1).
# This seeds the generator of the current context. Other generators are not touched.
# To seed a specific device context, set the optional argument `ctx`.
>>> with mx.Context(mx.gpu(0)):
...     mx.random.seed_context(99)
...     print(mx.nd.random.uniform(0, 1, 3))
[0.29560053 0.07938761 0.29997164]
<NDArray 3 @gpu(0)>
>>> with mx.Context(mx.gpu(1)):
...     mx.random.seed_context(99)
...     print(mx.nd.random.uniform(0, 1, 3))
[0.29560053 0.07938761 0.29997164]
<NDArray 3 @gpu(1)>

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage:
• Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
• Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
• Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
• Code is well-documented:
• For user-facing API changes, API doc string has been updated.
• For new C++ functions in header files, their functionalities and arguments are documented.
• For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
• Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
• To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

[piiswrong]

I would add a ctx=None argument to seed instead

[asitstands]

@piiswrong Omitting ctx argument usually means that it is the current context. So I think that adding ctx=None to seed could be potentially confusing.

[piiswrong]

I still think its better than adding an API. seed_context looks ugly

[piiswrong]

maybe you can make it defaults to 'all'

[asitstands]

I removed seed_context and added ctx="all" argument to seed.

[anirudh2290]

@piiswrong can this be merged. will add this to api changes in release notes.

[marcoabreu]

Thanks a lot for this change, this really something we've been looking for!

@asmushetzel @KellenSunderland