[DO NOT MERGE] New memory abstraction and AMRL implementation

[ervteng]

Proposed change(s)

New memory layer abstraction that lets a memory define it's memory size (w.r.t. the "memory size" specified in the config). Then the ActorCritic has a memory_size property which the Policy uses to set it's internal m_size. This allows the actual m_size that is used during training to be different than what is specified in the YAML. We might want to do this, for instance, if we want memory_size: 64 to mean the Policy has a memory size of 64, but the Critic network has memory size of 64 (total of 128) - so the Policy object can store both during training, but only export 64 for inference.

Also an implementation of https://openreview.net/forum?id=Bkl7bREtDr using this abstraction.

Types of change(s)

• Bug fix
• New feature
• Code refactor
• Breaking change
• Documentation update
• Other (please describe)

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)
• Updated the migration guide (if applicable)

Other comments

[ervteng]

Outdated - develop-amrl has the latest experimental code.