Extensibility to multi-agent settings

[drewstone]

Hello, I'm very new to this repo. Is this easily extensible to multi-agent settings? A bit about the problem I want to codify:

• N agents uniformly distributed on the real line, with N odd.
• Reward for agent $i$ at timestep $t$ is the magnitude of the intervals that are closest to $i$ and not $j$ forall $j\neq i$.
• If two agents share the same interval, they each either half the reward or I'll flip a coin with probability 1/2 (or 1/k for k sharing) to select the agent to reward.

If you're familiar, it's a hotelling model with no Nash equilibrium. Gonna dive into the repo but maybe you can point me to a good place to start! Thanks :)

[tspooner]

Hey, good to hear from you. I realise there's not a lot of documentation or diverse use case examples yet so I apologise for that.

The problem sounds interesting but none of the files in "examples" will be relevant. What I suggest is to avoid the SerialExperiment-type handlers I've included. These are for single-agent problems only at the moment. Instead, you'll need to code up a Rust version of your model and get it working without any RL code. Then, once you can pass a Vec (for example) of actions and return a commensurate Vec of rewards, then start creating instances of the algos I've implemented. You will need to do all of the "for i in episode { for step in steps {"-type code yourself for now I'm afraid.

I'll keep this issue open so you can ask any further Qs and I'll do my best to help!

Regards,
Tom

N.B.
As I've mentioned on other threads, this is still under active development. In fact, I have some major changes that I am finishing off that I will push up to the repo soon. One of the things I will focus on next is "infrastructure" code. I.E. automating runs for both single- and multi-agent cases. The framework actually does support this, but it's all a bit hacky at the moment as mentioned - I recently completed some work using RSRL for a two-player zero-sum game project; simple but still multi-agent I guess!

[drewstone]

Awesome, thanks for the info @tspooner !