Parametric action space

[ilyalasy]

So RLlib supports parametric action spaces:

Meaning of actions can be completely conditioned on the observation, i.e., the a in Q(s, a) becomes just a token in [0, MAX_AVAIL_ACTIONS) that only has meaning in the context of s.

I have exact that case and I tried to copy their implementation of the model from example and use it with DQN in machin framework (model predicts embedding vector then dot products with all action embeddings for current state), but it doesn't seem to work.
Any advice here?

[iffiX]

I just take a look at the RLlib example and documentation in your description, and this paper written by Masson et al which introduced parametric actions, from my point of view parametric action space could be understood as a higher dimensional action representation where available actions are embedded as sub-dimensions. So it should be compatible with the current DQN and other algorithm models without any additional framework modifications.

Could you please provide a more detailed description of "how it doesn't work" (eg: tables, visualizations, etc.) and your full implementation?

[ilyalasy]

Hi, it's been a while since this question and it seems reason for bad convergence of my algorithm was not because of usage of this approach, sorry.
P.S. Just as a suggestion: maybe it would be useful to add couple of examples of "unusual" cases in RL: parametric/hybrid actions, dynamic actions, etc.

[iffiX]

Certainly!