Python simulations (obs selection, reward functions etc.)

[maxpumperla]

No description provided.

[maxpumperla]

@kepricon @slinlee would be great if you could review. I had to rename the old package name to pathmind_training to make space for the pathmind simulation package. This makes the file changes a bit clumsy to view on GitHub. Might be best to check the branch out locally and run the new example from the README.

[slinlee]

This enables a lot of cool stuff and I like the changes to build in support for reward terms and obs selection.

This is a basic question, but how would an end-user get access to use Simulation interface?

And for now there's no built-in way to let a user use a trained policy with their python simulation, right? We're going to build that requests to policy server, yeah?

[maxpumperla]

@slinlee the reason pathmind (the new one) is now standalone is so that we can put it on PyPI and open source it, when ready. Then users can simply pip install pathmind and define their simulations. For that to work we need the web app integration ready, as pathmind should then also have an API to kick off a web app training run.

Also, you're right, we need to add glue code to make simulations work with policy server. Very good point!

[kepricon]

looks good to me.
will test it from my local and push integration code with webapp in this branch(if needed).

[slinlee]

@kepricon

will test it from my local and push integration code with webapp in this branch(if needed).

What level of integration are you planning for right now? I think the first priority will be to upload and train a simulation that includes observations and reward function already.

Later we can add the UI for 'editing the reward function'

PathmindAI/pathmind-webapp#2934

[kepricon]

yes, my plan is exactly the same with upload and train a simulation that includes observations and reward function

[maxpumperla]

@slinlee @kepricon good to go from my side now, once we get this into dev I'd like to make sure we get #262 (reward balancing) over the finish line for @brettskymind.

[kepricon]

looks like we need to set is_gym to TRUE or NOT and is_pathmind_simulation to TRUE or NOT from train.sh depends on webapp's training request.

I think ModelAnalyzer should check if the model is valid or not and what id the type of model.
fyi, the current model analyzer use test() from run.py to just check if it is valid or not from the exit code.

@maxpumperla
can you make test() method can return the type of model?
for example, gym_single, gym_multi, pathmind_single, pathmind_multi
I believe webapp can pass environment: str and model file to test().

[slinlee]

@kepricon feel free to merge and deploy when you're ready. Thanks, both!