How do I create a custom neural network in TensorFlow and run as an external brain?

[RyanRizzo96]

Hi everyone,

I have completed all the necessary steps to install ML-Agents on Windows 10 following this excellent tutorial:

https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation-Windows.md 👍

Next I followed this comprehensive guide to get started with the balance ball project. All has worked well.

Now, I want to go a step further and create not just my own new scenes and environments but also create the neural networks in Python. Since v 0.3 has done away with the PPO.ipynb file which I could edit and run the code in Jupyter notebook, I am stuck as to where I can actually code my custom neural network using Python and TensorFlow to use this as an external brain to my agent.

1. 1. Is it possible to create a custom neural network using Python and TensorFlow and somehow attach this as an external brain to my agent?
2. 2. Which file should I edit to start coding my custom neural network?
3. Is this necessary from my side to build a neural network from scratch, or does unity ML-Agents provide all the coverage needed?
4. Is it possible to create a custom neural network using Python and TensorFlow and somehow attach this as an external brain to my agent?
5. Which file should I edit to start coding my custom neural network?
6. Is this necessary from my side to build a neural network from scratch, or does unity ML-Agents provide all the coverage needed?

To give you a broader view of what I am trying to accomplish. I have a small robotic system modeled in unity to include friction coefficients and servo motor PID tuning to mimic a real-life robotic system. I now want to train this system using TensorFlow and Python by coding up a recurrent neural network. I was wondering whether this would be the way to go, since machine learning in Python and TensorFlow is 'easier' than in C#.

I cam across the Training ML-Agents documentation, but I am not talking about simply editing the config file training_config.yaml, but rather building a neural network from scratch.

In short I am looking to alter the Python API.

I could not seem to find any information regarding the above two points and have been stuck trying to figure this out. Any help would be greatly appreciated.

Thanks!

[joaquines]

yes you can. all the python code that creates the TF layers is included. you do not need to touch it for it to work, but if might be a good place to start if you want a custom topology.
Take a look at https://github.com/Unity-Technologies/ml-agents/blob/master/python/unitytrainers/models.py#L162

[mmattar]

Hi @joaquines - are you still facing any issues?

[xiaomaogy]

Thanks for reaching out to us. Hopefully you were able to resolve your issue. We are closing this due to inactivity, but if you need additional assistance, feel free to reopen the issue.

[UgurUysal86]

Is it possible to use keras-rl in combination with ML-agents? I would like to create custom networks. Thank you.

[xiaomaogy]

@UgurUysal86 You can of course write your own trainer code and not use our trainer code at all.

[UgurUysal86]

Thank you! Is there any guide on how to create your own trainer?

[xiaomaogy]

Emm....@UgurUysal86 if you don't know how to do that, then a guide won't help you much.

[UgurUysal86]

Ouch. Thank you anyway.

[MarkTension]

yes you can. all the python code that creates the TF layers is included. you do not need to touch it for it to work, but if might be a good place to start if you want a custom topology.
Take a look at https://github.com/Unity-Technologies/ml-agents/blob/master/python/unitytrainers/models.py#L162

This link would be helpful to me, but is broken, any new address to get here? Thanks!

[preston-willis]

yes you can. all the python code that creates the TF layers is included. you do not need to touch it for it to work, but if might be a good place to start if you want a custom topology.
Take a look at https://github.com/Unity-Technologies/ml-agents/blob/master/python/unitytrainers/models.py#L162

This link would be helpful to me, but is broken, any new address to get here? Thanks!

Here you go: https://github.com/Unity-Technologies/ml-agents/blob/master/ml-agents/mlagents/trainers/models.py

Please tell me if you figure anything out. I'm trying to do the same thing.

[MarkTension]

Thanks! Will get back at you when I get it working for me

[preston-willis]

Thanks! Will get back at you when I get it working for me

I just got it working. Since there isn't any documentation for the latest version, I downgraded to unity 2018.1.0. From there, I just followed this medium article and created my own academy, brain, and agent. I also created my own script to interface with the environment python/genetic_algorithm.py since I don't want to use RL.

Here is all my code:
https://github.com/iamPres/Bipedal-Walker

Hope this helps

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.