[Question] Customizing my algorithem

[fu-li-ck]

Required prerequisites

• I have read the documentation https://omnisafe.readthedocs.io.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Questions

As a beginner in safety RL, I'm very grateful to omnisafe'help for me.

I would like to implement customised algorithm based on algorithms in omnisafe. For example:

`
from omnisafe.algorithms.on_policy.base.trpo import TRPO
class Cust_algo(TRPO):
......

env_id = 'SafetyPointGoal0-v0'
agent = omnisafe.Agent('Cust_algo', env_id)
agent.learn()
`

What should I do to make it works? Or is there any other method making the same target achieved?

Once again, thank you for your efforts, and I look forward to your response. Thank you!

[Gaiejj]

Thank you very much for your recognition. OmniSafe provides support for flexible algorithm customization. You can achieve algorithm customization by simple inheritance and overriding.

Suppose you need to implement a CustomTRPO based on TRPO. You can follow these steps:

• Create a new file named custom_trpo.py in the directory of your choice (in this example, we'll choose omnisafe/algorithms/on_policy/base).
• Implement the customization for TRPO in the created file.
  e.g.

# Copyright 2023 OmniSafe Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Implementation of the customization of the TRPO algorithm."""

from omnisafe.algorithms import registry
from omnisafe.algorithms.on_policy.base.trpo import TRPO


@registry.register
class CustomTRPO(TRPO):
    """The customization of the TRPO algorithm."""
    
    # You can customize the algorithm by overriding the methods in TRPO
    # or adding new methods.

• Register the custom algorithm in the __init__.py file located in the omnisafe/algorithms/on_policy/base directory.

# Copyright 2023 OmniSafe Team. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Basic Reinforcement Learning algorithms."""

from omnisafe.algorithms.on_policy.base.natural_pg import NaturalPG
from omnisafe.algorithms.on_policy.base.policy_gradient import PolicyGradient
from omnisafe.algorithms.on_policy.base.ppo import PPO
from omnisafe.algorithms.on_policy.base.trpo import TRPO
from omnisafe.algorithms.on_policy.base.custom_trpo import CustomTRPO


__all__ = [
    'NaturalPG',
    'PolicyGradient',
    'PPO',
    'TRPO',
    'CustomTRPO'
]

• Create a CustomTRPO.yaml file in omnisafe/configs/on-policy directory (note that the file name should match the name of your customized algorithm class). Write appropriate parameters in the YAML file. You can copy the contents from `TRPO.yaml`` and modify them as needed.
• Finally, you can run your CustomTRPO algorithm in a similar way to the already implemented algorithms in OmniSafe. For example, you can use the command:

cd examples
python train_policy.py --algo CustomTRPO.

By following these steps, you will be able to customize TRPO and run your CustomTRPO algorithm within OmniSafe.

[fu-li-ck]

Thank you for your reply, it really solve my question.

[Gaiejj]

Feel free to open an issue whenever you encounter problem.