Skip to content

Commit

Permalink
Updated contributing information
Browse files Browse the repository at this point in the history
  • Loading branch information
Alexander Kuhnle committed Jan 29, 2018
1 parent b8a3165 commit f1c49de
Show file tree
Hide file tree
Showing 2 changed files with 102 additions and 50 deletions.
138 changes: 93 additions & 45 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,26 @@
Contribution guide
------------------

Below are some pointers for new contributions. In general, it is probably always a good
idea to join the community to discuss a contribution, unless it's a smaller bug fix. You
can join the community by filling in [this short form](https://docs.google.com/forms/d/1_UD5Pb5LaPVUviD0pO0fFcEnx_vwenvuc00jmP2rRIc/)
Below are some pointers for new contributions. In general, it is probably always a good idea to
join the community to discuss a contribution, unless it's a smaller bug fix. You can join the
community by filling in
[this short form](https://docs.google.com/forms/d/1_UD5Pb5LaPVUviD0pO0fFcEnx_vwenvuc00jmP2rRIc/)
which will take you to the chat after.

### 1. Code style

In general, we try to follow the [Google Python style guide](https://google.github.io/styleguide/pyguide.html) with
a few particulars. Another good rule of thumb is that if something is a pep8 warning in your editor, it is
probably worth looking at.

Some things to pay attention to:

- Lines should have max length of 120 characters, better 100
- Comments, even line comments, should be capitalised
- We prefer line comments to be above the line of code they are commenting on
for shorter lines:
In general, we try to follow the
[Google Python style guide](https://google.github.io/styleguide/pyguide.html) with a few
particulars. Another good rule of thumb is that if something is a PEP8 warning in your editor, it
is probably worth looking at.

```python
# This is a line comment.
input_values = dict()
```
Some things to pay attention to:

Instead of:
- Lines should have a max length of 120 characters, 100 for documentation comments.

```python
input_values = dict() # this is a non-capitalised line comment making the line unnecessarily long
```
- When initializing objects such as dictionaries or lists and there are multiple entries, use the
following format:

When initialising objects such as dictionaries or lists and there are multiple entries,
use the following format:
```python
# One key-value pair per line, one indent level.
dict(
Expand All @@ -43,8 +32,8 @@ dict(
)
```

When calling TensorFlow functions, we prefer to use named arguments
for readability:
- When calling TensorFlow functions, use named arguments for readability wherever possible:

```python
scaffold = tf.train.Scaffold(
init_op=init_op,
Expand All @@ -57,25 +46,84 @@ scaffold = tf.train.Scaffold(
saver=saver,
copy_from_scaffold=None
)
```
```

- Indentations should always be tab-spaced (tab size: 4), instead of based on alignments to the previous line:

```python
states_preprocessing_variables = [
variable for name in self.states_preprocessing.keys()
for variable in self.states_preprocessing[name].get_variables()
]
```

instead of:

```python
states_preprocessing_variables = [variable for name in self.states_preprocessing.keys()
for variable in self.states_preprocessing[name].get_variables()]
```

or:

```python
kwargs['fn_loss'] = (lambda: self.fn_loss(
states=states,
internals=internals,
actions=actions,
terminal=terminal,
reward=reward,
update=update
))
```

instead of:

```python
kwargs['fn_loss'] = (
lambda: self.fn_loss(states=states, internals=internals, actions=actions,
terminal=terminal, reward=reward, update=update)
)
```

- Binary operators should always be surrounded by a single space: `z = x + y` instead of `z=x+y`.

- Lines (apart from documentation comments), including empty lines, should never contain trailing
spaces.

- Comments, even line comments, should be capitalised.

- We prefer line comments to be above the line of code they are commenting on for shorter lines:

```python
# This is a line comment.
input_values = dict()
```

instead of:

```python
input_values = dict() # this is a non-capitalised line comment making the line unnecessarily long
```


### 2. Architecture

New contributions should integrate into the existing design ideas. To this end,
reading our [blog](https://reinforce.io/blog/)) can be very helpful. The key
design elements to understand are the optimizer package (as described in the blog),
the idea to move all reinforcement learning control flow into the TensorFlow graph,
and the general object hierarchy of models. Again, for detailed questions do join
the chat.
New contributions should integrate into the existing design ideas. To this end, reading our
[blog](https://reinforce.io/blog/)) can be very helpful. The key design elements to understand are
the optimizer package (as described in the blog), the idea to move all reinforcement learning
control flow into the TensorFlow graph, and the general object hierarchy of models. Again, for
detailed questions do join the chat.


### 3. Areas of contribution

Below are some potential areas of contribution. Feel free to make new
suggestions on your own.
Below are some potential areas of contribution. Feel free to make new suggestions on your own.

Environments:

TensorForce provides a generic [enviroment class](https://github.com/reinforceio/tensorforce/blob/master/tensorforce/environments/environment.py).
TensorForce provides a generic
[enviroment class](https://github.com/reinforceio/tensorforce/blob/master/tensorforce/environments/environment.py).
Applications do not need to implement this but it provides the advantage of using tne ```Runner```
execution infrastructure. A number of implemented environments can be found in the contrib folder.
Implementing a binding for a new environment is a great way to better understand the agent API and
Expand All @@ -96,14 +144,14 @@ makes for a good first contribution. Below is a list of environments one might l

Models:

Reinforcement learning is a highly active field of research and new models are appearing with
high frequency. Our main development focus is on providing abstractions and architecture, so model
contributions are very welcome. Note that integrating some models may require some discussion
on interfacing the existing models, especially in the case of newer architectures with complex internal
models. Some model suggestions:
Reinforcement learning is a highly active field of research and new models are appearing with high
frequency. Our main development focus is on providing abstractions and architecture, so model
contributions are very welcome. Note that integrating some models may require some discussion on
interfacing the existing models, especially in the case of newer architectures with complex
internal models. Some model suggestions:

- ACER - [paper](https://arxiv.org/abs/1611.01224)
- Direct future prediction (n.b. this will require architecture changes)- [paper](https://arxiv.org/abs/1611.01779)
- Direct future prediction (n.b. this will require architecture changes) - [paper](https://arxiv.org/abs/1611.01779)
- Categorical DQN reimplementation. A categorical DQN implementation was part of 0.2.2 but was removed
because it did not easily integrate into the optimizer architecture. If you are interested in this model,
please comment in the issue or join the chat for discussion.
Expand All @@ -112,7 +160,7 @@ models. Some model suggestions:

Ecosystem integrations:

If you are interested in general usability, another area of contribution is integrations into the wider
machine learning and data processing ecosystem. For example, providing scripts to run
TensorForce on one of a number of cloud service providers, or to run jobs on data infrastructure frameworks
like Kubernetes, Spark, etc is a great way to make RL more accessible.
If you are interested in general usability, another area of contribution is integrations into the
wider machine learning and data processing ecosystem. For example, providing scripts to run
TensorForce on one of a number of cloud service providers, or to run jobs on data infrastructure
frameworks like Kubernetes, Spark, etc is a great way to make RL more accessible.
14 changes: 9 additions & 5 deletions tensorforce/models/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -759,7 +759,7 @@ def initialize(self, custom_getter):
self.summary_configuration_op = None
if self.summary_spec and 'meta_param_recorder_class' in self.summary_spec:
self.summary_configuration_op = self.summary_spec['meta_param_recorder_class'].build_metagraph_list()

# self.fn_summarization = tf.make_template(
# name_='summarization',
# func_=self.tf_summarization,
Expand Down Expand Up @@ -1048,10 +1048,14 @@ def get_optimizer_kwargs(self, states, internals, actions, terminal, reward, upd
kwargs = dict()
kwargs['time'] = self.timestep
kwargs['variables'] = self.get_variables()
kwargs['fn_loss'] = (
lambda: self.fn_loss(states=states, internals=internals, actions=actions,
terminal=terminal, reward=reward, update=update)
)
kwargs['fn_loss'] = (lambda: self.fn_loss(
states=states,
internals=internals,
actions=actions,
terminal=terminal,
reward=reward,
update=update
))
if self.global_model is not None:
kwargs['global_variables'] = self.global_model.get_variables()
return kwargs
Expand Down

0 comments on commit f1c49de

Please sign in to comment.