Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to Edition 3 #17

Merged
merged 3 commits into from
Mar 18, 2019
Merged

Update to Edition 3 #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
256044e
fix calc of expected reward
icoxfog417 Jan 20, 2019
dd0f042
fix calc of expected reward of IRL
icoxfog417 Jan 20, 2019
470d186
Merge pull request #7 from icoxfog417/issue_6
icoxfog417 Jan 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions DP/planner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,9 @@ def estimate_by_policy(self, gamma, threshold):
r += action_prob * prob * \
(reward + gamma * V[next_state])
expected_rewards.append(r)
max_reward = max(expected_rewards)
delta = max(delta, abs(max_reward - V[s]))
V[s] = max_reward
value = sum(expected_rewards)
delta = max(delta, abs(value - V[s]))
V[s] = value
if delta < threshold:
break

Expand Down
8 changes: 4 additions & 4 deletions IRL/planner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def initialize(self):
self.policy = np.ones((self.env.observation_space.n,
self.env.action_space.n))
# First, take each action uniformly.
self.polidy = self.policy / self.env.action_space.n
self.policy = self.policy / self.env.action_space.n

def policy_to_q(self, V, gamma):
Q = np.zeros((self.env.observation_space.n,
Expand Down Expand Up @@ -108,9 +108,9 @@ def estimate_by_policy(self, gamma, threshold):
reward += action_prob * p * \
(r + gamma * V[n_s] * (not done))
expected_rewards.append(reward)
max_reward = max(expected_rewards)
delta = max(delta, abs(max_reward - V[s]))
V[s] = max_reward
value = sum(expected_rewards)
delta = max(delta, abs(value - V[s]))
V[s] = value

if delta < threshold or count > self._limit_count:
break
Expand Down