Skip to content

Improve Boids Reward#246

Closed
PLAZMAMA wants to merge 39 commits intoPufferAI:3.0from
PLAZMAMA:improve_boids
Closed

Improve Boids Reward#246
PLAZMAMA wants to merge 39 commits intoPufferAI:3.0from
PLAZMAMA:improve_boids

Conversation

@PLAZMAMA
Copy link
Copy Markdown
Contributor

@PLAZMAMA PLAZMAMA commented Jun 5, 2025 •
edited
Loading

Description

This PRs goal is to improve the reward calculation of the boids env and train a policy on it

Todo

  • Improve reward calculations

    • margin factor
    • separation factor
    • cohesion factor
    • alignment factor

  • Train policy on two factors

    • Margin and seperation
    • Cohesion and factor

  • Train policy successfully on all factors

@PLAZMAMA PLAZMAMA marked this pull request as draft June 5, 2025 07:58
PLAZMAMA added 27 commits July 27, 2025 16:32
@PLAZMAMA
changed policy and rename factors to match common names
0734d89
@PLAZMAMA
remove unused log fields
e94a4aa
@PLAZMAMA
remove unused variable
817dc14
@PLAZMAMA
remove unused commented code
e232d3e
@PLAZMAMA
remove unused boid_logs and fix logs calculation
4eb410a
@PLAZMAMA
fix overflow and zero report_interval
1d424d7
@PLAZMAMA
add above zero checks for num_boids and report_interval
7376334
@PLAZMAMA
remove unused commented flat_actions
ff483f6
@PLAZMAMA
simplify seperation reward and test it
26bebef
@PLAZMAMA
test out only avoid factor
06878eb
@PLAZMAMA
remove unused avg_reward and change seperation factor reward
35de375
@PLAZMAMA
fix factor names
9709a46
@PLAZMAMA
remove unused commented code
1c28c72
@PLAZMAMA
fix seperation factor reward calculation
bf8c75f
@PLAZMAMA
remove unused commented params
23e2399
@PLAZMAMA
remove normalization from separation factor calculation
bb162fb
@PLAZMAMA
fix visual range
85d5891
@PLAZMAMA
remove positve margin rewards and remove commented code
435ac9e
@PLAZMAMA
add factors to env run with "boids.c"
463f60a
@PLAZMAMA
add debug margin lines and adjust reward normalization
00af443
@PLAZMAMA
only turn on margin turn factor and adjust total timesteps
fc4e722
@PLAZMAMA
change top/bottom margins
a413221
@PLAZMAMA
account for boid width and hight in margin reward calculation
342d83c
@PLAZMAMA
increase max steps
01a84c0
@PLAZMAMA
remove debug margin lines
117e9b6
@PLAZMAMA
fix observations for margin factor
618cb0b
@PLAZMAMA
remove single agent params
7526366
@PLAZMAMA
Copy link
Copy Markdown
Contributor Author

PLAZMAMA commented Oct 1, 2025

Hi @jsuarez5341, sorry for being out for the last 3 months, I got a new job and stuff.

However, I was able to improve the policy and solve Boids successfully with all of the rewards.
All that's left is to do a sweep which I don't have the ability to do right now.

P.S. When I get situated in my new job(couple weeks hopefully) I'll come back to PufferLib and contribute in my free time.
I look forward to working on cool stuff together again soon!

@PLAZMAMA PLAZMAMA marked this pull request as ready for review March 2, 2026 06:22
@PLAZMAMA
Copy link
Copy Markdown
Contributor Author

PLAZMAMA commented Mar 2, 2026

@jsuarez5341 I see that this PR has not been look at( Probably because I forgot to leave it as a draft). But I think it will need to be rebased to Puffer 4.0. If you would like me to do that, then let me know, BOIDS is not relevant longer and isn't worth working on anymore please let me know as well.

@PLAZMAMA PLAZMAMA mentioned this pull request Apr 10, 2026
Draft
@PLAZMAMA
Copy link
Copy Markdown
Contributor Author

PLAZMAMA commented Apr 10, 2026

Closing PR as I just made a new PR migrating the improvements of this PR to 4.0

@PLAZMAMA PLAZMAMA closed this Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@PLAZMAMA