Graph Update & Cleanup Tasks

[SouravPanda11]

• Friendly Health over time

  • Change legend → fu1, fu2, …, sum_fh

• Enemy Health over time

  • Change legend → eu1, eu2, …, sum_eh

• Health/Kill Reward over time

  • ➕ Add new graph showing rewards based on friendly/enemy HP changes
  • Legend:
    • 🟦 Blue: positive reward for enemy HP loss
    • 🟧 Orange: negative reward for friendly HP loss
    • 🟩 Green: overall (sum of above)
  • Center y-axis around 0

• Navigation Reward over time

  • Rename to Navigation Distance Reward over time
  • Change legend → fu1, fu2

• Combat Reward over time

  • Rename to Combat Distance Reward over time
  • Change legend → eu1, eu2

• Rewards over time / Rewards over time (decomposed)

  • Merge or delete duplicate → Overall Navigation vs Overall Combat
  • Plot shows terminal rewards:
    • Solid line for combat = distance + HP + kill + terminal
    • Solid line for navigation = distance + terminal
  • Center y-axis around 0
  • Adjust for large terminal reward scaling

• Predicted vs Actual

  • Rename to Actual vs Agent Estimate Reward over time
  • Legend → env reward, agent estimate
  • No unit-level data

• Units over time

  • Rename to Actions stacked over time
  • Y-axis → count of units
  • Legend → colors represent actions

• Per Unit Actions over time

  • Add legend → colors represent actions

• Environment reward vs value estimate

  • 🗑️ Delete this graph

• Clustering graphs

  • 🗑️ Delete all clustering-related graphs

• Update ingestion schema

  • Modify schema to support new graph structure (e.g., health_kill_reward, overall_nav_vs_combat)

[bnmajor]

@SouravPanda11 Thanks for the issue! Please see the Google Drive folder that I recently shared with you for updated images for all of the plots after these changes.

Some follow-up questions that I have:

1. For the fu1, fu2, ... and eu1, eu2, ... changes that you requested, are these literal strings that you would like to use? Or just indicating that we should add a friendly/enemy prefix where relevant? For now I just added friendly_ or enemy_.
2. I don't really understand what you are looking for in the remaining Units over time tasks and the rewards over time tasks.
   1. Are we not using data that we already have properly?
   2. Do we need to ingest new data altogether?
   3. Are these just styling requests?
3. I need specific information on the new data that we need to support ingesting.
   1. What new data fields do we need? What data types are we expecting?
   2. Are these new fields at the agent level or unit level or both?
   3. The health_kill_reward is for the new plot that you described above. What is overall_nav_vs_combat? A new plot?

[SouravPanda11]

@bnmajor, thanks for resolving the previous set of changes!
Here are a few additional clarifications and requests for consistency across plots

• fu1, fu2, etc. are not literal strings — they denote unit associations (friendly unit 1, 2, …).
• The fu, eu abbreviations are used instead of full words (“friendly”, “enemy”) to keep text minimal.

---

Friendly Health Over Time

• We will display only one episode at a time, meaning one agent at a time — so there’s no need to mention agent_0.
• The solid line represents the sum of health points across all friendly units.
• We want to avoid creating the misconception that the agent itself has health points; rather, the units do — this distinction keeps the concept clear.
• We also intentionally avoid referring to specific unit types (e.g., marine), and instead simply use the term unit to keep the explanation straightforward.

Changes requested:

• Replace friendly_agent_0 → FHP_sum
• Replace friendly_marine_X → fuX

---

Enemy health over time -

• Although the enemy is controlled by the built-in AI, we want to keep focus on our RL-AI only.
• Avoid using “agent” for the built-in AI to prevent confusion.

Changes requested:

• Replace enemy_agent_0_enemy_agent_0 → EHP_sum
• Replace enemy_agent_0_enemy_unit_X → euX

---

Combat Distance Reward over time

• Rewards apply only to friendly units.
• The team reward = mean of all unit-level combat distance rewards.

Changes requested:

• Replace enemy_marine_X → fuX (use dashed lines)
• Add a solid line for the team mean reward (Team)

---

Navigation Distance Reward over time

• The team reward = mean of all unit-level navigation distance rewards.

Changes requested:

• Replace friendly_marine_X → fuX (use dashed lines)
• Add a solid line for the team mean reward (Team)

---

Action stacked over time

• The y-axis = number of units performing each action (not individual unit IDs).
• Reference: “actions stacked” plot in the TwoBridgeMap repo.
• Which unit took which action is already shown in per-unit actions over time.

Changes requested:

• Change y-axis to unit count (max = 5)
• Ensure this uses the correct data reference (mostly styling; no new ingestion)

---

Combat Objective Components

• The combat objective = { distance, hp, terminal } components.
• HP-based reward is a new addition — previously not logged/plotted.
• Need to confirm whether adding this to CSV counts as a new data field or just an update.

---

Overall Combat vs Navigation Reward

• Both objectives are decomposed:
  • Combat = distance + hp + terminal
  • Navigation = distance + terminal
• This is not a new plot, but a refinement of the existing decomposed reward visualization.
• Refer to the TwoBridgeMap repo for the latest version.

Changes requested:

• Adjust y-axis scaling to handle large terminal rewards (so smaller variations remain visible)
• Clarify what episode_20_total represents

---

Summary
Most of these are naming, clarity, or visualization updates — not structural or ingestion-level changes (except for the new HP-based reward field, which needs discussion).

[bnmajor]

Thanks @SouravPanda11!

1. WRT naming: Great news is a lot of this can be achieved simply by setting the unit field to whatever you would like the unit name to be (in this case fu1, etc.). The same goes for the agent. My example JSON file just chooses marine_{n} and agent_{n} as defaults. This is not the case for enemy units though, so I will hard-code those accordingly.
2. Total instead of agent: One quick solution for replacing the agent in plots with the total would be to just name the agent Team in the JSON file. This would mean that ALL plots that list agent would instead say Team though. If that is not what you need then I can change the plot logic itself.
   1. I will make the appropriate changes internally to append the (sum) and (mean) suffixes to the appropriate plots.
3. New Data: I think that we can get around the need for updating the ingest step and the database by adding any missing plots to the custom_metrics field at the agent level. I just need to know:
   1. What new fields are you adding? What form do you intend to pass the data in as?
   2. Which fields do you want to associate with each new plot?
   3. What should the new plot(s) be named?
   4. Are there any specific requirements for how the new data is plotted? Or if you have an example PNG can you point me to the one that you would like me to use as a reference?
   5. EXAMPLE: So just to err on the side of too much detail rather than too little... If I was creating a new scatter line plot with time plotted along the x-axis and data foo along the y-axis:
      1. Just foo (time steps are already computed dynamically on ingest and don't need to be listed explicitly). foo will be a list of float values that map to the number of units that we have.
      2. Time steps + foo for the new plot
      3. Should be named Foo over time
      4. Dashed lines for units, solid lines for totals.
4. Got it, I will add that!

I am finalizing some of these updates right now but please check the Google Drive folder again in about 30 minutes to see the updated images for all of the plots.