Skip to content

Add roll up guide #178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Parent: Develop
merged 5 commits into from
Jan 29, 2024
Merged

Add roll up guide #178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6bc9673
Add roll up guide
brosenberg42 Jan 25, 2024
6b6dbac
Minor changes.
jrobble Jan 26, 2024
600409d
Add clarity.
jrobble Jan 26, 2024
6775694
Fix grammar
brosenberg42 Jan 29, 2024
3cd29f6
Merge branch 'develop' into document-rollup
brosenberg42 Jan 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/docs/Roll-Up-Guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
**NOTICE:** This software (or technical data) was produced for the U.S. Government under contract,
and is subject to the Rights in Data-General Clause 52.227-14, Alt. IV (DEC 2007). Copyright 2023
The MITRE Corporation. All Rights Reserved.


# Roll Up Overview

The Workflow Manager can be configured to replace the values of track and detection properties after
receiving tracks and detections from a component. This feature is commonly used to replace specific
terms with a more general category. For example, the "CLASSIFICATION" property may be set to "car",
"bus", and "truck". Those are all a kind of "vehicle". To use this feature, a JSON file in the
format described below must be created. Then, the `ROLL_UP_FILE` job property must be set to the
file path where that file is located.


# Roll Up File

The JSON below is an example of a roll up file.

```json
[
{
"propertyToProcess": "CLASSIFICATION",
"originalPropertyCopy": "ORIGINAL CLASSIFICATION",
"groups": [
{
"rollUp": "vehicle",
"members": [
"truck",
"car",
"bus"
]
},
{
"rollUp": "sandwich",
"members": [
"grilled cheese",
"reuben",
"hamburger",
"hot dog"
]
}
]
},
{
"propertyToProcess": "COLOR",
"groups": [
{
"rollUp": "purple",
"members": [
"indigo"
]
}
]
},
{
"propertyToProcess": "PROP3",
"groups": [
{
"rollUp": "new name",
"members": [
"old name"
]
}
]
}
]
```

At the top level, the roll up file contains an array where each element defines a detection property
that should be modified. In this example, there is one element for "CLASSIFICATION", one for
"COLOR", and one for "PROP3". Each element contains the following fields:

- `propertyToProcess`: (Required) A detection property key. The value will be modified according to
the `groups` key.
- `originalPropertyCopy`: (Optional) Copies the value of `propertyToProcess` prior to roll up to
another property. The copy is made even if the property is not modified.
- `groups`: (Optional) Array containing an element for each roll up name. If the value of the
detection property specified by `propertyToProcess` matches a string listed in `members`, it
will be replaced by the content of the `rollUp` property.

In the example above, the value of the "CLASSIFICATION" detection property will be copied to
"ORIGINAL CLASSIFICATION" before the roll up is performed. If the "CLASSIFICATION" detection
property is set to "truck", "car", or "bus", the value of the detection property will be replaced
by "vehicle".

In a real use case there will generally be multiple roll up groups for a single detection property.
The "sandwich" group shows how to include an additional mapping for the same "CLASSIFICATION"
property. The "COLOR" and "PROP3" sections show examples of how to apply roll up to different
detection properties with different configurations.

If the roll up above was applied to these detection properties:

```json
{
"CLASSIFICATION": "truck",
"COLOR": "red",
"PROP3": "truck",
"PROP4": "other"
}
```

it would result in:

```json
{
"CLASSIFICATION": "vehicle",
"ORIGINAL CLASSIFICATION": "truck",
"COLOR": "red",
"PROP3": "truck",
"PROP4": "other"
}
```

"COLOR" was not modified since it does not define a roll up group with "red" as a member. "PROP3"
was not modified because only the "CLASSIFICATION" property has a roll up group with "truck" as a
member. "PROP4" was not modified because it is not in the roll up file.
1 change: 1 addition & 0 deletions docs/mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ pages:
- Markup Guide: Markup-Guide.md
- TiesDb Guide: TiesDb-Guide.md
- Trigger Guide: Trigger-Guide.md
- Roll Up Guide: Roll-Up-Guide.md
- Health Check Guide: Health-Check-Guide.md
- REST API: REST-API.md
- Component Development:
Expand Down
4 changes: 4 additions & 0 deletions docs/site/404.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@
</li>
<li class="">

<a class="" href="/Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="/Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Acknowledgements/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Admin-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/CPP-Batch-Component-API/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/CPP-Streaming-Component-API/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Component-API-Overview/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Component-Descriptor-Reference/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Contributor-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Derivative-Media-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Development-Environment-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Feed-Forward-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/GPU-Support-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
8 changes: 6 additions & 2 deletions docs/site/Health-Check-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,10 @@
<li class="">

<a class="" href="../Trigger-Guide/index.html">Trigger Guide</a>
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class=" current">

Expand Down Expand Up @@ -299,7 +303,7 @@ <h1 id="the-ini-file">The INI File</h1>
<a href="../REST-API/index.html" class="btn btn-neutral float-right" title="REST API">Next <span class="icon icon-circle-arrow-right"></span></a>


<a href="../Trigger-Guide/index.html" class="btn btn-neutral" title="Trigger Guide"><span class="icon icon-circle-arrow-left"></span> Previous</a>
<a href="../Roll-Up-Guide/index.html" class="btn btn-neutral" title="Roll Up Guide"><span class="icon icon-circle-arrow-left"></span> Previous</a>

</div>

Expand Down Expand Up @@ -327,7 +331,7 @@ <h1 id="the-ini-file">The INI File</h1>
<span class="rst-current-version" data-toggle="rst-current-version">


<span><a href="../Trigger-Guide/index.html" style="color: #fcfcfc;">&laquo; Previous</a></span>
<span><a href="../Roll-Up-Guide/index.html" style="color: #fcfcfc;">&laquo; Previous</a></span>


<span style="margin-left: 15px"><a href="../REST-API/index.html" style="color: #fcfcfc">Next &raquo;</a></span>
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Install-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Java-Batch-Component-API/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/License-And-Distribution/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Markup-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Media-Segmentation-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Node-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Object-Storage-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/OpenID-Connect-Guide/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/Python-Batch-Component-API/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class="">
Expand Down
4 changes: 4 additions & 0 deletions docs/site/REST-API/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@
</li>
<li class="">

<a class="" href="../Roll-Up-Guide/index.html">Roll Up Guide</a>
</li>
<li class="">

<a class="" href="../Health-Check-Guide/index.html">Health Check Guide</a>
</li>
<li class=" current">
Expand Down
Loading