Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tao 3 #627

Merged
merged 3 commits into from
Sep 28, 2023
Merged

Tao 3 #627

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0739b5b
dataset updates for TAO 3
Sep 27, 2023
ff50c72
fix calc field order
Sep 27, 2023
18e0e54
added changelog
Sep 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions changes/CHANGELOG-tao.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
# What's new in TAO Dashboard

## TAO Dashboard v3.0.0:
**Important:** Update to this version requires cid-cmd v0.2.26. Please update cid-cmd first before updating the dashboard. During the update you'd need to provide a path to s3 folder where your Trusted Advisor data is stored. QuickSight dataset ta-organizational-view will be updated, please make a copy if you've made any customizations to the dataset. To update run these commands in your CloudShell (recommended) or other terminal:

```
python3 -m ensurepip --upgrade
pip3 install --upgrade cid-cmd
cid-cmd update --dashboard-id ta-organizational-view --recursive
```

**Changes:**

**Fault Tolerance tab:** Added historical trends for all checks
**Bug fixes and improvements:** Historical trends visuals across all tabs now include not only actual checks to better represent trends. Bug fix for detailed view visuals across all tabs to exclude duplicates in some corner cases

## TAO Dashboard v2.0.0:

**Important:** Update to this version requires cid-cmd v0.2.12. Please update cid-cmd first before updating the dashboard. During the update you'd need to provide a path to s3 folder where your Trusted Advisor data is stored. QuickSight dataset ta-organizational-view will be updated, please make a copy if you've made any customizations to the dataset. To update run these commands in your CloudShell (recommended) or other terminal:
Expand Down
80 changes: 58 additions & 22 deletions cid/builtin/core/data/datasets/tao/dataset.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1023,9 +1023,9 @@
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Resource Name",
"ColumnId": "c32a279b-d728-4947-b6db-f122b7f5250f",
"Expression": "ifelse(\n{Check Name} = 'Idle Load Balancers', {load balancer name},\n{Check Name} = 'Amazon RDS Idle DB Instances', {db instance name},\n{Check Name} = 'Low Utilization Amazon EC2 Instances', {instance id},\n{Check Name} = 'Unassociated Elastic IP Addresses', {ip address},\n{Check Name} = 'Underutilized Amazon EBS Volumes', {volume id},\n{Check Name} = 'Underutilized Amazon Redshift Clusters', cluster,\n{Check Name} = 'Amazon EBS Public Snapshots', {volume id},\n{Check Name} = 'Amazon RDS Public Snapshots', {db instance or cluster id},\n{Check Name} = 'AWS Lambda Functions with High Error Rates', {function arn},\n{Check Name} = 'AWS Lambda Functions Using Deprecated Runtimes', {function arn},\ncheckid = 'Wxdfp4B1L1' OR checkid = 'Wxdfp4B1L2' OR checkid = 'Wxdfp4B1L3' OR checkid = 'Wxdfp4B1L4',{workload arn},\nresourceid\n)"
"ColumnName": "Account",
"ColumnId": "bc17d09c-5a52-4e82-a166-925f8178166d",
"Expression": "ifelse(\n accountname = '' OR isNull(accountname),\n accountid,\n concat(accountname,': ',accountid)\n )"
}
]
}
Expand All @@ -1034,9 +1034,9 @@
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Datetime",
"ColumnId": "a22e1199-f893-48d1-9066-e37c4c7d0f94",
"Expression": "epochDate(parseInt(timestamp))"
"ColumnName": "Resource Id",
"ColumnId": "88b4612e-f9b6-4fff-9e4a-32cdf90b9a86",
"Expression": "concat({resourceid},{Account})"
}
]
}
Expand All @@ -1045,9 +1045,20 @@
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Account",
"ColumnId": "bc17d09c-5a52-4e82-a166-925f8178166d",
"Expression": "ifelse(\n accountname = '' OR isNull(accountname),\n accountid,\n concat(accountname,': ',accountid)\n )"
"ColumnName": "Resource Name",
"ColumnId": "c32a279b-d728-4947-b6db-f122b7f5250f",
"Expression": "ifelse(\n{Check Name} = 'Idle Load Balancers', {load balancer name},\n{Check Name} = 'Amazon RDS Idle DB Instances', {db instance name},\n{Check Name} = 'Low Utilization Amazon EC2 Instances', {instance id},\n{Check Name} = 'Unassociated Elastic IP Addresses', {ip address},\n{Check Name} = 'Underutilized Amazon EBS Volumes', {volume id},\n{Check Name} = 'Underutilized Amazon Redshift Clusters', cluster,\n{Check Name} = 'Amazon EBS Public Snapshots', {volume id},\n{Check Name} = 'Amazon RDS Public Snapshots', {db instance or cluster id},\n{Check Name} = 'AWS Lambda Functions with High Error Rates', {function arn},\n{Check Name} = 'AWS Lambda Functions Using Deprecated Runtimes', {function arn},\ncheckid = 'Wxdfp4B1L1' OR checkid = 'Wxdfp4B1L2' OR checkid = 'Wxdfp4B1L3' OR checkid = 'Wxdfp4B1L4',{workload arn},\n{Resource Id}\n)"
}
]
}
},
{
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Datetime",
"ColumnId": "a22e1199-f893-48d1-9066-e37c4c7d0f94",
"Expression": "epochDate(parseInt(timestamp))"
}
]
}
Expand All @@ -1058,7 +1069,7 @@
{
"ColumnName": "First Check for Resource",
"ColumnId": "b2d7393f-c258-45ce-aa35-e93bdd879652",
"Expression": "minOver(Datetime, [{Resource Name}],PRE_AGG)"
"Expression": "minOver(Datetime, [{Resource Name}],PRE_AGG)"
}
]
}
Expand All @@ -1069,7 +1080,7 @@
{
"ColumnName": "Last Check for Resource",
"ColumnId": "da58a4e8-1246-46a2-b1f2-416d33eb1d51",
"Expression": "ifelse(\ncategory = 'Cost Optimizing',\nmaxOver(Datetime, [{Resource Name},{instance type}],PRE_AGG),\n{Check Name} = 'Amazon EBS Snapshots',\nmaxOver(Datetime, [{volume id},{Account},{snapshot age},{volume attachment}],PRE_AGG),\n{Check Name} = 'Amazon EC2 Availability Zone Balance',\nmaxOver(Datetime, [{Account},{region}, {instances in zone a},{instances in zone b},{instances in zone c},{instances in zone d},{instances in zone e},{instances in zone f}],PRE_AGG),\n\nmaxOver(Datetime, [{Resource Name}],PRE_AGG)\n)\n"
"Expression": "ifelse(\ncategory = 'Cost Optimizing',\nmaxOver(Datetime, [{Resource Id},{instance type}],PRE_AGG),\n{Check Name} = 'Amazon EBS Snapshots',\nmaxOver(Datetime, [{volume id},{Account},{snapshot age},{volume attachment}],PRE_AGG),\n{Check Name} = 'Amazon EC2 Availability Zone Balance',\nmaxOver(Datetime, [{Account},{region}, {instances in zone a},{instances in zone b},{instances in zone c},{instances in zone d},{instances in zone e},{instances in zone f}],PRE_AGG),\n\nmaxOver(Datetime, [{Resource Id}],PRE_AGG)\n)\n"
}
]
}
Expand All @@ -1091,7 +1102,7 @@
{
"ColumnName": "Resource in Last Check",
"ColumnId": "f3f28fd1-5b05-48d4-8b73-54f7981213e4",
"Expression": "ifelse(dateDiff({Last Check for Resource},{Last Check for Account and CheckId}) >= 7, 'no', 'yes')"
"Expression": "ifelse(dateDiff({Last Check for Resource},{Last Check for Account and CheckId}) >= 8, 'no', 'yes')"
}
]
}
Expand Down Expand Up @@ -1135,7 +1146,7 @@
{
"ColumnName": "Deprecation Date",
"ColumnId": "e351a4a5-1a61-4ee5-9927-69040296e120",
"Expression": "maxOver(parseDate({deprecation date},'MM/dd/yyyy'),[{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseDate({deprecation date},'MM/dd/yyyy'),[{Resource Name}],PRE_AGG)"
}
]
}
Expand All @@ -1146,7 +1157,7 @@
{
"ColumnName": "Average Daily Invokes",
"ColumnId": "0108dd54-9ae3-447d-97d8-788894e6a8ae",
"Expression": "maxOver(parseDecimal({average daily invokes}),[{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseDecimal({average daily invokes}),[{Resource Name}],PRE_AGG)"
}
]
}
Expand All @@ -1157,7 +1168,7 @@
{
"ColumnName": "Number of Days Low Utilization",
"ColumnId": "b9a05d2d-5997-4677-9ad4-73f1f0534210",
"Expression": "maxOver(parseInt(replace({number of days low utilization}, ' days','')), [{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseInt(replace({number of days low utilization}, ' days','')), [{Resource Id}],PRE_AGG)"
}
]
}
Expand All @@ -1168,7 +1179,7 @@
{
"ColumnName": "14-Day Average CPU Utilization",
"ColumnId": "d04502b9-c9df-4ef0-a78d-a11771b5967f",
"Expression": "maxOver(parseDecimal(replace({14-day average cpu utilization}, '%','')), [{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseDecimal(replace({14-day average cpu utilization}, '%','')), [{Resource Id}],PRE_AGG)"
}
]
}
Expand All @@ -1179,7 +1190,7 @@
{
"ColumnName": "14-Day Average Network I/O",
"ColumnId": "c059301f-0228-44be-b827-4337615995ac",
"Expression": "maxOver(parseDecimal(replace({14-day average network i/o}, 'MB','')), [{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseDecimal(replace({14-day average network i/o}, 'MB','')), [{Resource Id}],PRE_AGG)"
}
]
}
Expand All @@ -1190,7 +1201,7 @@
{
"ColumnName": "Max Daily Error Rate",
"ColumnId": "b6ea96d4-6a88-49ac-b24d-b12942c72fd4",
"Expression": "maxOver(parseDecimal({max daily error rate}),[{Resource Name}],PRE_AGG)"
"Expression": "maxOver(parseDecimal({max daily error rate}),[{Resource Id}],PRE_AGG)"
}
]
}
Expand All @@ -1201,7 +1212,7 @@
{
"ColumnName": "Date for Max Error Rate",
"ColumnId": "ae799e1f-17d8-44dd-9374-f538a041200b",
"Expression": "maxOver(parseDate({date for max error rate},'MM/dd/yyyy'),[{Resource Name}],PRE_AGG)\n"
"Expression": "maxOver(parseDate({date for max error rate},'MM/dd/yyyy'),[{Resource Id}],PRE_AGG)\n"
}
]
}
Expand All @@ -1212,7 +1223,7 @@
{
"ColumnName": "Average Daily Error Rate",
"ColumnId": "e22c306a-1ef8-472e-9c98-0b487cf917c2",
"Expression": "maxOver(parseDecimal({average daily error rate}),[{Resource Name}],PRE_AGG)\n"
"Expression": "maxOver(parseDecimal({average daily error rate}),[{Resource Id}],PRE_AGG)\n"
}
]
}
Expand Down Expand Up @@ -1349,6 +1360,28 @@
]
}
},
{
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Estimated Monthly Savings",
"ColumnId": "5334421f-d73e-4b5f-bdab-b7553553cc25",
"Expression": "ifelse( maxOver(Datetime, [{Resource Id}] , PRE_AGG) = Datetime, {Estimated Monthly Savings Numeric}, null)\n"
}
]
}
},
{
"CreateColumnsOperation": {
"Columns": [
{
"ColumnName": "Estimated Monthly Savings Trend",
"ColumnId": "2f67fc2d-9c64-4f14-9d28-f962cad070d8",
"Expression": "ifelse( maxOver(Datetime, [resourceid,extract('MM', Datetime),extract('YYYY', Datetime)] , PRE_AGG) = Datetime, {Estimated Monthly Savings Numeric}, null)"
}
]
}
},
{
"ProjectOperation": {
"ProjectedColumns": [
Expand Down Expand Up @@ -1593,7 +1626,7 @@
"number of identified hris for cost optimization",
"cluster name",
"Check Name",
"Resource Name",
"Resource Name",
"Datetime",
"First Check for Resource",
"Last Check for Resource",
Expand Down Expand Up @@ -1622,7 +1655,10 @@
"Max Daily Median",
"Number of Days Over",
"DateTime is Last Check for Resource",
"One Check Selected"
"One Check Selected",
"Resource Id",
"Estimated Monthly Savings",
"Estimated Monthly Savings Trend"
]
}
}
Expand Down
Loading