chore(agw): Update metrics naming on exporting #11668

ardzoht · 2022-02-17T19:04:19Z

Summary

Currently, metrics collector on magmad is using a deprecated proto file (metricsd.proto) to set the name for each Metric family (collection of metrics on a service)
This change updates the filling of the metric name to use the name from the prometheus metric name rather from the protobuf description name.
Updates s1ap tests to remove metricsd proto enum values

Test Plan

make integ_test
running local environment and making sure metrics are still collected and exported successfully to the cloud:

Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Collected 9 from state...
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Collected 18 from pipelined...
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Metrics upload success for service pipelined (chunk 0)
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Metrics upload success for service magmad (chunk 0)
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Metrics upload success for service sessiond (chunk 0)
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Metrics upload success for service state (chunk 0)
Feb 08 22:45:33 magma-dev-focal magmad[51255]: DEBUG:root:Metrics upload success for service enodebd (chunk 0)

service303_cli.py metrics pipelined before:

name: "50"
type: GAUGE
metric {
  label {
    name: "10"
    value: "3.8.5"
  }
  label {
    name: "11"
    value: "CPython"
  }
  label {
    name: "9"
    value: "3"
  }
  label {
    name: "8"
    value: "8"
  }
  label {
    name: "7"
    value: "5"
  }
  gauge {
    value: 1.0
  }
  timestamp_ms: 1644366222080
}

name: "309"
type: COUNTER

name: "550"
type: COUNTER
metric {
  counter {
    value: 2.0
  }
  timestamp_ms: 1644366222080
}

name: "350"
type: COUNTER

name: "351"
type: COUNTER
metric {
  counter {
    value: 0.0
  }
  timestamp_ms: 1644366222080
}

name: "352"
type: COUNTER

name: "353"
type: COUNTER
metric {
  counter {
    value: 0.0
  }
  timestamp_ms: 1644366222080
}

service303_cli.py metrics pipelined after:

name: "python_info"
type: GAUGE
metric {
  label {
    name: "10"
    value: "3.8.5"
  }
  label {
    name: "11"
    value: "CPython"
  }
  label {
    name: "9"
    value: "3"
  }
  label {
    name: "8"
    value: "8"
  }
  label {
    name: "7"
    value: "5"
  }
  gauge {
    value: 1.0
  }
  timestamp_ms: 1644374679693
}

name: "streamer_responses"
type: COUNTER

name: "service_errors"
type: COUNTER
metric {
  counter {
    value: 2.0
  }
  timestamp_ms: 1644374679693
}

name: "dp_send_msg_error"
type: COUNTER

name: "arp_default_gw_mac_error"
type: COUNTER
metric {
  counter {
    value: 0.0
  }
  timestamp_ms: 1644374679693
}

name: "openflow_error_msg"
type: COUNTER

name: "unknown_pkt_direction"
type: COUNTER
metric {
  counter {
    value: 0.0
  }
  timestamp_ms: 1644374679693
}

name: "enforcement_rule_install_fail"
type: COUNTER

name: "enforcement_stats_rule_install_fail"
type: COUNTER

Additional Information

This change is backwards-breaking

github-actions · 2022-02-17T19:04:37Z

Thanks for opening a PR! 💯

A couple initial guidelines

All commits must be signed off. This is enforced by DCO check.
All PR titles must follow the semantic commits format. This is enforced by Semantic PR.

Howto

Reviews. The "Reviewers" listed for this PR are the Magma maintainers who will shepherd it.
Checks. All required CI checks must pass before merge.
Merge. Once approved and passing CI checks, use the ready2merge label to indicate the maintainers can merge your PR.

More info

Please take a moment to read through the Magma project's

Contributing Conventions for norms around contributed code

If this is your first Magma PR, also consider reading

Developer Onboarding for onboarding as a new Magma developer
Development Workflow for guidance on your first pull request
CI Checks for points of contact for failing or flaky CI checks
GitHub-to-Slack mappings for Magma maintainers for guidance on how to contact maintainers on Slack

github-actions · 2022-02-17T19:12:55Z

feg-workflow

    2 files 202 suites 32s ⏱️
362 tests 362 ✔️ 0 💤 0 ❌
376 runs 376 ✔️ 0 💤 0 ❌

Results for commit 4e54f54.

♻️ This comment has been updated with latest results.

themarwhal · 2022-02-17T19:15:27Z

orc8r/gateway/python/magma/magmad/tests/BUILD.bazel

@@ -29,12 +29,6 @@ pytest_test(
    deps = ["//orc8r/gateway/python/magma/magmad:magmad_lib"],
 )

-pytest_test(
-    name = "metrics_tests",
-    srcs = ["metrics_tests.py"],


is this a deleted test?

oh i see it below :p

ulaskozat · 2022-02-17T19:27:57Z

lte/gateway/python/integ_tests/s1aptests/test_guti_attach.py

@@ -0,0 +1,220 @@
+"""


what is the reason for file name change?

what is the difference from test_guti_attach_with_zero_mtmsi.py

I do not see the connection of this test to the metrics naming.

i think this file got included as part of a rebase with other commit I had on my local history, I will remove it, this is not supposed to be added

github-actions · 2022-02-17T19:36:50Z

cloud-workflow

    7 files 351 suites 2m 45s ⏱️
960 tests 960 ✔️ 0 💤 0 ❌

Results for commit 4e54f54.

♻️ This comment has been updated with latest results.

github-actions · 2022-02-17T19:44:48Z

agw-workflow

    76 files   121 suites 5m 55s ⏱️
1 101 tests 1 092 ✔️ 9 💤 0 ❌
1 102 runs 1 093 ✔️ 9 💤 0 ❌

Results for commit 4e54f54.

♻️ This comment has been updated with latest results.

ulaskozat

LGTM

Signed-off-by: Alex Rodriguez <alexrod@fb.com>

github-actions · 2022-03-29T01:24:26Z

Unit Test Results

157 files 157 suites 2h 6m 0s ⏱️
307 tests 304 ✔️ 3 💤 0 ❌

Results for commit c0f4b05.

♻️ This comment has been updated with latest results.

Signed-off-by: Alex Rodriguez <alexrod@fb.com>

…1668) Signed-off-by: Alex Rodriguez <alexrod@fb.com>

ardzoht requested review from a team, nstng, ssanadhya and themarwhal February 17, 2022 19:04

pull-request-size bot added the size/XL Denotes a Pull Request that changes 500-999 lines. label Feb 17, 2022

github-actions bot added component: agw Access gateway-related issue component: orc8r Orchestrator-related issue labels Feb 17, 2022

themarwhal approved these changes Feb 17, 2022

View reviewed changes

themarwhal reviewed Feb 17, 2022

View reviewed changes

ulaskozat reviewed Feb 17, 2022

View reviewed changes

ardzoht force-pushed the update_metrics_naming branch from d331b18 to 296ca20 Compare February 17, 2022 20:11

pull-request-size bot added size/L Denotes a Pull Request that changes 100-499 lines. and removed size/XL Denotes a Pull Request that changes 500-999 lines. labels Feb 17, 2022

ulaskozat approved these changes Feb 17, 2022

View reviewed changes

ardzoht force-pushed the update_metrics_naming branch 5 times, most recently from 14454b2 to 303a22c Compare March 2, 2022 16:17

ardzoht force-pushed the update_metrics_naming branch from 303a22c to 571c6e1 Compare March 28, 2022 05:48

chore(agw): Update metrics naming on exporting (magma#11474)

4e54f54

Signed-off-by: Alex Rodriguez <alexrod@fb.com>

ardzoht force-pushed the update_metrics_naming branch from 571c6e1 to 4e54f54 Compare March 28, 2022 17:26

ardzoht merged commit c0f4b05 into magma:master Mar 29, 2022

ardzoht added a commit that referenced this pull request Mar 30, 2022

chore(agw): Update metrics naming on exporting (#11474) (#11668)

52cf525

Signed-off-by: Alex Rodriguez <alexrod@fb.com>

ardzoht mentioned this pull request May 9, 2022

Deprecate proto enum parsing on cloud metricsd service #12660

Closed

vroon2703 mentioned this pull request Jun 2, 2022

chore(orc8r): Cleanup metric naming to not use deprecated protobuf file #12906

Merged

emakeev pushed a commit to emakeev/magma that referenced this pull request Aug 5, 2022

chore(agw): Update metrics naming on exporting (magma#11474) (magma#1…

6503a88

…1668) Signed-off-by: Alex Rodriguez <alexrod@fb.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(agw): Update metrics naming on exporting #11668

chore(agw): Update metrics naming on exporting #11668

ardzoht commented Feb 17, 2022 •

edited

github-actions bot commented Feb 17, 2022

github-actions bot commented Feb 17, 2022 •

edited

themarwhal Feb 17, 2022

themarwhal Feb 17, 2022

ulaskozat Feb 17, 2022

ulaskozat Feb 17, 2022

ulaskozat Feb 17, 2022

ardzoht Feb 17, 2022

github-actions bot commented Feb 17, 2022 •

edited

github-actions bot commented Feb 17, 2022 •

edited

ulaskozat left a comment

github-actions bot commented Mar 29, 2022 •

edited

chore(agw): Update metrics naming on exporting #11668

chore(agw): Update metrics naming on exporting #11668

Conversation

ardzoht commented Feb 17, 2022 • edited

Summary

Test Plan

Additional Information

github-actions bot commented Feb 17, 2022

Howto

More info

github-actions bot commented Feb 17, 2022 • edited

feg-workflow

themarwhal Feb 17, 2022

Choose a reason for hiding this comment

themarwhal Feb 17, 2022

Choose a reason for hiding this comment

ulaskozat Feb 17, 2022

Choose a reason for hiding this comment

ulaskozat Feb 17, 2022

Choose a reason for hiding this comment

ulaskozat Feb 17, 2022

Choose a reason for hiding this comment

ardzoht Feb 17, 2022

Choose a reason for hiding this comment

github-actions bot commented Feb 17, 2022 • edited

cloud-workflow

github-actions bot commented Feb 17, 2022 • edited

agw-workflow

ulaskozat left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 29, 2022 • edited

Unit Test Results

ardzoht commented Feb 17, 2022 •

edited

github-actions bot commented Feb 17, 2022 •

edited

github-actions bot commented Feb 17, 2022 •

edited

github-actions bot commented Feb 17, 2022 •

edited

github-actions bot commented Mar 29, 2022 •

edited