Usage Telemetry v3 by clintoncwolfe · Pull Request #6012 · inspec/inspec

clintoncwolfe · 2022-04-25T13:33:22Z

Description

Adds EULA-specified usage telemetry to InSpec.

This telemetry relates to the consumption of the licenses, and is being collected under section 1.12 of the the EULA. You may also refer to our privacy policy.

Removes the vestigial v1 telemetry code that was never used.

Related Issue

Types of changes

Bug fix (non-breaking change which fixes an issue)
New content (non-breaking change)
Breaking change (a content change which would break existing functionality or processes)

Checklist:

I have read the CONTRIBUTING document.

netlify · 2022-04-25T13:33:26Z

✅ Deploy Preview for chef-inspec canceled.

Name	Link
🔨 Latest commit	`b50ab0e`
🔍 Latest deploy log	https://app.netlify.com/sites/chef-inspec/deploys/6266a3237719a900089b164b

netlify · 2024-01-31T09:56:22Z

✅ Deploy Preview for chef-inspec ready!

Name	Link
🔨 Latest commit	`1313e5b`
🔍 Latest deploy log	https://app.netlify.com/sites/chef-inspec/deploys/669e45046c205b00087924c0
😎 Deploy Preview	https://deploy-preview-6012--chef-inspec.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

+  datetime: /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)((-(\d{2}):(\d{2})|Z)?)$/,
+  uuid: /^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$/,
+  transport: /^[a-z0-9\-\_]+$/,
+  sha256: /^[0-9a-fA-F]{64}|unknown$/,


Vasu1105

LGTM

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Added initial changes to jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Feature flag changes for telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> * move base, debug and null to its own file structure Signed-off-by: Sathish <sbabu@progress.com> * make HTTP client post requests Signed-off-by: Sathish <sbabu@progress.com> * remove old logic Signed-off-by: Sathish <sbabu@progress.com> * make backend class as `HTTP` Signed-off-by: Sathish <sbabu@progress.com> * CHEF-7258 Fetch and use licensing information for telemetry (#6964) * Added method to fetch license ids for inspec Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added free license check for performing telemetry api call Signed-off-by: Nik08 <nikita.mathur@progress.com> * move base, debug and null to its own file structure Signed-off-by: Sathish <sbabu@progress.com> * make HTTP client post requests Signed-off-by: Sathish <sbabu@progress.com> * remove old logic Signed-off-by: Sathish <sbabu@progress.com> * make backend class as `HTTP` Signed-off-by: Sathish <sbabu@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com> * Updated control tags and desc value to be used in jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added checks for automate run context and free license check Signed-off-by: Nik08 <nikita.mathur@progress.com> * capture target mode and id Signed-off-by: Sathish <sbabu@progress.com> * profile doesn't need ID Signed-off-by: Sathish <sbabu@progress.com> * use run context to set environment data Signed-off-by: Sathish <sbabu@progress.com> * refactor `create_wrapper` to be localized Signed-off-by: Sathish <sbabu@progress.com> * change all timestamps to be UTC Signed-off-by: Sathish <sbabu@progress.com> * Null checks for response and corrected job api endpoint Signed-off-by: Nik08 <nikita.mathur@progress.com> * Fixed tag values to be sent as string in api call Signed-off-by: Nik08 <nikita.mathur@progress.com> * make version as float Signed-off-by: Sathish <sbabu@progress.com> * add platform name Signed-off-by: Sathish <sbabu@progress.com> * Added control result data in jobs api payload Signed-off-by: Nik08 <nikita.mathur@progress.com> * Debug logs added for telemetry call Signed-off-by: Nik08 <nikita.mathur@progress.com> * Removed unwanted telemetry debug class Signed-off-by: Nik08 <nikita.mathur@progress.com> * Payload fix to pass features data only on per control basis Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added class function to list all invoked features by feature sub system Signed-off-by: Nik08 <nikita.mathur@progress.com> * Using feature system to get all invoked features list to be used in jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Unit tests cases updated and fixed Signed-off-by: Nik08 <nikita.mathur@progress.com> * License type check downcased Signed-off-by: Nik08 <nikita.mathur@progress.com> * Lint fix Signed-off-by: Nik08 <nikita.mathur@progress.com> * CHEF-7265 Telemetry opt-in for CINC users (#6966) * Enabled telemtry opt-in Signed-off-by: Nik08 <nikita.mathur@progress.com> * Removed old comments Signed-off-by: Nik08 <nikita.mathur@progress.com> * Unit test case added to validate the disabling telemetry behaviour for inspec user Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com>

…icense users Signed-off-by: Nik08 <nikita.mathur@progress.com>

Signed-off-by: Nik08 <nikita.mathur@progress.com>

…out feature flag env set Signed-off-by: Nik08 <nikita.mathur@progress.com>

…est for telemetry data Signed-off-by: Nik08 <nikita.mathur@progress.com>

… set in env Signed-off-by: Nik08 <nikita.mathur@progress.com>

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Removed usage of feature system to enable telemetry - making it opt-in by default Signed-off-by: Nik08 <nikita.mathur@progress.com> * Telemetry disable check fix when no option is passed in args Signed-off-by: Nik08 <nikita.mathur@progress.com> * Fix in test to use license specs defined for testing Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com>

* load default telemetry url conditionally Signed-off-by: Sathish <sbabu@progress.com> * remove version base path version base path is defined in jobs path already Signed-off-by: Sathish <sbabu@progress.com> * use `CHEF_` prefix for the ENV Signed-off-by: Sathish <sbabu@progress.com> --------- Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com>

Signed-off-by: Nik08 <nikita.mathur@progress.com>

…license usage Signed-off-by: Nik08 <nikita.mathur@progress.com>

* WIP chef telemetry env variable usage updated Signed-off-by: Nik08 <nikita.mathur@progress.com> * WIP intro added for chef telemetry - requires edit Signed-off-by: Nik08 <nikita.mathur@progress.com> * Correction in opt in behaviour of telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc update after default opt in changes Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc edit from product Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc edit Signed-off-by: Nik08 <nikita.mathur@progress.com> * Edits Signed-off-by: Ian Maddaus <ian.maddaus@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Ian Maddaus <ian.maddaus@progress.com> Co-authored-by: Ian Maddaus <ian.maddaus@progress.com>

…etry Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Remove unused telemetry v1 code Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com> * Sketch out basics of telemetry, with start/stop of invocation telemetry Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com> * Data structure for run telemetry - job capture Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com> * Add per-control and per-run feature detection Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com> * CHEF-4017 Telemetry job api updations (#6965) * Added initial changes to jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Feature flag changes for telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> * move base, debug and null to its own file structure Signed-off-by: Sathish <sbabu@progress.com> * make HTTP client post requests Signed-off-by: Sathish <sbabu@progress.com> * remove old logic Signed-off-by: Sathish <sbabu@progress.com> * make backend class as `HTTP` Signed-off-by: Sathish <sbabu@progress.com> * CHEF-7258 Fetch and use licensing information for telemetry (#6964) * Added method to fetch license ids for inspec Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added free license check for performing telemetry api call Signed-off-by: Nik08 <nikita.mathur@progress.com> * move base, debug and null to its own file structure Signed-off-by: Sathish <sbabu@progress.com> * make HTTP client post requests Signed-off-by: Sathish <sbabu@progress.com> * remove old logic Signed-off-by: Sathish <sbabu@progress.com> * make backend class as `HTTP` Signed-off-by: Sathish <sbabu@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com> * Updated control tags and desc value to be used in jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added checks for automate run context and free license check Signed-off-by: Nik08 <nikita.mathur@progress.com> * capture target mode and id Signed-off-by: Sathish <sbabu@progress.com> * profile doesn't need ID Signed-off-by: Sathish <sbabu@progress.com> * use run context to set environment data Signed-off-by: Sathish <sbabu@progress.com> * refactor `create_wrapper` to be localized Signed-off-by: Sathish <sbabu@progress.com> * change all timestamps to be UTC Signed-off-by: Sathish <sbabu@progress.com> * Null checks for response and corrected job api endpoint Signed-off-by: Nik08 <nikita.mathur@progress.com> * Fixed tag values to be sent as string in api call Signed-off-by: Nik08 <nikita.mathur@progress.com> * make version as float Signed-off-by: Sathish <sbabu@progress.com> * add platform name Signed-off-by: Sathish <sbabu@progress.com> * Added control result data in jobs api payload Signed-off-by: Nik08 <nikita.mathur@progress.com> * Debug logs added for telemetry call Signed-off-by: Nik08 <nikita.mathur@progress.com> * Removed unwanted telemetry debug class Signed-off-by: Nik08 <nikita.mathur@progress.com> * Payload fix to pass features data only on per control basis Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added class function to list all invoked features by feature sub system Signed-off-by: Nik08 <nikita.mathur@progress.com> * Using feature system to get all invoked features list to be used in jobs api Signed-off-by: Nik08 <nikita.mathur@progress.com> * Unit tests cases updated and fixed Signed-off-by: Nik08 <nikita.mathur@progress.com> * License type check downcased Signed-off-by: Nik08 <nikita.mathur@progress.com> * Lint fix Signed-off-by: Nik08 <nikita.mathur@progress.com> * CHEF-7265 Telemetry opt-in for CINC users (#6966) * Enabled telemtry opt-in Signed-off-by: Nik08 <nikita.mathur@progress.com> * Removed old comments Signed-off-by: Nik08 <nikita.mathur@progress.com> * Unit test case added to validate the disabling telemetry behaviour for inspec user Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com> * Product team review changes - only disable telemetry for commercial license users Signed-off-by: Nik08 <nikita.mathur@progress.com> * Connection failure handling for telemetry http call Signed-off-by: Nik08 <nikita.mathur@progress.com> * Testing fix - Remove usage of deleted library Signed-off-by: Nik08 <nikita.mathur@progress.com> * Telemetry test case fix - Issue caused because unit test are run without feature flag env set Signed-off-by: Nik08 <nikita.mathur@progress.com> * Fixed and replaced tightly coupled semver versioning regex matching test for telemetry data Signed-off-by: Nik08 <nikita.mathur@progress.com> * Telemery test fix to use license key from env or a dummy value if not set in env Signed-off-by: Nik08 <nikita.mathur@progress.com> * Added error logs in case the http call is not successful for telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> * Error handling for telemetry start and run calls Signed-off-by: Nik08 <nikita.mathur@progress.com> * Telemetry opt-in changes (#7055) * Removed usage of feature system to enable telemetry - making it opt-in by default Signed-off-by: Nik08 <nikita.mathur@progress.com> * Telemetry disable check fix when no option is passed in args Signed-off-by: Nik08 <nikita.mathur@progress.com> * Fix in test to use license specs defined for testing Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> * (Restoring) CHEF-10392 load default telemetry url conditionally (#7059) * load default telemetry url conditionally Signed-off-by: Sathish <sbabu@progress.com> * remove version base path version base path is defined in jobs path already Signed-off-by: Sathish <sbabu@progress.com> * use `CHEF_` prefix for the ENV Signed-off-by: Sathish <sbabu@progress.com> --------- Signed-off-by: Sathish <sbabu@progress.com> Co-authored-by: Sathish <sbabu@progress.com> * Typo fix in features list Signed-off-by: Nik08 <nikita.mathur@progress.com> * Stub added for CI license key Signed-off-by: Nik08 <nikita.mathur@progress.com> * License usage telemetry correction - not track control results (#7060) Signed-off-by: Nik08 <nikita.mathur@progress.com> * Changes to disable telemetry for other InSpec distros (#7065) Signed-off-by: Nik08 <nikita.mathur@progress.com> * Lint issue fix Signed-off-by: Nik08 <nikita.mathur@progress.com> * Removing disable telemetry test - breaks on CI because of commercial license usage Signed-off-by: Nik08 <nikita.mathur@progress.com> * CHEF-13228 Chef licensing telemetry documentation (#7056) * WIP chef telemetry env variable usage updated Signed-off-by: Nik08 <nikita.mathur@progress.com> * WIP intro added for chef telemetry - requires edit Signed-off-by: Nik08 <nikita.mathur@progress.com> * Correction in opt in behaviour of telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc update after default opt in changes Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc edit from product Signed-off-by: Nik08 <nikita.mathur@progress.com> * Doc edit Signed-off-by: Nik08 <nikita.mathur@progress.com> * Edits Signed-off-by: Ian Maddaus <ian.maddaus@progress.com> --------- Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Ian Maddaus <ian.maddaus@progress.com> Co-authored-by: Ian Maddaus <ian.maddaus@progress.com> * Updated version pinning of chef licensing to version 1 for chef telemetry Signed-off-by: Nik08 <nikita.mathur@progress.com> --------- Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com> Signed-off-by: Nik08 <nikita.mathur@progress.com> Signed-off-by: Sathish <sbabu@progress.com> Signed-off-by: Ian Maddaus <ian.maddaus@progress.com> Co-authored-by: Nikita Mathur <Nik08@users.noreply.github.com> Co-authored-by: Sathish <sbabu@progress.com> Co-authored-by: Nik08 <nikita.mathur@progress.com> Co-authored-by: Ian Maddaus <ian.maddaus@progress.com>

chef-expeditor Bot assigned clintoncwolfe Apr 25, 2022

ahasunos added the inspec-6 label Sep 30, 2023

Nik08 force-pushed the cw/general-telemetry-v3 branch from b50ab0e to bcf6425 Compare January 31, 2024 09:56

github-advanced-security AI found potential problems Jan 31, 2024

View reviewed changes

Comment thread test/unit/utils/telemetry_test.rb Fixed

Comment thread test/unit/utils/telemetry_test.rb Fixed

Nik08 mentioned this pull request Feb 15, 2024

CHEF-4017 Telemetry job api updations #6965

Merged

4 tasks

clintoncwolfe marked this pull request as ready for review April 10, 2024 12:15

clintoncwolfe requested a review from a team as a code owner April 10, 2024 12:15

clintoncwolfe requested review from Nik08 and ahasunos April 10, 2024 12:15

Nik08 force-pushed the cw/general-telemetry-v3 branch 2 times, most recently from c4aa244 to f02267b Compare April 16, 2024 09:26

github-advanced-security AI found potential problems Apr 18, 2024

View reviewed changes

Nik08 force-pushed the cw/general-telemetry-v3 branch from 47764db to 005e370 Compare May 13, 2024 07:57

Nik08 force-pushed the cw/general-telemetry-v3 branch 2 times, most recently from 802d261 to 9608019 Compare June 12, 2024 10:04

Vasu1105 reviewed Jun 12, 2024

View reviewed changes

Comment thread etc/features.yaml Outdated

Vasu1105 reviewed Jun 12, 2024

View reviewed changes

Comment thread lib/inspec/utils/telemetry/http.rb Outdated

Nik08 force-pushed the cw/general-telemetry-v3 branch 2 times, most recently from fca9946 to 9898c19 Compare June 13, 2024 10:49

Vasu1105 approved these changes Jul 3, 2024

View reviewed changes

Nik08 force-pushed the cw/general-telemetry-v3 branch from 9db27cc to d083b81 Compare July 3, 2024 07:36

Nik08 changed the title ~~General Telemetry v3~~ [DO NOT MERGE] General Telemetry v3 Jul 3, 2024

clintoncwolfe changed the title ~~[DO NOT MERGE] General Telemetry v3~~ [DO NOT MERGE] Usage Telemetry v3 Jul 3, 2024

Nik08 requested a review from a team as a code owner July 11, 2024 11:00

github-actions Bot added the Documentation ZH multi-repo label for the docs-team label Jul 11, 2024

Nik08 changed the title ~~[DO NOT MERGE] Usage Telemetry v3~~ Usage Telemetry v3 Jul 22, 2024

Vasu1105 added the Expeditor: Bump Minor Version label Jul 22, 2024

Remove unused telemetry v1 code

4dc2067

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

clintoncwolfe and others added 22 commits July 22, 2024 17:09

Sketch out basics of telemetry, with start/stop of invocation telemetry

476b8c0

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

Data structure for run telemetry - job capture

9accd3b

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

Add per-control and per-run feature detection

2a6a04e

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

Product team review changes - only disable telemetry for commercial l…

7be2962

…icense users Signed-off-by: Nik08 <nikita.mathur@progress.com>

Connection failure handling for telemetry http call

6024df8

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Testing fix - Remove usage of deleted library

1531357

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Telemetry test case fix - Issue caused because unit test are run with…

133ea1a

…out feature flag env set Signed-off-by: Nik08 <nikita.mathur@progress.com>

Fixed and replaced tightly coupled semver versioning regex matching t…

9cd3f7c

…est for telemetry data Signed-off-by: Nik08 <nikita.mathur@progress.com>

Telemery test fix to use license key from env or a dummy value if not…

bc4bfd7

… set in env Signed-off-by: Nik08 <nikita.mathur@progress.com>

Added error logs in case the http call is not successful for telemetry

2a36105

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Error handling for telemetry start and run calls

34850da

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Typo fix in features list

5d2a55b

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Stub added for CI license key

fe2f73c

Signed-off-by: Nik08 <nikita.mathur@progress.com>

License usage telemetry correction - not track control results (#7060)

d7f593d

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Changes to disable telemetry for other InSpec distros (#7065)

131add5

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Lint issue fix

f6232c0

Signed-off-by: Nik08 <nikita.mathur@progress.com>

Removing disable telemetry test - breaks on CI because of commercial …

1a96ba3

…license usage Signed-off-by: Nik08 <nikita.mathur@progress.com>

Updated version pinning of chef licensing to version 1 for chef telem…

1313e5b

…etry Signed-off-by: Nik08 <nikita.mathur@progress.com>

Nik08 force-pushed the cw/general-telemetry-v3 branch from cf2193e to 1313e5b Compare July 22, 2024 11:39

Vasu1105 merged commit 29242de into main Jul 22, 2024

Vasu1105 deleted the cw/general-telemetry-v3 branch July 22, 2024 13:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Usage Telemetry v3#6012

Usage Telemetry v3#6012
Vasu1105 merged 23 commits intomainfrom
cw/general-telemetry-v3

clintoncwolfe commented Apr 25, 2022 •

edited by Nik08

Loading

Uh oh!

netlify Bot commented Apr 25, 2022 •

edited

Loading

Uh oh!

netlify Bot commented Jan 31, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Check failure

Uh oh!

Uh oh!

Vasu1105 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

clintoncwolfe commented Apr 25, 2022 • edited by Nik08 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Types of changes

Checklist:

Uh oh!

netlify Bot commented Apr 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for chef-inspec canceled.

Uh oh!

netlify Bot commented Jan 31, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for chef-inspec ready!

Uh oh!

Uh oh!

Uh oh!

Check failure

Uh oh!

Uh oh!

Vasu1105 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

clintoncwolfe commented Apr 25, 2022 •

edited by Nik08

Loading

netlify Bot commented Apr 25, 2022 •

edited

Loading

netlify Bot commented Jan 31, 2024 •

edited

Loading