[installation-telemetry]: initial commit plus installer setup #7503

mrsimonemms · 2022-01-07T15:24:22Z

Description

This creates the installation-telemetry component with room to expand. At the moment, there is anticipated to be an availableUpdate command to search a known source for the next available version on the same channel. In future, there could be more commands to run from this binary.

~~This does not add the Segment IO token to the binary at build time from a secret source, but that will be added in a future PR.~~ This is now added

This also configures the Installer to run this as a @daily cronjob. Typically, this will run at the server's midnight. We may wish to change this to a random time once per day in future if using our infrastructure, but whilst using Segment, this is an acceptable compromise.

The identification of an installation is done by hashing the domain name as an SHA512 string, which should be complex enough to prevent identification of the original domain name. It is likely that an optional secret with a salt will be added in future to prevent brute force identification and further reassure users that their data is truly anonymised - as the data isn't currently being sent, this is out of scope for this PR.

Related Issue(s)

Fixes #7499
Fixes #7500
Fixes #7526
Fixes #7539

How to test

Deploy to a self-hosted instance. You will see a cronjob appear, but the logs should say installation-telemetry is not permitted to send - exiting. Consider overwriting the TelemetryCronSchedule to * * * * * to send every minute.

Release Notes

[installation-telemetry]: initial commit plus installer setup

Documentation

codecov · 2022-01-07T15:32:12Z

Codecov Report

Merging #7503 (e813000) into main (aa47428) will increase coverage by 1.15%.
The diff coverage is n/a.

❗ Current head e813000 differs from pull request most recent head 717909a. Consider uploading reports for the commit 717909a to get more accurate results

@@           Coverage Diff            @@
##            main   #7503      +/-   ##
========================================
+ Coverage   8.38%   9.53%   +1.15%     
========================================
  Files         33      18      -15     
  Lines       2410    1080    -1330     
========================================
- Hits         202     103      -99     
+ Misses      2204     976    -1228     
+ Partials       4       1       -3

Flag	Coverage Δ
components-gitpod-cli-app	`9.53% <ø> (ø)`
components-installation-telemetry-app	`∅ <ø> (?)`
components-local-app-app-darwin-amd64	`?`
components-local-app-app-darwin-arm64	`?`
components-local-app-app-linux-amd64	`?`
components-local-app-app-linux-arm64	`?`
components-local-app-app-windows-386	`?`
components-local-app-app-windows-amd64	`?`
components-local-app-app-windows-arm64	`?`
installer-raw-app	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
installer/pkg/common/common.go
installer/pkg/common/display.go
installer/pkg/common/storage.go
installer/pkg/components/ws-manager/configmap.go
installer/pkg/common/render.go
installer/pkg/components/ws-manager/deployment.go
...components/ws-manager/unpriviledged-rolebinding.go
installer/pkg/components/ws-manager/tlssecret.go
installer/pkg/common/objects.go
installer/pkg/components/ws-manager/rolebinding.go
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aa47428...717909a. Read the comment docs.

csweichel · 2022-01-10T08:34:58Z

We should find a more descriptive name for this, e.g.

installation-telemetry
telemetry-client

I'm don't like any of them particularly, but can't think of a better one right now. telemetry by itself is too vague - it's unclear who's doing what with telemetry. This will likely cause naming conflicts further down the road.

mrsimonemms · 2022-01-10T08:45:23Z

@csweichel fair point. I'll go for installation-telemetry unless anyone has any better ideas as to what we might want to call it

These are deleted after 60 seconds which causes the uninstallation script not to delete anything that comes after jobs in the ordering, such as cron jobs.

mrsimonemms · 2022-01-10T16:38:10Z

/werft run

👍 started the job as gitpod-build-sje-telemetry-client.12

mrsimonemms · 2022-01-12T10:05:52Z

/hold I'm going to build with the Segment token in now I know how to do that

mrsimonemms · 2022-01-12T12:17:18Z

/unhold

corneliusludmann

/hold
/approve
/lgtm

Added hold in case you want to remove the lib build. However, shouldn't be a blocker. We can remove it later as well.

corneliusludmann · 2022-01-12T13:43:54Z

components/installation-telemetry/BUILD.yaml

+# See License-AGPL.txt in the project root for license information.
+
+packages:
+  - name: lib


Sounds odd. I tried it here https://werft.gitpod-dev.com/job/gitpod-build-clu-test.28 and it works even without lib. Was maybe another issue you had?

installer/pkg/components/gitpod/cronjob.go

roboquat · 2022-01-12T13:48:23Z

LGTM label has been added.

Git tree hash: 38a17bab6ea83a7b146467d37285078060fcb974

This will not actually send telemetry data as the database call is hard-coded to always return false. Also configures installation telemetry to be sent periodically in the Installer

corneliusludmann · 2022-01-12T14:01:52Z

/lgtm

roboquat · 2022-01-12T14:01:55Z

LGTM label has been added.

Git tree hash: c631e70847cdd4aea19175644f3764cca9edba27

roboquat · 2022-01-12T14:01:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: corneliusludmann

Associated issue: #7499

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [corneliusludmann]
~~installer/OWNERS~~ [corneliusludmann]
~~installer/pkg/config/versions/OWNERS~~ [corneliusludmann]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mrsimonemms · 2022-01-12T14:30:30Z

/unhold

roboquat added do-not-merge/work-in-progress do-not-merge/release-note-label-needed size/XXL labels Jan 7, 2022

mrsimonemms force-pushed the sje/telemetry-client branch 2 times, most recently from 3a75f50 to aed07de Compare January 7, 2022 15:31

mrsimonemms force-pushed the sje/telemetry-client branch 5 times, most recently from 700978b to 2d145bf Compare January 7, 2022 16:03

mrsimonemms changed the title ~~[telemetry]: initial commit~~ [installation-telemetry]: initial commit Jan 10, 2022

mrsimonemms force-pushed the sje/telemetry-client branch 4 times, most recently from 8dece75 to da8746a Compare January 10, 2022 16:13

roboquat added the team: delivery Issue belongs to the self-hosted team label Jan 10, 2022

[installer]: remove jobs from the gitpod-app config map

eb2ce5d

These are deleted after 60 seconds which causes the uninstallation script not to delete anything that comes after jobs in the ordering, such as cron jobs.

mrsimonemms force-pushed the sje/telemetry-client branch from da8746a to 5857dbd Compare January 10, 2022 16:33

mrsimonemms force-pushed the sje/telemetry-client branch 5 times, most recently from 96af4ca to 27450a8 Compare January 10, 2022 17:21

roboquat added team: IDE team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team labels Jan 10, 2022

mrsimonemms requested a review from corneliusludmann January 11, 2022 12:14

mrsimonemms marked this pull request as draft January 12, 2022 10:05

roboquat added the do-not-merge/work-in-progress label Jan 12, 2022

roboquat added the do-not-merge/hold label Jan 12, 2022

mrsimonemms force-pushed the sje/telemetry-client branch 2 times, most recently from e813000 to 762c392 Compare January 12, 2022 11:06

mrsimonemms marked this pull request as ready for review January 12, 2022 11:07

roboquat removed the do-not-merge/work-in-progress label Jan 12, 2022

roboquat removed the do-not-merge/hold label Jan 12, 2022

corneliusludmann approved these changes Jan 12, 2022

View reviewed changes

roboquat added do-not-merge/hold lgtm labels Jan 12, 2022

roboquat added the approved label Jan 12, 2022

[installation-telemetry]: initial commit

717909a

This will not actually send telemetry data as the database call is hard-coded to always return false. Also configures installation telemetry to be sent periodically in the Installer

mrsimonemms force-pushed the sje/telemetry-client branch from 762c392 to 717909a Compare January 12, 2022 13:57

roboquat removed the lgtm label Jan 12, 2022

roboquat added the lgtm label Jan 12, 2022

roboquat removed the do-not-merge/hold label Jan 12, 2022

roboquat merged commit 2a451d6 into main Jan 12, 2022

roboquat deleted the sje/telemetry-client branch January 12, 2022 14:31

roboquat added deployed: workspace Workspace team change is running in production deployed: webapp Meta team change is running in production labels Jan 12, 2022

Pothulapati mentioned this pull request Apr 4, 2022

Epic: Telemetry #7498

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[installation-telemetry]: initial commit plus installer setup #7503

[installation-telemetry]: initial commit plus installer setup #7503

mrsimonemms commented Jan 7, 2022 •

edited

codecov bot commented Jan 7, 2022 •

edited

csweichel commented Jan 10, 2022

mrsimonemms commented Jan 10, 2022

mrsimonemms commented Jan 10, 2022 •

edited by werft-gitpod-dev-com bot

mrsimonemms commented Jan 12, 2022

mrsimonemms commented Jan 12, 2022

corneliusludmann left a comment

corneliusludmann Jan 12, 2022

roboquat commented Jan 12, 2022

corneliusludmann commented Jan 12, 2022

roboquat commented Jan 12, 2022

roboquat commented Jan 12, 2022

mrsimonemms commented Jan 12, 2022

[installation-telemetry]: initial commit plus installer setup #7503

[installation-telemetry]: initial commit plus installer setup #7503

Conversation

mrsimonemms commented Jan 7, 2022 • edited

Description

Related Issue(s)

How to test

Release Notes

Documentation

codecov bot commented Jan 7, 2022 • edited

Codecov Report

csweichel commented Jan 10, 2022

mrsimonemms commented Jan 10, 2022

mrsimonemms commented Jan 10, 2022 • edited by werft-gitpod-dev-com bot

mrsimonemms commented Jan 12, 2022

mrsimonemms commented Jan 12, 2022

corneliusludmann left a comment

Choose a reason for hiding this comment

corneliusludmann Jan 12, 2022

Choose a reason for hiding this comment

roboquat commented Jan 12, 2022

corneliusludmann commented Jan 12, 2022

roboquat commented Jan 12, 2022

roboquat commented Jan 12, 2022

mrsimonemms commented Jan 12, 2022

mrsimonemms commented Jan 7, 2022 •

edited

codecov bot commented Jan 7, 2022 •

edited

mrsimonemms commented Jan 10, 2022 •

edited by werft-gitpod-dev-com bot