Provide telemetry.json and document what's collected on the website

[ghost]

Is your feature request related to a problem? Please describe.
Users currently sometimes know there is telemetry, but they do not know which kind of telemetry.

Describe the solution you'd like
A clear message saying VSCode uses telemetry, what is sent, a consent button and a do not consent button. An optional panel in the titlebar showing the user what information is being sent, when and how.
E.G: a panel in the Help titlebar button, that when clicked, opens a pane, with a table, showing all the requests and what exactly they send, and how much requests have been sent the last week.

Describe alternatives you've considered
Show the user a summary of all the information they've sent, put together.

Additional context
I understand why telemetry can be useful for something as big as VSCode, but I believe there's 2 keys here missing: consent and clarification.

EDIT: use vscode's built-in telemetry API.

[DanTup]

We used to have our own flag for enabling analytics, but VS Code requested that we switch to using their flag. My understanding was that they wanted a single opt-in for the whole of VS Code (which you should get the first time you run Code) instead of extensions all popping up their own consent notifications.

The discussion was offline, but there is some info about how extensions handle this here:

• https://code.visualstudio.com/api/extension-guides/telemetry
• https://code.visualstudio.com/api/references/vscode-api#env.isTelemetryEnabled

VS Code updated the telemetry settings to be more explicit that this includes extensions:

I'm not sure what is best to do here. We could add our own prompt for Dart, but I feel this is not what VS Code wants. @isidorn do you have any advice here? I certainly don't want to appear to collect data without consent - but if it's not clear from VS Code's prompt that it's enabling for extensions maybe we should?

@aitor-gomila I agree it should be much clearer what is collected (probably a page on dartcode.org and clear links from the readme/etc.). In the meantime you can see anything that's collected in

Dart-Code/src/extension/analytics.ts

Lines 108 to 145 in 5e550c1

	public logExtensionStartup(timeInMS: number) {
	this.event(Category.Extension, EventAction.Activated).catch((e) => this.logger.info(`${e}`));
	this.time(Category.Extension, TimingVariable.Startup, timeInMS).catch((e) => this.logger.info(`${e}`));
	}
	public logExtensionRestart(timeInMS: number) {
	this.event(Category.Extension, EventAction.Restart).catch((e) => this.logger.info(`${e}`));
	this.time(Category.Extension, TimingVariable.Startup, timeInMS).catch((e) => this.logger.info(`${e}`));
	}
	public logExtensionShutdown(): PromiseLike<void> { return this.event(Category.Extension, EventAction.Deactivated); }
	public logSdkDetectionFailure() { this.event(Category.Extension, EventAction.SdkDetectionFailure).catch((e) => this.logger.info(`${e}`)); }
	public logError(description: string, fatal: boolean) { this.error(description, fatal).catch((e) => this.logger.info(`${e}`)); }
	public logAnalyzerStartupTime(timeInMS: number) { this.time(Category.Analyzer, TimingVariable.Startup, timeInMS).catch((e) => this.logger.info(`${e}`)); }
	public logDebugSessionDuration(debuggerType: string, timeInMS: number) { this.time(Category.Debugger, TimingVariable.SessionDuration, timeInMS, debuggerType).catch((e) => this.logger.info(`${e}`)); }
	public logAnalyzerFirstAnalysisTime(timeInMS: number) { this.time(Category.Analyzer, TimingVariable.FirstAnalysis, timeInMS).catch((e) => this.logger.info(`${e}`)); }
	public logDebuggerStart(debuggerType: string, runType: string, sdkDap: boolean) {
	const customData = {
	cd15: debuggerType,
	cd16: runType,
	cd18: sdkDap ? "SDK" : "Legacy",
	cd6: this.getDebuggerPreference(),
	};
	this.event(Category.Debugger, EventAction.Activated, customData).catch((e) => this.logger.info(`${e}`));
	}
	public logDebuggerRestart() { this.event(Category.Debugger, EventAction.Restart).catch((e) => this.logger.info(`${e}`)); }
	public logDebuggerHotReload() { this.event(Category.Debugger, EventAction.HotReload).catch((e) => this.logger.info(`${e}`)); }
	public logDebuggerOpenObservatory() { this.event(Category.Debugger, EventAction.OpenObservatory).catch((e) => this.logger.info(`${e}`)); }
	public logDebuggerOpenTimeline() { this.event(Category.Debugger, EventAction.OpenTimeline).catch((e) => this.logger.info(`${e}`)); }
	public logDebuggerOpenDevTools() { this.event(Category.Debugger, EventAction.OpenDevTools).catch((e) => this.logger.info(`${e}`)); }
	public logFlutterSurveyShown() { this.event(Category.FlutterSurvey, EventAction.Shown).catch((e) => this.logger.info(`${e}`)); }
	public logFlutterSurveyClicked() { this.event(Category.FlutterSurvey, EventAction.Clicked).catch((e) => this.logger.info(`${e}`)); }
	public logFlutterSurveyDismissed() { this.event(Category.FlutterSurvey, EventAction.Dismissed).catch((e) => this.logger.info(`${e}`)); }
	public logFlutterOutlineActivated() {
	if (this.hasLoggedFlutterOutline)
	return;
	this.hasLoggedFlutterOutline = true;
	this.event(Category.FlutterOutline, EventAction.Activated).catch((e) => this.logger.info(`${e}`));
	}
	public logCommand(command: EventCommand) { this.event(Category.Command, EventCommand[command]).catch((e) => this.logger.info(`${e}`)); }

. It's mostly version numbers and some info about which features are used (or which versions of features you use, such as new DAP vs Legacy). The data is sent to an account that only I have access to.

[ghost]

My bad!!! Oh my god. I wanted to create this issue in the vscode repo :p

I didn't EVEN know the dart extension collected any telemetry, so that is something to improve on, both your side but mostly VSCode.

Issue in VSCode repo

[isidorn]

@DanTup I suggest to keep the current behavior and keep this issue open to collect more user feedback. Thanks

[DanTup]

My bad!!! Oh my god. I wanted to create this issue in the vscode repo :p

Ah, maybe that makes a little more sense :-)

I think it's fair to say it should be clearer for this extension in particular though. I do also have an open issue (#4377) to switch to VS Code's new telemetry classes. Perhaps VS Code could record what's being sent through that and make it accessible to the user. That way a user could see all data sent by all extensions (as long as they adopted these APIs) which I think would be nice and transparent.

[ghost]

Can I work on #4377? (Looking to get experience on VSCode's extension API for a personal project - a compatibility layer like coc.nvim so other IDEs can use VSCode extensions) :)

[DanTup]

@aitor-gomila absolutely! There's info in https://github.com/Dart-Code/Dart-Code/blob/master/CONTRIBUTING.md about how to set up to run/dev the extension (it's mostly just clone, npm install, open in VS Code and press F5).

Feel free to ping me there if you get stuck or have any questions. Thanks!

[ghost]

It appears there's a flag that lets users see which telemetry is being sent. Still, without reading the docs or the --help menu, most people won't see this option, as there's not even a "consent to telemetry" dialog.

This is not your problem, this is about VSCode, just wanted to mention it :)

[DanTup]

It appears there's a flag

Ah, nice - I'd missed that. Although it says "that VS Code produces" and mentions extensions creating their own telemetry.json. @isidorn if an extension uses TelemetryLogger, will it automatically be included in VS Code's generated file?

[isidorn]

Yes, I think it will. @lramos15 can confirm.

[lramos15]

It appears there's a flag

Ah, nice - I'd missed that. Although it says "that VS Code produces" and mentions extensions creating their own telemetry.json. @isidorn if an extension uses TelemetryLogger, will it automatically be included in VS Code's generated file?

You would have to manually generate your own. The new API will allow users to see live flowing telemetry via the Extension Telemetry output channel when their log level is trace, but we have no way of knowing a static list of every telemetry event you're sending as you could be using runtime calculated names.

[DanTup]

Oh, I see. From the description I thought it meant VS Code would log all telemetry it is sending in that session, but actually it's just a file describing what's collected.

It sounds like the Extension Telemetry output channel noted above is more like what I was expecting, thanks!

[DanTup]

@aitor-gomila since we already have #4377 about adopting the new APIs, I'm going to rename this issue back to be for providing a telemetry.json and documenting what's sent on the website. I'll do this after #4377 is complete, since I might see if we generate this automatically somehow to avoid it getting out-of-date with the actual code.

[ghost]

Okay! Afterwards, I want to work on a visual tool that lets you see what data is collected visually. But I need to work on this PR first.

[DanTup]

The extension now includes telemetry.json and there's a page on the website containing info derived from that file (and some additional notes on opting out, etc.):

https://dartcode.org/docs/analytics/