Telemetry #247

omefire · May 2, 2016

Here is the related proposal: apache/cordova-discuss#43

This code relies on the insight package: https://www.npmjs.com/package/insight
However, due to a bug within insight, when the command prompt timeout expires, it won't automatically opt the user out, leading to the prompt being displayed again the next time cordova is run.

I have a patch to fix this issue here:
yeoman/insight#47

Use the patch to test this PR.

omefire · May 8, 2016

This code relies on the insight package: https://www.npmjs.com/package/insight
However, due to a bug within insight, when the command prompt timeout expires, it won't automatically opt the user out, leading to the prompt being displayed again the next time cordova is run.

I have a patch to fix this issue here:
yeoman/insight#47

Use the patch to test this PR.

LGTM

csantanapr · May 9, 2016

LGTM

LGTM!!

stevengill · May 10, 2016

LGTM!!

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

doc/cordova.txt

+    cordova-cli telemetry on
+    cordova-cli telemetry off
+    cordova-cli build --no-telemetry


These examples are not necessarily needed here as they are simple and not common scenarios - it will be nice to keep this output as small as it could be.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

doc/cordova.txt

@@ -23,6 +23,12 @@ Project Commands
    serve .............................. Run project with a local webserver
                                            (including prepare)

+Telemetry Commands
+    telemetry on ....................... Turn telemetry collection on


Sounds like telemetry is a "global command" and should go along with create & help. All details about telemetry command e.g. privacy notice, on/off should not be specified here. The goal of this top level help is to be succinct without going into details.

Also, we need to document these commands and options here: https://github.com/apache/cordova-cli/blob/master/doc/readme.md

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

doc/telemetry.txt

+    off ...................... Turns telemetry collection off  
+
+Details
+    A timed prompt asking the user to opt-in or out is displayed the first time cordova is run.


Can omit "or out"

It reinforces that they are making a choice, they have options.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

doc/telemetry.txt

+    A timed prompt asking the user to opt-in or out is displayed the first time cordova is run.
+    It lasts for 30 seconds, after which the user is automatically opted-out if he doesn't provide any answer.
+    In CI environments, the `CI` environment variable can be set, which will prevent the prompt from showing up.
+    Telemetry collection can also be turned off on a single command by using the `--no-telemetry-flag`.


I believe you meant --no-telemetry flag

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

doc/telemetry.txt

+    cordova-cli build --no-telemetry
+
+For details, see our privacy notice: https://cordova.apache.org/privacy-notice.html 
+and the proposal: https://github.com/cordova/cordova-discuss/pull/43


Best not to reference the pull request here which will get closed and the doc itself will eventually get updated.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

spec/cli.spec.js

+                cli(["node", "cordova", "-v"]).then(function() {
+                    expect(console.log.mostRecentCall.args[0]).toMatch(version);
+                    done();
+                }).fail(function() {


Can you elaborate why these tests had to be changed and made async?

They have changed because telemetry.showPrompt() is async, therefore the return of cli turned into async, and the tests had to be updated to reflect that.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/telemetry.js

+ * Has the user already answered the telemetry prompt? (thereby opting in or out?)
+ */
+function hasUserOptedInOrOut() {
+    return !(insight.optOut === undefined);


to test for undefined in javascript you have to use typeof - http://stackoverflow.com/questions/3390396/how-to-check-for-undefined-in-javascript

Quesiton: Why do we 'have' to use 'typeof' ? According to the same link, it seems like this way is fine as well, with the difference between the two being that one throws when the variable is undeclared.

the typeof === discussion is a matter of preference, I personally have never had an issue with
if (x) { // do something } so I can check for null at the same time as I typically set js vars to null, but I am a freak.
As long as you are sure that the value will never actually be null, this should be fine; otherwise change it to == for a simple truthy comparison.
There is however an anti-pattern that is not obvious here. A boolean is being used to hold 3 values, no-answer, user-opted-in, user-opted-out. When I see return !(insight.optOut === undefined); my immediate thought would be to correct it to return (insight.optOut !== undefined ) which is not exactly the same meaning.
I would prefer 2 bools in this case. Something like: if( wasPrompted && isOptedIn ) ...
... but again, preference.

undefined in JavaScript can be redefined to anything. typeof prevents that from having an effect

typeof insight.optOut !== "undefined"

> typeof null === 'undefined' < false

My concern is only for clarity of intent. If someone decides to redefine undefined, I would think things will break all over.

If undefined was to be redefined, that wouldn't be our biggest concern. Plus, modern webviews/browsers don't allow that anymore.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/cli.js

            });
-
-            if (someChecksFailed) throw new CordovaError('Some of requirements check failed');
-        }).done();


Looks like you end up removing "done()" - does that not cause us to lose exceptions?

Good point. This will end up requiring a rework of how the tests are structured, since calling done() will end the promise chain.

I've refactored and introduced done() into the main function. This required introducing a callback into the main/exported function for testing purposes.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/cli.js

+            if (shouldCollectTelemetry) {
+                telemetry.track(cmd, result.isFulfilled() ? 'successful' : 'unsuccessful');
+            }
+            return result;


why return result? - also, this is not being done consistently.

returning result is done for consistency's sake.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/cli.js

-        create();
+        var result = create();
+        return result.finally(function() {
+            if (shouldCollectTelemetry) {


Instead of duplicating this code - consider the caller taking the promise and calling "finally" and recording telemetry calling "done()"

As discussed offline, I'm going ahead with this refactoring. but it will require retesting the whole CLI as it affects everything. Looking at things from the benefits/costs perspective, I'm not sure if that's the best route, but let's do it ...

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/telemetry.js

+    insight.askPermission(msg, function (unused, optIn) {
+        var EOL = require('os').EOL;
+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us better cordova");


better = improve. Also end with a period.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/telemetry.js

+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us better cordova");
+        } else {
+            console.log(EOL + "You have been opted out of telemetry. To change this, run: cordova telemetry on");


End with a period.

nikhilkh · May 10, 2016

nikhilkh reviewed May 10, 2016

View changes

nikhilkh · May 10, 2016

src/cli.js

+        var result = cordova.raw.serve(port);
+        return result.finally(function() {
+            if(shouldCollectTelemetry) {
+                telemetry.track(cmd, result.isFulfilled() ? 'successful' : 'unsuccessful');


Looks like "track" itself checks if telemetry should should be collected - why should every calling point also check?

Related: apache/cordova-docs#599

omefire · May 10, 2016

Related: apache/cordova-docs#599

nikhilkh · May 11, 2016

nikhilkh reviewed May 11, 2016

View changes

nikhilkh · May 11, 2016

doc/cordova.txt

@@ -21,7 +23,7 @@ Project Commands
    run ................................ Run project
                                            (including prepare && compile)
    serve .............................. Run project with a local webserver
-                                            (including prepare)
+                                            (including prepare)                                          


Trailing spaces after (including prepare).

nikhilkh · May 11, 2016

nikhilkh reviewed May 11, 2016

View changes

nikhilkh · May 11, 2016

doc/cordova.txt

@@ -5,6 +5,8 @@ Synopsis
 Global Commands
    create ............................. Create a project
    help ............................... Get help for a command
+    telemetry on ....................... Turn telemetry collection on


Like other commands - let's not list sub-commands like on and off.

ping ...

omefire · May 12, 2016

ping ...

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

csantanapr · May 12, 2016

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

Other CLIs that use lib (as a lib) would not want this code. Off by default is interesting, but I think this is cleaner, and it's done.

On May 12, 2016, at 3:12 AM, Carlos Santana notifications@github.com wrote:

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

purplecabbage · May 12, 2016

Other CLIs that use lib (as a lib) would not want this code. Off by default is interesting, but I think this is cleaner, and it's done.

On May 12, 2016, at 3:12 AM, Carlos Santana notifications@github.com wrote:

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

nikhilkh · May 12, 2016

nikhilkh reviewed May 12, 2016

View changes

nikhilkh · May 12, 2016

doc/cordova.txt

@@ -5,6 +5,7 @@ Synopsis
 Global Commands
    create ............................. Create a project
    help ............................... Get help for a command
+    telemetry .......................... Turn telemetry collection on or off


The CLI reference docs need an update.

Hmm, I thought it was generated from these files. Anyway, it's been updated.

They live in doc/readme.md.

Yes, I've already updated it and pushed to this PR. thanks.

nikhilkh · May 12, 2016

nikhilkh reviewed May 12, 2016

View changes

nikhilkh · May 12, 2016

src/telemetry.js

+    insight.askPermission(msg, function (unused, optIn) {
+        var EOL = require('os').EOL;
+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us improve cordova.");


Perhaps we should log opt-ins too. It will help us determine opt-in % easily.

We can derive this percentage, noo ?

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform > plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

I first wanted to go that route, but I then realized doing it in CLI minimizes polluting LIB for a functionality that's not intended to be used by downstream tools. It seems cleaner to handle it here, as mentioned by @purplecabbage

omefire · May 12, 2016

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform > plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

I first wanted to go that route, but I then realized doing it in CLI minimizes polluting LIB for a functionality that's not intended to be used by downstream tools. It seems cleaner to handle it here, as mentioned by @purplecabbage

Sounds good to me to proceed then, it was a random idea

csantanapr · May 12, 2016

Sounds good to me to proceed then, it was a random idea

LGTM

nikhilkh · May 12, 2016

LGTM

Was there ever a specific Jira issue for adding telemetry? I don't see it.
I think we need to revisit some of the output/prompt formatting, here is what I just saw:

dline2@1.0.1, run-async@0.1.0, lodash@3.10.1)
/usr/local/bin/cordova -> /usr/local/lib/node_modules/cordova/bin/cordova
? May Cordova anonymously report usage statistics to improve the tool over time? Yes
(Y/n) Yrdova anonymously report usage
Thanks for opting into telemetry to help us improve cordova.
6.2.1-dev (cordova-lib@6.2.1-dev)
jessem-osx:cordova-cli jessem$

Multiple overlapping lines, could be a bash issue as well.

purplecabbage · May 14, 2016

Was there ever a specific Jira issue for adding telemetry? I don't see it.
I think we need to revisit some of the output/prompt formatting, here is what I just saw:

dline2@1.0.1, run-async@0.1.0, lodash@3.10.1)
/usr/local/bin/cordova -> /usr/local/lib/node_modules/cordova/bin/cordova
? May Cordova anonymously report usage statistics to improve the tool over time? Yes
(Y/n) Yrdova anonymously report usage
Thanks for opting into telemetry to help us improve cordova.
6.2.1-dev (cordova-lib@6.2.1-dev)
jessem-osx:cordova-cli jessem$

Multiple overlapping lines, could be a bash issue as well.

@omefire Can you please take a look into this?

nikhilkh · May 16, 2016

@omefire Can you please take a look into this?

@purplecabbage, I didn't create a JIRA issue to track the telemetry work, I should have. It was an oversight on my part. My bad.

I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your issue.
What OS are you using ? I suspect this could be an issue with bash. I checked and couldn't find any such issue reported for inquirer, the underlying package that handles this.

omefire · May 16, 2016

@purplecabbage, I didn't create a JIRA issue to track the telemetry work, I should have. It was an oversight on my part. My bad.

I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your issue.
What OS are you using ? I suspect this could be an issue with bash. I checked and couldn't find any such issue reported for inquirer, the underlying package that handles this.

I am using windows 10, but I typically use the git-bash shell which may be
why we aren't seeing the same things.

@purplecabbage
risingj.com

On Mon, May 16, 2016 at 10:52 AM, Omar Mefire notifications@github.com
wrote:

@purplecabbage https://github.com/purplecabbage, I didn't create a JIRA
issue to track the telemetry work, I should have. My bad.
I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your
issue.
What OS are you using ? I suspect this could be an issue with bash. I
checked and couldn't find any such issue reported for inquirer, the
underlying package that handles this.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#247 (comment)

purplecabbage · May 16, 2016

I am using windows 10, but I typically use the git-bash shell which may be
why we aren't seeing the same things.

@purplecabbage
risingj.com

On Mon, May 16, 2016 at 10:52 AM, Omar Mefire notifications@github.com
wrote:

@purplecabbage https://github.com/purplecabbage, I didn't create a JIRA
issue to track the telemetry work, I should have. My bad.
I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your
issue.
What OS are you using ? I suspect this could be an issue with bash. I
checked and couldn't find any such issue reported for inquirer, the
underlying package that handles this.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#247 (comment)

@purplecabbage , I can't repro this with git-bash shell on Win10 because I'm running into another issue: nodejs/node#3006

The gist of my issue is that the process is not recognized to be running as TTY, which leads to insight ending the process and opting out:

Furthermore, commenting that code out, leads to yet another error: SBoudrias/Inquirer.js#290

Insight, which we depend on doesn't play well with git-bash/MinGW.

omefire · May 16, 2016

@purplecabbage , I can't repro this with git-bash shell on Win10 because I'm running into another issue: nodejs/node#3006

The gist of my issue is that the process is not recognized to be running as TTY, which leads to insight ending the process and opting out:

Furthermore, commenting that code out, leads to yet another error: SBoudrias/Inquirer.js#290

Insight, which we depend on doesn't play well with git-bash/MinGW.

@purplecabbage , could you please file a JIRA issue for what you've encountered ?
I just did the same for the issues I just mentioned: https://issues.apache.org/jira/browse/CB-11276

omefire · May 16, 2016

@purplecabbage , could you please file a JIRA issue for what you've encountered ?
I just did the same for the issues I just mentioned: https://issues.apache.org/jira/browse/CB-11276

Telemetry: Update README.md documentation

304a9c3

asfgit closed this in 94190b8 May 12, 2016

apache/cordova-cli

Join GitHub today

Telemetry #247

Conversation

omefire commented May 2, 2016 • edited Edited 1 time omefire edited May 9, 2016 (most recent)

This comment has been minimized.

omefire commented May 8, 2016 • edited Edited 1 time omefire edited May 9, 2016 (most recent)

This comment has been minimized.

csantanapr commented May 9, 2016

This comment has been minimized.

stevengill commented May 10, 2016

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

omefire May 10, 2016 • edited Edited 1 time omefire edited May 10, 2016 (most recent)

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

omefire May 11, 2016 • edited Edited 1 time omefire edited May 11, 2016 (most recent)

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 10, 2016 View changes

This comment has been minimized.

This comment has been minimized.

omefire commented May 10, 2016

nikhilkh reviewed May 11, 2016 View changes

This comment has been minimized.

nikhilkh reviewed May 11, 2016 View changes

This comment has been minimized.

This comment has been minimized.

omefire commented May 12, 2016

This comment has been minimized.

csantanapr commented May 12, 2016

This comment has been minimized.

purplecabbage commented May 12, 2016

nikhilkh reviewed May 12, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

omefire May 12, 2016 • edited Edited 1 time omefire edited May 12, 2016 (most recent)

nikhilkh reviewed May 12, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

omefire commented May 12, 2016 • edited Edited 1 time omefire edited May 12, 2016 (most recent)

This comment has been minimized.

omefire commented May 2, 2016 •

edited

Edited 1 time

omefire edited May 9, 2016 (most recent)

omefire commented May 8, 2016 •

edited

Edited 1 time

omefire edited May 9, 2016 (most recent)

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

omefire May 10, 2016 •

edited

Edited 1 time

omefire edited May 10, 2016 (most recent)

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

omefire May 11, 2016 •

edited

Edited 1 time

omefire edited May 11, 2016 (most recent)

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 10, 2016

View changes

nikhilkh reviewed May 11, 2016

View changes

nikhilkh reviewed May 11, 2016

View changes

nikhilkh reviewed May 12, 2016

View changes

omefire May 12, 2016 •

edited

Edited 1 time

omefire edited May 12, 2016 (most recent)

nikhilkh reviewed May 12, 2016

View changes

omefire commented May 12, 2016 •

edited

Edited 1 time

omefire edited May 12, 2016 (most recent)