Telemetry #247

omefire · 2016-05-02T20:43:10Z

Here is the related proposal: apache/cordova-discuss#43

omefire · 2016-05-08T21:39:15Z

This code relies on the insight package: https://www.npmjs.com/package/insight
However, due to a bug within insight, when the command prompt timeout expires, it won't automatically opt the user out, leading to the prompt being displayed again the next time cordova is run.

I have a patch to fix this issue here:
yeoman/insight#47

Use the patch to test this PR.

csantanapr · 2016-05-09T18:17:21Z

LGTM

stevengill · 2016-05-10T00:08:44Z

LGTM!!

nikhilkh · 2016-05-10T16:25:56Z

doc/cordova.txt

+    cordova-cli telemetry on
+    cordova-cli telemetry off
+    cordova-cli build --no-telemetry


These examples are not necessarily needed here as they are simple and not common scenarios - it will be nice to keep this output as small as it could be.

nikhilkh · 2016-05-10T16:27:30Z

doc/cordova.txt

@@ -23,6 +23,12 @@ Project Commands
    serve .............................. Run project with a local webserver
                                            (including prepare)

+Telemetry Commands
+    telemetry on ....................... Turn telemetry collection on


Sounds like telemetry is a "global command" and should go along with create & help. All details about telemetry command e.g. privacy notice, on/off should not be specified here. The goal of this top level help is to be succinct without going into details.

Also, we need to document these commands and options here: https://github.com/apache/cordova-cli/blob/master/doc/readme.md

nikhilkh · 2016-05-10T16:28:03Z

doc/telemetry.txt

+    off ...................... Turns telemetry collection off  
+
+Details
+    A timed prompt asking the user to opt-in or out is displayed the first time cordova is run.


Can omit "or out"

It reinforces that they are making a choice, they have options.

nikhilkh · 2016-05-10T16:29:28Z

doc/telemetry.txt

+    A timed prompt asking the user to opt-in or out is displayed the first time cordova is run.
+    It lasts for 30 seconds, after which the user is automatically opted-out if he doesn't provide any answer.
+    In CI environments, the `CI` environment variable can be set, which will prevent the prompt from showing up.
+    Telemetry collection can also be turned off on a single command by using the `--no-telemetry-flag`.


I believe you meant --no-telemetry flag

nikhilkh · 2016-05-10T16:31:06Z

doc/telemetry.txt

+    cordova-cli build --no-telemetry
+
+For details, see our privacy notice: https://cordova.apache.org/privacy-notice.html 
+and the proposal: https://github.com/cordova/cordova-discuss/pull/43


Best not to reference the pull request here which will get closed and the doc itself will eventually get updated.

nikhilkh · 2016-05-10T16:32:35Z

spec/cli.spec.js

+                cli(["node", "cordova", "-v"]).then(function() {
+                    expect(console.log.mostRecentCall.args[0]).toMatch(version);
+                    done();
+                }).fail(function() {


Can you elaborate why these tests had to be changed and made async?

They have changed because telemetry.showPrompt() is async, therefore the return of cli turned into async, and the tests had to be updated to reflect that.

nikhilkh · 2016-05-10T16:40:59Z

src/telemetry.js

+ * Has the user already answered the telemetry prompt? (thereby opting in or out?)
+ */
+function hasUserOptedInOrOut() {
+    return !(insight.optOut === undefined);


to test for undefined in javascript you have to use typeof - http://stackoverflow.com/questions/3390396/how-to-check-for-undefined-in-javascript

Quesiton: Why do we 'have' to use 'typeof' ? According to the same link, it seems like this way is fine as well, with the difference between the two being that one throws when the variable is undeclared.

the typeof === discussion is a matter of preference, I personally have never had an issue with
if (x) { // do something } so I can check for null at the same time as I typically set js vars to null, but I am a freak.
As long as you are sure that the value will never actually be null, this should be fine; otherwise change it to == for a simple truthy comparison.
There is however an anti-pattern that is not obvious here. A boolean is being used to hold 3 values, no-answer, user-opted-in, user-opted-out. When I see return !(insight.optOut === undefined); my immediate thought would be to correct it to return (insight.optOut !== undefined ) which is not exactly the same meaning.
I would prefer 2 bools in this case. Something like: if( wasPrompted && isOptedIn ) ...
... but again, preference.

undefined in JavaScript can be redefined to anything. typeof prevents that from having an effect

typeof insight.optOut !== "undefined"

> typeof null === 'undefined' < false

My concern is only for clarity of intent. If someone decides to redefine undefined, I would think things will break all over.

If undefined was to be redefined, that wouldn't be our biggest concern. Plus, modern webviews/browsers don't allow that anymore.

nikhilkh · 2016-05-10T16:48:32Z

src/cli.js

            });
-
-            if (someChecksFailed) throw new CordovaError('Some of requirements check failed');
-        }).done();


Looks like you end up removing "done()" - does that not cause us to lose exceptions?

Good point. This will end up requiring a rework of how the tests are structured, since calling done() will end the promise chain.

I've refactored and introduced done() into the main function. This required introducing a callback into the main/exported function for testing purposes.

nikhilkh · 2016-05-10T16:48:54Z

src/cli.js

+            if (shouldCollectTelemetry) {
+                telemetry.track(cmd, result.isFulfilled() ? 'successful' : 'unsuccessful');
+            }
+            return result;


why return result? - also, this is not being done consistently.

returning result is done for consistency's sake.

nikhilkh · 2016-05-10T16:49:52Z

src/cli.js

-        create();
+        var result = create();
+        return result.finally(function() {
+            if (shouldCollectTelemetry) {


Instead of duplicating this code - consider the caller taking the promise and calling "finally" and recording telemetry calling "done()"

As discussed offline, I'm going ahead with this refactoring. but it will require retesting the whole CLI as it affects everything. Looking at things from the benefits/costs perspective, I'm not sure if that's the best route, but let's do it ...

nikhilkh · 2016-05-10T16:50:53Z

src/telemetry.js

+    insight.askPermission(msg, function (unused, optIn) {
+        var EOL = require('os').EOL;
+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us better cordova");


better = improve. Also end with a period.

nikhilkh · 2016-05-10T16:52:27Z

src/telemetry.js

+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us better cordova");
+        } else {
+            console.log(EOL + "You have been opted out of telemetry. To change this, run: cordova telemetry on");


End with a period.

nikhilkh · 2016-05-10T16:57:21Z

src/cli.js

+        var result = cordova.raw.serve(port);
+        return result.finally(function() {
+            if(shouldCollectTelemetry) {
+                telemetry.track(cmd, result.isFulfilled() ? 'successful' : 'unsuccessful');


Looks like "track" itself checks if telemetry should should be collected - why should every calling point also check?

omefire · 2016-05-10T21:38:33Z

Related: apache/cordova-docs#599

nikhilkh · 2016-05-11T18:00:13Z

doc/cordova.txt

@@ -21,7 +23,7 @@ Project Commands
    run ................................ Run project
                                            (including prepare && compile)
    serve .............................. Run project with a local webserver
-                                            (including prepare)
+                                            (including prepare)                                          


Trailing spaces after (including prepare).

omefire · 2016-05-12T05:31:45Z

ping ...

csantanapr · 2016-05-12T10:12:19Z

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

purplecabbage · 2016-05-12T16:17:37Z

Other CLIs that use lib (as a lib) would not want this code. Off by default is interesting, but I think this is cleaner, and it's done.

On May 12, 2016, at 3:12 AM, Carlos Santana notifications@github.com wrote:

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

nikhilkh · 2016-05-12T16:55:59Z

doc/cordova.txt

@@ -5,6 +5,7 @@ Synopsis
 Global Commands
    create ............................. Create a project
    help ............................... Get help for a command
+    telemetry .......................... Turn telemetry collection on or off


The CLI reference docs need an update.

Hmm, I thought it was generated from these files. Anyway, it's been updated.

They live in doc/readme.md.

Yes, I've already updated it and pushed to this PR. thanks.

nikhilkh · 2016-05-12T16:58:10Z

src/telemetry.js

+    insight.askPermission(msg, function (unused, optIn) {
+        var EOL = require('os').EOL;
+        if (optIn) {
+            console.log(EOL + "Thanks for opting into telemetry to help us improve cordova.");


Perhaps we should log opt-ins too. It will help us determine opt-in % easily.

We can derive this percentage, noo ?

omefire · 2016-05-12T17:22:35Z

I have one question, maybe I missed it in discussion why is this implemented in CLI and not LIB

I think it make more sense to implement in the top level of API entry points in LIB like create platform > plugin etc.

In LIB will be disable by default
CLI will call API with new parameter to enable telemetry
CLI telemetry will
Be enable by default and CLI will offer user a CLI flag and env to disable telemetry

I first wanted to go that route, but I then realized doing it in CLI minimizes polluting LIB for a functionality that's not intended to be used by downstream tools. It seems cleaner to handle it here, as mentioned by @purplecabbage

csantanapr · 2016-05-12T17:53:13Z

Sounds good to me to proceed then, it was a random idea

…, More documentation, etc...

nikhilkh · 2016-05-12T19:31:58Z

LGTM

purplecabbage · 2016-05-14T00:31:09Z

Was there ever a specific Jira issue for adding telemetry? I don't see it.
I think we need to revisit some of the output/prompt formatting, here is what I just saw:

dline2@1.0.1, run-async@0.1.0, lodash@3.10.1)
/usr/local/bin/cordova -> /usr/local/lib/node_modules/cordova/bin/cordova
? May Cordova anonymously report usage statistics to improve the tool over time? Yes
(Y/n) Yrdova anonymously report usage
Thanks for opting into telemetry to help us improve cordova.
6.2.1-dev (cordova-lib@6.2.1-dev)
jessem-osx:cordova-cli jessem$

Multiple overlapping lines, could be a bash issue as well.

nikhilkh · 2016-05-16T17:45:53Z

@omefire Can you please take a look into this?

omefire · 2016-05-16T17:52:05Z

@purplecabbage, I didn't create a JIRA issue to track the telemetry work, I should have. It was an oversight on my part. My bad.

I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your issue.
What OS are you using ? I suspect this could be an issue with bash. I checked and couldn't find any such issue reported for inquirer, the underlying package that handles this.

purplecabbage · 2016-05-16T17:55:38Z

I am using windows 10, but I typically use the git-bash shell which may be
why we aren't seeing the same things.

@purplecabbage
risingj.com

On Mon, May 16, 2016 at 10:52 AM, Omar Mefire notifications@github.com
wrote:

@purplecabbage https://github.com/purplecabbage, I didn't create a JIRA
issue to track the telemetry work, I should have. My bad.
I've tested this on MAC(El Capitan) and Windows10, but couldn't repro your
issue.
What OS are you using ? I suspect this could be an issue with bash. I
checked and couldn't find any such issue reported for inquirer, the
underlying package that handles this.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#247 (comment)

omefire · 2016-05-16T19:27:18Z

@purplecabbage , I can't repro this with git-bash shell on Win10 because I'm running into another issue: nodejs/node#3006

The gist of my issue is that the process is not recognized to be running as TTY, which leads to insight ending the process and opting out:

Furthermore, commenting that code out, leads to yet another error: SBoudrias/Inquirer.js#290

Insight, which we depend on doesn't play well with git-bash/MinGW.

omefire · 2016-05-16T20:00:18Z

@purplecabbage , could you please file a JIRA issue for what you've encountered ?
I just did the same for the issues I just mentioned: https://issues.apache.org/jira/browse/CB-11276

omefire force-pushed the omefire:Telemetry branch 4 times, most recently from b3f0bab to 289619c May 8, 2016

omefire force-pushed the omefire:Telemetry branch 2 times, most recently from 2652b45 to 6f0c505 May 9, 2016

omefire force-pushed the omefire:Telemetry branch 3 times, most recently from f24b513 to 0e3c6de May 9, 2016

omefire force-pushed the omefire:Telemetry branch 2 times, most recently from cfb2838 to 5f0f839 May 10, 2016

nikhilkh reviewed May 10, 2016
View changes

omefire force-pushed the omefire:Telemetry branch from 44c26ea to 64a0c9e May 11, 2016

nikhilkh reviewed May 11, 2016
View changes

omefire force-pushed the omefire:Telemetry branch 4 times, most recently from 4676c01 to f5133de May 12, 2016

nikhilkh reviewed May 12, 2016
View changes

omefire force-pushed the omefire:Telemetry branch from f5133de to dcfb06a May 12, 2016

omefire force-pushed the omefire:Telemetry branch 2 times, most recently from bb1901e to f4e1298 May 12, 2016

omefire added 5 commits Apr 29, 2016

Telemetry: The Foundational work

eafb8b9

Telemetry: 'cordova telemetry on' && 'cordova telemetry off'

330dea1

Telemetry: Adding Unit Tests

55df898

Telemetry: CLI Documentation

50016c9

Telemetry: Refactor telemetry calls into a single spot, Change prompt…

Loading status checks…

39e5f17

…, More documentation, etc...

omefire force-pushed the omefire:Telemetry branch from f4e1298 to 39e5f17 May 12, 2016

Telemetry: Update README.md documentation

Loading status checks…

304a9c3

omefire force-pushed the omefire:Telemetry branch from adeb7b6 to 304a9c3 May 12, 2016

asfgit closed this in 94190b8 May 12, 2016

apache / cordova-cli

Telemetry #247

Telemetry #247

Conversation

omefire commented May 2, 2016 • edited

omefire commented May 8, 2016 • edited

csantanapr commented May 9, 2016

stevengill commented May 10, 2016

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 10, 2016 • edited Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 10, 2016 Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 10, 2016 Author Contributor

This comment has been minimized.

purplecabbage May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

purplecabbage May 10, 2016 Contributor

This comment has been minimized.

omefire May 11, 2016 Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 11, 2016 Author Contributor

This comment has been minimized.

omefire May 11, 2016 Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 11, 2016 Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

omefire May 11, 2016 • edited Author Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

This comment has been minimized.

nikhilkh May 10, 2016 Contributor

omefire commented May 10, 2016

This comment has been minimized.

nikhilkh May 11, 2016 Contributor

omefire commented May 12, 2016

csantanapr commented May 12, 2016

purplecabbage commented May 12, 2016

This comment has been minimized.

nikhilkh May 12, 2016 Contributor

This comment has been minimized.

omefire May 12, 2016 Author Contributor

This comment has been minimized.

nikhilkh May 12, 2016 Contributor

This comment has been minimized.

omefire May 12, 2016 • edited Author Contributor

This comment has been minimized.

nikhilkh May 12, 2016 Contributor

This comment has been minimized.

omefire May 12, 2016 Author Contributor

This comment has been minimized.

omefire May 12, 2016 Author Contributor

omefire commented May 12, 2016 • edited

csantanapr commented May 12, 2016

omefire commented May 2, 2016 •

edited

omefire commented May 8, 2016 •

edited

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

omefire May 10, 2016 •

edited

Author Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

omefire May 10, 2016
Author Contributor

nikhilkh May 10, 2016
Contributor

omefire May 10, 2016
Author Contributor

purplecabbage May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

purplecabbage May 10, 2016
Contributor

omefire May 11, 2016
Author Contributor

nikhilkh May 10, 2016
Contributor

omefire May 11, 2016
Author Contributor

omefire May 11, 2016
Author Contributor

nikhilkh May 10, 2016
Contributor

omefire May 11, 2016
Author Contributor

nikhilkh May 10, 2016
Contributor

omefire May 11, 2016 •

edited

Author Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 10, 2016
Contributor

nikhilkh May 11, 2016
Contributor

nikhilkh May 12, 2016
Contributor

omefire May 12, 2016
Author Contributor

nikhilkh May 12, 2016
Contributor

omefire May 12, 2016 •

edited

Author Contributor

nikhilkh May 12, 2016
Contributor

omefire May 12, 2016
Author Contributor

omefire May 12, 2016
Author Contributor

omefire commented May 12, 2016 •

edited

omefire commented May 16, 2016 •

edited