GPII-3333: Improve production tests to cover more functionality #718

        config: {
            configName: "gpii.tests.productionConfigTests.config",
            configPath: "%gpii-universal/tests/configs"
+        },
+        components: {
+            healthRequest: {


There are test definitions that work with development configs, which start both the local flow manager and the cloud locally, for all end points provided by the cloud. It would be beneficial for reducing the work load and future reusability if these test definitions can be reused for the production configs, where the cloud is served by docker containers.

For example, the existing tests for /settings GET is at tests/platform/cloud/SettingsGetTests.js, and for /settings PUT is at tests/platform/cloud/SettingsPutTests.js. The successful and failed cases for /access_tokens can also be found there.

@cindyli These are the changes thus far for using the test definitions in SettingsGetTests.js for both development and production configurations. It's still rough, but I'm pushing it to see if it works in CI.

…ction testing Refactoring and defining common test defintions based on existing development settings tests for use with production testing of the cloud based flow manager. Note: too early for use; committing to save the work thus far.

Re-factored and tweaked to have the capability of runnning the "setttings set" tests in either development or production mode.

Updated the developement tests for "put settings" to work with the changes introduced for running "get settings" tests in both development and production configuration.

Renamed to keep the test definitions separate for the "set settings" tests vs. the "put settings" tests.

Previous refactoring of test definitions and code for use in a production configuration requires local declaration of a development configuration for developement tests.

gpii-bot · 2019-01-09T15:44:51Z

CI job passed: https://ci.gpii.net/job/universal-tests/1429/

amb26 · 2019-03-21T15:38:38Z

tests/shared/UserLogonRequestTestDefs.js

@@ -12,9 +12,9 @@

 "use strict";

-var fluid = require("infusion"),
+var fluid  = require("infusion"),


Clownspacing has arisen

I protest my innocence, as I didn't directly touch this file. The extra space is due to merging with @the-t-in-rtf 's GPII-3619 branch before it went into master. See:

universal/tests/shared/UserLogonRequestTestDefs.js

Line 17 in ab5bee7

gpii = fluid.registerNamespace("gpii");

@cindyli noted a few similar issues above.

Perhaps the definition of "Clownspacing" is the act of copying other author's mistakes before they make a final cleanup for merging into master. I can live with that.

Please still remove these extra spaces at line 15 and 17. Thanks.

Used npm to run the production configuration tests with separate invocations of npm to add and remove the test gpii keys and prefsssafes.

gpii-bot · 2019-03-22T14:38:50Z

CI job passed: https://ci.gpii.net/job/universal-tests/1658/

gpii-bot · 2019-03-25T22:58:03Z

CI job failed: https://ci.gpii.net/job/universal-tests/1659/

Fixed missed file name.

gpii-bot · 2019-03-26T00:43:46Z

CI job passed: https://ci.gpii.net/job/universal-tests/1660/

…sts.

gpii-bot · 2019-03-27T20:51:44Z

CI job passed: https://ci.gpii.net/job/universal-tests/1669/

klown · 2019-03-27T20:58:18Z

@cindyli ready for review -- thanks.

amb26 · 2019-03-28T16:24:40Z

tests/shared/FlowManagerSettingsPutTestDefs.js

+gpii.tests.cloud.oauth2.settingsPut.updateSnapset.testLifecycleResponse = function (data, request) {
+    gpii.tests.cloud.oauth2.settingsPut.updateSnapset.testStatusCode(data, request);
+
+    var lifeCycle = JSON.parse(data);


Could you normalise capitalisation of "lifeCycle" to "lifecycle" throughout, consistent with usage in the core. Note that this function itself already uses "Lifecycle" in its name. Thanks!

gpii-bot · 2019-03-29T15:57:45Z

CI job passed: https://ci.gpii.net/job/universal-tests/1677/

gpii-bot · 2019-03-29T17:33:37Z

CI job passed: https://ci.gpii.net/job/universal-tests/1678/

cindyli · 2019-03-29T18:47:55Z

tests/production/DeleteUserSettingsFromCouchTests.js

+});
+
+// Tests for deleting test 'user' preferences from the database
+gpii.tests.productionConfigTesting.deleteTestRecordsFromDatabaseTests = [{


Sorry I should have thought about this much earlier rather than discovering it now. Long story in short, only deleting these records is not enough.

If you look into the CouchDB after running "npm run test:vagarantProduction", there are still some new records in there:

Open one of them, it looks like below and all these new records are in this structure:

These are access tokens assigned to OAuth clients. Refer to the documentation about OAuth authentication GPII supports. Every GET or PUT requests to /settings endpoint requires a valid access token first. This is when they are created. They need to be removed from the database too when the production test completes.

Followup, based on a teleconference with @cindyli

Add a new view to _design/views that will find these access tokens based on:
-- gpiiKey
-- clientCredentials
-- type (always "gpiiAppInstallationAuthorization")

(Note: The gpiiKey and clientCredentials are the same as those used in the test scenario that created the access_token in the first place.)

Also, this new view will be of use for GPII-3646 where we free up the database of all the expired access tokens created from day-to-day use of the system.

gpii-bot · 2019-04-02T18:33:38Z

CI job passed: https://ci.gpii.net/job/universal-tests/1684/

klown · 2019-04-02T18:41:56Z

The technique to remove the extra access tokens is okay, but could be improved. It uses the GPII Key IDs used in the tests to find a subset of access tokens in the database and remove that group. One ID, however, is used elsewhere, namely "carla"; so this will remove all of carla's access tokens, even ones created outside of these tests.

An improvement is to use the timestamps within the access tokens, relying on the fact that the "test" access tokens were all created within the last hour (practically speaking, within the last five minutes, or less).

Ideally, the code should remove all-and-only the access tokens that were created by the tests. But, because the database cleanup code is run in a separate node process, there is no way to store token references in memory and share them with the cleanup code. The references could be written to disk and read in by the deletion code, but that feels clunky.

klown · 2019-04-08T15:10:28Z

A followup to my comments above. I wrote:

Ideally, the code should remove all-and-only the access tokens that were created by the tests.

Note that the set-up code in vagrantCloudBasedContainers.sh also asks for an access token for "carla" as a way to make sure the flowmanager container is ready. That check also creates a "spurious" access token.

Why not use the flowmanager's /ready endpoint instead?

gpii-bot · 2019-04-09T15:19:01Z

CI job passed: https://ci.gpii.net/job/universal-tests/1698/

Improved technique to remove only the access tokens create by the tests.

gpii-bot · 2019-04-15T17:29:04Z

CI job passed: https://ci.gpii.net/job/universal-tests/1705/

klown · 2019-04-16T14:45:24Z

The latest technique is an improvement over the previous one.. It stores the access tokens that the tests create when making flowmanager /access requests, and deletes them when the tests using them are done.

A few access tokens are still missed, however:

There are a pair of login/logout tests that create access tokens as a side effect. It's not clear how to catch them since the request is /user/:gpiiKey/login and not a direct /access request,
As noted previously, the shell script vagrantCloudBasedContainers.sh tests the readiness of the database by issuing a flowmanager /access request, but that's well in advance of the test code, and could be replaced with a flowmanager /ready request (which does not create access tokens as a side effect).

- modified vagrantCloudBasedContainers.sh to use the flowmanager's "/ready" endpoint to check its readiness rather than using its "/access_token" endpoint -- avoids creation of extra access token, - modified the deletion of extra gpii keys and pref safes to also remove the access tokens for the login tests for "testUser1" and "nonexistent_gpii_key".

gpii-bot · 2019-04-22T19:29:01Z

CI job passed: https://ci.gpii.net/job/universal-tests/1723/

klown · 2019-04-22T19:58:13Z

These modifications take care of the three remaining extra access tokens created by the tests.

Ready for reivew, @cindyli

cindyli · 2019-04-23T13:43:14Z

Merged at 78a7e7a

klown added 12 commits November 9, 2018 18:11

GPII-3333: Improve production config tests to test more functionality.

214c5fa

Preliminary checkin to keep a copy of the work so far on github. Not ready for use.

GPII-3333: Fixed "health" flowmanager test.

ea8d1b2

Used "hostname" instead of host (and correct path). "Ready" test still fails.

GPII-3333: Fixed port configuration for flowmanager tests.

3b4d44a

GPII-3333: Added '/access_token' flowmanager test.

c227777

GPII-3333: Use 'testUser1' instead of 'carla' for /accestoken test.

3368e99

GPII-3333: Improved access token test by testing the response.

e8c9eb8

GPII-3333: Improved access token test to use variable for user id.

e1bb67f

GPII-3333: Added test for /%gpiiKey/settings/%device

d044422

GPII-3333: Tidied up the onComplete listener specs.

b2e26e3

GPII-3333: Merged upstream master GPII branch

5cb24c3

GPII-3333: Merged changes from branch 'GPII-3551'

5c060ef

GPII-3333: Updated production tests using fixes from GPII-3551

a6af970

- Use bug fix for cloud based flowmanger /ready endpoint (GPII-3551) - Also, revert docker command to run production tests

GPII-3333: Parameterized information for some tests.

515210c

- used user ID veriable and termMap instead of hard-coding ID, - used directModel argument to send() for passing access token.

GPII-3333: Improved path generation for life cycle settings test

8e6b35e

GPII-3333: Mergeed changes from GPII-3551

81acd12

klown mentioned this pull request Dec 12, 2018

GPII-3551: Improvements and bug fix for /ready and /health endpoint #713

Merged

cindyli reviewed Dec 12, 2018

View reviewed changes

klown added 7 commits December 13, 2018 09:57

GPII-3333: Merged upstream master GPII branch

2928a28

GPII-3333: Modified to handle set settings tests

54cbb66

Re-factored and tweaked to have the capability of runnning the "setttings set" tests in either development or production mode.

GPII-3333: Modified to ensure tests still run in development mode

8d67a9c

Updated the developement tests for "put settings" to work with the changes introduced for running "get settings" tests in both development and production configuration.

GPII-3333: Renamed test definitions file

c50fb98

Renamed to keep the test definitions separate for the "set settings" tests vs. the "put settings" tests.

GPII-3333: Added development configuration for "put settings" tests.

e748824

Previous refactoring of test definitions and code for use in a production configuration requires local declaration of a development configuration for developement tests.

GPII-3333: Removed "debugger;" statements.

ecafb34

GPII-3333: Merged changes from upstream master GPII branch.

05e0a47

amb26 reviewed Mar 21, 2019

View reviewed changes

GPII-3333: Separate execution of adding/removing test 'user' prefs

6fe7b0a

Used npm to run the production configuration tests with separate invocations of npm to add and remove the test gpii keys and prefsssafes.

GPII-3333: Used a grade to supplant use of gpii.test.push()

18a062c

GPII-3333: Used a grade to supplant use of gpii.test.push()

b490985

Fixed missed file name.

GPII-3333: Modified the documentation regarding running production te…

3264c1b

…sts.

amb26 reviewed Mar 28, 2019

View reviewed changes

GPII-3333: Tweaks to spelling and spacing

767e32e

GPII-3333: Merged changes from upstream master GPII branch

6c2b287

cindyli reviewed Mar 29, 2019

View reviewed changes

GPII-3333: Remove access tokens created by the tests.

6a1d3d0

GPII-3333: Merged changes from upstream master GPII branch

167ef58

GPII-3333: Remove extra access tokens added by tests

ae7226e

Improved technique to remove only the access tokens create by the tests.

cindyli merged commit 074a835 into GPII:master Apr 23, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GPII-3333: Improve production tests to cover more functionality #718

GPII-3333: Improve production tests to cover more functionality #718

klown commented Dec 7, 2018

gpii-bot commented Dec 7, 2018

gpii-bot commented Dec 10, 2018

gpii-bot commented Dec 11, 2018

gpii-bot commented Dec 11, 2018

cindyli Dec 12, 2018

klown Jan 9, 2019 •

edited

gpii-bot commented Jan 9, 2019

amb26 Mar 21, 2019

klown Mar 21, 2019

cindyli Mar 28, 2019

klown Mar 29, 2019

gpii-bot commented Mar 22, 2019

gpii-bot commented Mar 25, 2019

gpii-bot commented Mar 26, 2019

gpii-bot commented Mar 27, 2019

klown commented Mar 27, 2019

amb26 Mar 28, 2019

klown Mar 29, 2019

gpii-bot commented Mar 29, 2019

gpii-bot commented Mar 29, 2019

cindyli Mar 29, 2019 •

edited

klown Mar 29, 2019 •

edited

gpii-bot commented Apr 2, 2019

klown commented Apr 2, 2019 •

edited

klown commented Apr 8, 2019 •

edited

gpii-bot commented Apr 9, 2019

gpii-bot commented Apr 15, 2019

klown commented Apr 16, 2019 •

edited

gpii-bot commented Apr 22, 2019

klown commented Apr 22, 2019

cindyli commented Apr 23, 2019

GPII-3333: Improve production tests to cover more functionality #718

GPII-3333: Improve production tests to cover more functionality #718

Conversation

klown commented Dec 7, 2018

gpii-bot commented Dec 7, 2018

gpii-bot commented Dec 10, 2018

gpii-bot commented Dec 11, 2018

gpii-bot commented Dec 11, 2018

cindyli Dec 12, 2018

Choose a reason for hiding this comment

klown Jan 9, 2019 • edited

Choose a reason for hiding this comment

gpii-bot commented Jan 9, 2019

amb26 Mar 21, 2019

Choose a reason for hiding this comment

klown Mar 21, 2019

Choose a reason for hiding this comment

cindyli Mar 28, 2019

Choose a reason for hiding this comment

klown Mar 29, 2019

Choose a reason for hiding this comment

gpii-bot commented Mar 22, 2019

gpii-bot commented Mar 25, 2019

gpii-bot commented Mar 26, 2019

gpii-bot commented Mar 27, 2019

klown commented Mar 27, 2019

amb26 Mar 28, 2019

Choose a reason for hiding this comment

klown Mar 29, 2019

Choose a reason for hiding this comment

gpii-bot commented Mar 29, 2019

gpii-bot commented Mar 29, 2019

cindyli Mar 29, 2019 • edited

Choose a reason for hiding this comment

klown Mar 29, 2019 • edited

Choose a reason for hiding this comment

gpii-bot commented Apr 2, 2019

klown commented Apr 2, 2019 • edited

klown commented Apr 8, 2019 • edited

gpii-bot commented Apr 9, 2019

gpii-bot commented Apr 15, 2019

klown commented Apr 16, 2019 • edited

gpii-bot commented Apr 22, 2019

klown commented Apr 22, 2019

cindyli commented Apr 23, 2019

klown Jan 9, 2019 •

edited

cindyli Mar 29, 2019 •

edited

klown Mar 29, 2019 •

edited

klown commented Apr 2, 2019 •

edited

klown commented Apr 8, 2019 •

edited

klown commented Apr 16, 2019 •

edited