Commits to dev branch should run e2e tests

[abrahamq]

This means after the s3 publish artifacts action in jenkins, we should try to run the e2e tests against dev.petabencana.id/cards/test123

Maybe there is a smart way to do this with webhooks rather than waiting after the s3 publish?

[matthewberryman]

If you can give me a webhook URL (and any JSON/query parameters/etc. needed) I can set this up using https://wiki.jenkins-ci.org/display/JENKINS/Post+build+task as probably the best option (would need to then get that shell script to wait until CloudFront expires the objects)

[abrahamq]

So there's a way to get s3 buckets to push notifications to SNS or to AWS lambdas? No webhook stuff from my searching. Haven't ever worked with webhooks so I probably don't know what to look for.

[matthewberryman]

You can but on particular object events; in this case there are multiple objects and you don't know which one will finish first, so I would use a post-build task in jenkins as that runs after all objects are put to s3; are you then wanting to run the testing task as per README.md using browserstack? If so, probably makes sense to just run that directly in jenkins (against the outputs in s3) rather than calling a web hook somewhere else to run tests.

[abrahamq]

Yea, that's what I was thinking. @talltom and I were also worried about running the tests against unpublished, or still cached in cloudfront code instead of the new-just-committed and built code.

[matthewberryman]

upload to s3 is part of the build process, so a post-build task will ensure it runs against s3, and I can use sleep to ensure it waits until the old content has expired in cloudfront.

[abrahamq]

kk, I'm writing a short npm script that does:
"e2e-dev-livesite": "protractor protractor_config/protractor.browserstack.android.conf.js",

so we can do npm run-script e2e-dev-livesite as part of the post-build task and that'll kick this off. Have I got the right idea?

[abrahamq]

mhmm I also need to store the browserstack username and password somewhere... any thoughts on how to go about this? I can store them as env variables in the jenkins config. Is that secure enough?

[matthewberryman]

I would set them up in the post-build script.

[abrahamq]

Okay, um... let me get browserstack working from jenkins and once I have it working you can take a look at it if you want/have a chance? I'm working off the petabencana-dev-e2e-testing project on jenkins, which is set up to overwrite the dev s3 bucket.

[matthewberryman]

Ok I'm seeing the following, which I guess is down to your test (and interplay with build products as uploaded), not something jenkins side.

Failures:
1) /cards/:id  expect title to be correct
  Message:
[31m    Expected 'PetaBencana.id' to contain 'Flood reports'.[0m
  Stack:
    Error: Failed expectation
        at Object.it (/var/lib/jenkins/workspace/petabencana-dev-e2e-testing/test/e2e/cards.spec.js:13:32)

[abrahamq]

Yeah this is a timing issue/ race condition. Aurelia doesn't have good support for protractor. I'll mess with the tests to make it work

…

On Apr 22, 2017 4:57 AM, "Matthew Berryman" ***@***.***> wrote: Ok I'm seeing the following, which I guess is down to your test (and interplay with build products as uploaded), not something jenkins side. Failures: 1) /cards/:id expect title to be correct Message: [31m Expected 'PetaBencana.id' to contain 'Flood reports'.[0m Stack: Error: Failed expectation at Object.it (/var/lib/jenkins/workspace/petabencana-dev-e2e-testing/test/e2e/cards.spec.js:13:32) — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#194 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIT_WznNqE9w3tSL2LghFgT4jqoJoJCmks5rycENgaJpZM4NErfz> .

[matthewberryman]

Noted. The sleep timer before your test script starts should be plenty long enough for objects to be updated in cloudfront, but sing out if that is an issue.

[tomasholderness]

Front-end dev moving to Angular.