Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a travis suite to run the staging store tests #602

Merged
merged 7 commits into from Jul 4, 2016
Merged

Add a travis suite to run the staging store tests #602

merged 7 commits into from Jul 4, 2016

Conversation

come-maiz
Copy link
Contributor

LP: #1596072

@come-maiz
Copy link
Contributor Author

skip ci

@come-maiz come-maiz force-pushed the bug/1596072/travis_store_integration branch from b9ef49f to 1d6b461 Compare June 24, 2016 21:36
@come-maiz come-maiz mentioned this pull request Jun 24, 2016
Closed
@come-maiz come-maiz force-pushed the bug/1596072/travis_store_integration branch 2 times, most recently from ade55cc to 359f9b1 Compare June 27, 2016 21:14
@snappy-m-o
Copy link

Test jenkins

@come-maiz come-maiz force-pushed the bug/1596072/travis_store_integration branch from 359f9b1 to d808372 Compare June 29, 2016 17:36
@come-maiz
Run the travis tests on the ubuntu-core team against the staging store
351c516 
In order to do this, the store tests need to handle an empty password.
As travis only sets the encrypted environment variable for the team branches,
the personal PRs will run against the fake store.

LP: #1596072
@come-maiz
Run the integration tests with the debug flag
c8ea201
@come-maiz
Wait between registrations.
e211fcd
@come-maiz
Remove the debug.
50f24ba
@come-maiz come-maiz force-pushed the bug/1596072/travis_store_integration branch from d808372 to 50f24ba Compare July 2, 2016 01:53
@come-maiz
Copy link
Contributor Author

This is good to go. As good as it gets at least. I launched the travis tests many many times, and in the end found that it will fail if the two executions sync and run a registration within 5 seconds. There's nothing we can do about it right now. We could sign up a different user for each test, but there is no way to accept the store agreement with a script.
The error happened like one every 10 times, so a retrigger would get us to green. And in addition to that, this will happen only when the branch is in snapcore/snapcraft, which we don't do often. And then the execution travis does for new pushes to master is a single one, and unlikely to sync up with another run against staging.

@come-maiz
Copy link
Contributor Author

retest this please

1 similar comment
@come-maiz
Copy link
Contributor Author

retest this please

@sergiusens sergiusens merged commit 1fafdda into master Jul 4, 2016
@sergiusens sergiusens deleted the bug/1596072/travis_store_integration branch July 4, 2016 19:08
cprov
cprov reviewed Jul 4, 2016
View reviewed changes
integration_tests/__init__.py
self.run_snapcraft(['register', snap_name])
# sleep a few seconds to avoid hitting the store restriction on
# following registrations.
time.sleep(10)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, Store staging has 5 s rate-limit and production 60 s, a comment on the test would not hurt.

@come-maiz
Copy link
Contributor Author

@celso-providelo I didn't use the specific rate, and didn't comment much about it because you told me you were not comfortable making that info public :p
I will have to come back and refactor this when I enable the tests to run in production. I can add more comments then.

@cprov
Copy link
Contributor

cprov commented Jul 5, 2016

@ElOpio slow-clapping for me, huh ?! As we discussed before, when this delay starts impacting the full-suite run we will have to find a better way (no rate-limit for specific users/groups).

kalikiana pushed a commit to kalikiana/snapcraft that referenced this pull request Apr 6, 2017
@come-maiz @sergiusens
Add a travis suite to run the staging store tests (canonical#602)
b252a17 
In order to do this, the store tests need to handle an empty password.
As travis only sets the encrypted environment variable for the team branches,
the personal PRs will run against the fake store.

LP: #1596072
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@come-maiz @snappy-m-o @cprov @sergiusens