global: setup automatic updates and releases #8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jorikvankemenade
added this to Pending review
in Elasticsearch 7 Sprint (July 22 - Aug 2)
jorikvankemenade
force-pushed
the
feature/automatic-release
branch
from
8f4afc9
to
1a0cc45
Compare
slint
reviewed
Jul 25, 2019
|
@slint I have applied your suggested changes. I am not sure about your comment about the deploy, but maybe we can quickly discuss that offline. You also linked |
jorikvankemenade
force-pushed
the
feature/automatic-release
branch
2 times, most recently
from
8a9710e
to
ff25d1b
Compare
Make-Data-Count uses a non up-to-date version of the COUNTER-Robots
list. However, the differentiation between robots and machines is nice.
To make sure that we have the latest list, but with differentiation we
diff the COUNTER-Robots robots.txt with the machines.txt and
unclassified.txt from the Make-Data-Count repository. If the resulting
classification contains errors, than this must be fixed by opening a PR
at either source repository.
Testruns on Python 2.7 had the following error during pip isntall:
ERROR: pytest-cov 2.7.1 has requirement pytest>=3.6,
but you'll have pytest 3.3.2 which is incompatible.
This caused problems during the test. Bumping the pytest version
resolves this.
jorikvankemenade
force-pushed
the
feature/automatic-release
branch
from
d32c432
to
ac1edbf
Compare
|
@slint I squashed my changes. Please let me know if I messed up the format or anything. A gentle reminder that we need to change the secret in |
slint
approved these changes
Jul 26, 2019
jorikvankemenade
force-pushed
the
feature/automatic-release
branch
3 times, most recently
from
1959632
to
60d1da1
Compare
jorikvankemenade
moved this from Pending review
to Pending merge
in Elasticsearch 7 Sprint (July 22 - Aug 2)
jirikuncar
reviewed
Aug 6, 2019
For the automatic deploy of cron triggered builds we added two scripts and changed the layout of the travis jobs. The scripts we added: run-update.sh triggers the update script for the user-agant lists. If an update is detect the changeg user-agent lists are commited. We also update the version of the package and create a seperate, tagged, commit for the release. push-update.sh adds the remote branch to the current environment (required for Travis builds) and pushes all changes including tags to the remote. For the Travis builds we decided to destinguish three different situations: 1. Regular build 2. Cron triggered build 3. Tagged build Regular builds run the test on three different Python versions: 2.7, 3.5 and 3.6 Cron triggered jobs only check if there are updates and do a simple test before pushing. Tagged builds first run the Test stage, similar to a regular build. After succes an addtional stage is triggered, that tests and creates the release.
jorikvankemenade
force-pushed
the
feature/automatic-release
branch
from
60d1da1
to
6f0acd9
Compare
topless
force-pushed
the
feature/automatic-release
branch
2 times, most recently
from
64d004e
to
8b8aff4
Compare
- closes inveniosoftware#9
topless
force-pushed
the
feature/automatic-release
branch
from
8b8aff4
to
7d0ee59
Compare
topless
approved these changes
Mar 10, 2020
|
I'm changing the "update" Travis stage to do the The reason is that the GH token would have to belong to one of the |
slint
moved this from Pending review
to Todo
in Sprint Week 10-11 (2020) - V3.3 Release
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces two major changes: the update script and the Travis builds.
Update script
We use the lists from Making Data Count for distinguishing user agents that are either robots or machines. These lists are a manually curated division of the lists from Counter Robots. The problem is that the original list is updated more often than the manually curated one. To mitigate this we updated the update script to always use the latest “Counter Robots” version, and use the “Making Data Count” lists for curating the
robots.txt.Travis builds
Since we can automatically sync with the external libraries it is also nice to automatically release an updated version of the user-agents list. This is done by creating some additional build stages in Travis that can be used for automatically releasing updated versions.
This PR distinguishes three kinds of builds:
Regular build
For the regular build nothing changes. It runs the tests for Python versions 2.7, 3.5 and 3.6.
Cron triggered build
We can set a
cronbuild in Travis that triggers every month. This build will run theupdate_script. If changes are detected the build is tested and if this is successful the changes are committed, tagged and pushed. This will automatically cause a Tagged build.Tagged build
A tagged build is like a regular build, but with an additional stage. If the
teststage is successful a deploy stage will run that will create a deploy in pypi. This allows us to release builds manually, by tagging a release, or via the cron triggered builds.Before merging
Before we merge this PR we must update the secret in
.travis.yml. This should be set to the Github token that we have for Invenio i.e.GH_TOKEN=<token>.After merging
After we have merged this PR we should enable a monthly cron build on the master branch. This will check monthly if there are any updates. If there are updates we will release a new version.
Closes: #7