-
Notifications
You must be signed in to change notification settings - Fork 811
/
RELEASING.TXT
211 lines (140 loc) · 7.74 KB
/
RELEASING.TXT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
0-a) Technical pre-conditions for release.
Overview
Code, schema, examples and supporting documentation should be in a stable state.
Pay particular attention to release-specific structures and docs/releases.html
(e.g. no stray '2.x', '2.*' etc. in hidden markup or tables)
In sdoapp.py we declare the current version. This should match the number
chosen in /docs/releases.html (check markup assigns an HTML ID too).
The following steps assume a general healthy freeze (tested / QA as below).
in sdoapp.py:
SCHEMA_VERSION=2.2 # or 2.x, as above.
in sdoapp.py:
releaselog = { "2.0": "2015-05-13", "2.1": "2015-08-06" }
becomes
releaselog = { "2.0": "2015-05-13", "2.1": "2015-08-06", "2.2": "2015-11-05" }
... this should have a release date for the current release and all
previous releases that are archived under data/releases/{version}/*
TODO: Add a unit test to ensure this.
TODO: Re-order this document to avoid forward references.
Under data/releases/{version}/ we need a versioned-snapshot structure:
e.g.
data/releases/2.1
data/releases/2.1/all-layers.nq
data/releases/2.1/core.nq
data/releases/2.1/ext-auto.nq
data/releases/2.1/ext-bib.nq
data/releases/2.1/README.md
data/releases/2.1/schema-all.html
data/releases/2.1/schema.nt
data/releases/2.1/schema.rdfa
Of these,
* schema.rdfa is a frozen one-time copy of data/schema.rdfa
* schema-all.html is generated from this via running the Web app and
capturing via curl of /version/build-latest,
e.g. http://localhost:8080/version/build-latest
when served locally via "dev_appserver ."
For now, you need to edit this html to remove webschemas.org from heading section.
* To create .nt, .nq, etc files use scripts/buildreleasefiles.sh
* README.md should be copied/updated accordingly. TODO: simplify this file.
Example usage:
First copy snapshot into place:
* cp data/schema.rdfa data/releases/2.2/schema.rdfa
* Now the HTML snapshot itself. Assuming a local server (e.g. from 'dev_appserver .') we can do:
* curl http://localhost:8080/version/build-latest > data/releases/3.2/schema-all.html (taking care with the version number, of course)
or this, to use webschemas: curl https://webschemas.org/version/build-latest > data/releases/3.4/schema-all.html
* clean up the html including the bit that says you're in a developer view etc.
Then generate N-Triples etc. versions:
* Ensure that the EXTENSIONS variable at the top of the script 'scripts/buildreleasefiles.sh' contains all the extensions for this release.
* scripts/buildreleasefiles.sh 3.1 - Check listed extensions is correct update if not
* Run './scripts/buildsitemap.py' make sure sitemap is up to date
* Inspect snapshot, add to git and push to repo.
* ls -l data/releases/2.2/
* git add data/releases/2.2/*
* git push
* update sites: e.g. ./scripts/updateAppEngine.sh to update sdo-phobos.appspot.com, webschemas.org
* Inspect /version/{x.y}/ on those sites manually; check for newly added terms etc.
* TODO: more QA and automation would be highly appropriate here.
TODO: list optional tasks post-launch (blog, email, refresh dydra sparql db etc.).
1) General preconditions / process and QA for release.
1-a) Steering Group have signed off on changes and release plan,
and no active and unresolved disputes in the Community Group.
1-b) All code is commited to appropriate release branch at Github (usually
configured as the current default github branch for /schemaorg/ project).
1-c) All tests pass.
It is best to test against a fresh checkout to avoid depending on uncommitted
files. The unit tests depend upon a local installation of Google AppEngine, e.g.
PYTHONPATH=/usr/local/google_appengine ./scripts/run_tests.py
To run the full unit tests (graph test of schema integrity) the 'rdflib'
python library is also needed: 'sudo easy_install rdflib' should do it.
Example transcript:
git clone https://github.com/schemaorg/schemaorg.git
Cloning into 'schemaorg'...
remote: Counting objects: 6375, done.
remote: Compressing objects: 100% (80/80), done.
remote: Total 6375 (delta 43), reused 0 (delta 0), pack-reused 6295
Receiving objects: 100% (6375/6375), 27.62 MiB | 1.01 MiB/s, done.
Resolving deltas: 100% (4074/4074), done.
Checking connectivity... done.
danbri-macbookpro2:tmp danbri$ cd schemaorg/
[...]
Ran 70 tests in 21.168s
OK (expected failures=3)
1-d) Latest candidate release branch is pushed to per-branch appspot instance
(e.g. sdo-phobos.appspot.com) and the generic unstable upstream site
(i.e. webschemas.org).
1-e) The manual QA page /docs/qa.html has been reviewed to ensure
representative pages of each type appear to be in a healthy state.
e.g. see http://webschemas.org/docs/qa.html
2.) Final checks
There is often some final tweaking, bugs, etc. When things settle down, a
final round of sanity-check tests is useful:
* Run again './scripts/buildsitemap.py' make sure sitemap is up to date
* Run again: PYTHONPATH=/usr/local/google_appengine ./scripts/run_tests.py
* Run again 'git status', confirm no uncommited files or changes.
* Run again: './scripts/updateAppEngine.sh' for development/upstream site sync.
3.) Previously: "Merge release branch into master"; now we develop in "master."
4.) Publish from a clean checkout of 'master' branch to schema.org.
Using gcloud app deploy
~~~~~~~~~~~~~~~~~~~~~~~
From 3.4.1 onwards use app deploy script (does a bit of housekeeping):
scripts/appdeploy.sh --project schemaorgae --no-promote --version=3-4-1 schemaorg.yaml
##### original deploy: gcloud app deploy --no-promote --project schemaorgae --version=3.4 schemaorg.yaml
Note: the --no-promote flage deploys the version but does not transfer traffic to it.
This can then be done when required via the appengine console, versions screen:
https://console.cloud.google.com/appengine/versions?project=schemaorgae&serviceId=default&versionssize=50
===========OLD============
Either by editing app.yaml to make sure 'application: schemaorgae' is active,
or by passing the name of the live appengine ID on the commandline.
===========OLD============
Currently @rvguha (as Steering Group chair) and @danbri (as Community Group chair and
webmaster / dogsbody) have write access to the corresponding appengine account.
===========OLD============
* e.g. update schemaorg.yaml then: appcfg.py update schemaorg.yaml -A schemaorgae
* Count to thirty. Log into appengine and you should see a new version.
* Send 5% of traffic to this version to catch any errors, then move to 100%.
===========OLD============
Note that appengine versions are visible using URLs with this pattern:
http://3-3.schemaorgae.appspot.com/docs/releases.html
5.) Related post-release tasks.
* Let the Community Group and Steering Group know.
* Blog post.
* There's a SPARQL database at DYDRA that can load NQuad dumps from
https://github.com/schemaorg/schemaorg/blob/sdo-phobos/data/releases/2.2/all-layers.nq
* Make a new per-release branch, reserve an appspot.com account for it.
* Configure GitHub with the new default branch.
See https://github.com/schemaorg/schemaorg/settings/branches
* Update issue tracker moving issues to new releases.
* Close issues that were addressed in this release.
* Make careful considered adjustment to expected release for the rest. Or
just tag them all for the next release.
* Update git release tags (TODO: we have no history of handling this yet).
For example, if we are in 'master' and 'git status' shows it in a clean state,
* git checkout -b sdo-deimos
* This tells us we can do:
* git push --set-upstream origin sdo-deimos
> To https://github.com/schemaorg/schemaorg.git
> * [new branch] sdo-deimos -> sdo-deimos
> Branch sdo-deimos set up to track remote branch sdo-deimos from origin.
TODO:
* clean up release tags
* see also: git log --pretty=fuller --since="2018-06-15"