report git sha on welcome request

[janl]

Closes #1309

[wohali]

Not 100% sure, but I think this breaks the release process.

[wohali]

unnecessary whitespace

[wohali]

nitpick: why? the way it is now does less work

[wohali]

I don't think this will t work when building from a release tarball, when we're not inside of git. Test case should be:

./configure -c
make dist
cd apache-couchdb-*
./configure -c
make couch

which I expect to fail.

[janl]

You’re right. lemme fix

[wohali]

@janl No, this is still broken. make dist should produce a tarball at anytime that includes the git sha.

Running my test case above on your branch, with no tags, I get this:

$ curl localhost:15984
{"couchdb":"Welcome","version":"2.2.0","git_sha":"","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

The git sha should be 'baked' into the result of make dist regardless of whether we're on a tag or not. This allows us to distinguish actual releases from RCs from merely snapshot builds.

[janl]

@wohali can you show all steps preceding that curl?

The way it should™ work is that it gets baked into rebar at make time, but I might be misreading things.

[wohali]

cd ~/couchdb
git reset --hard && git clean -ffdx
./configure -c
make dist
cd apache-couchdb-2.2.0-c8b0b0a2e
./configure -c
make couch
dev/run -n 1 >/dev/null 2>&1
sleep 10 && curl localhost:15984

[janl]

I tried a good while trying to get this going, but it is not going to be a trivial patch.

The problem is as follows: the git sha is only known at make dist time. At that point, make dist runs git archive to build the tarball. That means that only files that have been committed to the source tree make it into the source archive. In order to get the git sha in there, we’d have to make a commit. Or to ament the tarball with a new file that contains the git sha. The former doesn’t sound right to me as that would make the tarball dependent on a commit that will only exist on the release manager’s machine, plus those commits would never make sense to share. The second option might be worth investigating, but I ran out of time for this. Dropping from 2.2.0 unless someone picks this up.

[janl]

turns out this was easier after all. After the git archive, we do a bunch of things to the tarball, like copying the docs and fauxton builds into it, so I’m now appending git_sha to version.mk and have Makefile[.win] -> src/couch/rebar.config.script -> couch_server.erl pick it up:

{"couchdb":"Welcome","version":"2.2.0","git_sha":"88e9b5e25","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

[wohali]

@janl I will test this tomorrow.

What do you think about dropping the --short everywhere, so we get the full hash?

[janl]

@wohali no strong opinion either way. Feel free to amend my patch directly.

One thing I could see people ask is to not make this info available on a non-admin endpoint (I know), what's your thinking on that?

[wohali]

While reviewing this PR, I found that our fixes to improve the 2.1.2 RC process accidentally broke building "snapshot" dist tarballs from master (useful if you want to run bleeding edge releases, for instance). This, in turn, broke our Jenkins CI setup.

I've fixed the bug, and also added dirty detection (which marks a release as -dirty in the tag if it's been locally modified.)

Like the commits prior, this absolutely requires the repo to be inside of git. If we switch version control systems ever, we'll have to redo this part of the Makefile.

Examples

Building this PR

# ./configure && make dist
Done: apache-couchdb-2.2.0-794ea816c.tar.gz
# dev/run -n 1 >/dev/null &
# curl localhost:15984
{"couchdb":"Welcome","version":"2.2.0-794ea816c","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
# When building and running from the generated tarball:
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

I think that, perhaps, we want to keep the sha suffix on this one inside the tarball. It's not a tagged release of any sort. Then again, the Jenkins auto-built tarballs rarely make their way out of the developer community, and if they do, people use them at their own peril. The presence of the git_sha is all that's necessary to distinguish this from a real release.

This is a minor concern and I don't think blocks this PR.

Tagging this branch (locally!!!) 2.2.0-RC1:

# ./configure -c && make dist
Done: apache-couchdb-2.2.0-RC1.tar.gz
# dev/run -n 1 >/dev/null &
# curl localhost:15984
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
# When building and running from the generated tarball:
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

Everything is as expected: the tarball clearly says 2.2.0-RC1, but the build acts like a real 2.2.0 release, from tarball or from the free.

Tagging this branch (locally!!!) 2.2.0 without removing the 2.2.0-RC1 tag (mimicing a real release):

# ./configure -c && make dist
Done: apache-couchdb-2.2.0.tar.gz
# dev/run -n 1 >/dev/null &
# curl localhost:15984
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
# When building and running from the generated tarball:
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

Again, everything is as expected. Tarball and both built binaries match.

After making a trivial change to a file (dirty repo) and running the build again:

# ./configure -c && make dist
Done: apache-couchdb-2.2.0-dirty.tar.gz
# dev/run -n 1 >/dev/null &
# curl localhost:15984
{"couchdb":"Welcome","version":"2.2.0-dirty","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
# When building and running from the generated tarball:
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

This is fine for a first pass. It'd be nice if the asset inside the tarball reflected in the sha that it's been messed with, but that's a new ticket in my mind.

Removing all tags, then tagging this branch (locally!!!) 2.3.4-RC1:

# ./configure -c && make dist
Done: apache-couchdb-2.3.4-RC1.tar.gz
# dev/run -n 1 >/dev/null &
# curl localhost:15984
{"couchdb":"Welcome","version":"2.3.4","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}
# When building and running from the generated tarball:
{"couchdb":"Welcome","version":"2.2.0","git_sha":"794ea816c","features":["pluggable-storage-engines","scheduler"],"vendor":{"name":"The Apache Software Foundation"}}

This was an invalid test I did to satisfy my curiosity. The result is unusual, but expected: we still rely on the version.mk checked into git to be accurate inside a tarball. (Removing the need to have version.mk checked into the tree is something we could conceivably do later.)

@janl I thought about it, and no one in our repo is likely to collide hashes, and basically ~everyone uses short git hashes, so let's stick with it as is.

As for the info not being visible on a non-admin endpoint...prior art shows Cloudant exposes their internal build numbers on GET /... Honestly I don't think it's worth worrying about. If people are that worried they can block the entire GET / endpoint because they won't want to expose they're running 2.1.0 still! 😉

I still need to test this on Windows - which, incidentally, doesn't support make dist right now (bash dependency). That's up next.

[wohali]

And the windows build is broken, because the new hyper app doesn't support it /cc @kocolosk

I'll fix Windows issues in another ticket. For now this is +1

[wohali]

And the windows build is broken, because the new hyper app doesn't support it /cc @kocolosk

I'll fix Windows issues in another ticket. For now this is +1