-
Notifications
You must be signed in to change notification settings - Fork 468
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
deps: bump librdkafka for proper gssapi support #2801
Conversation
This is failing to build for me locally with
However, I was able to get the tag built by CI, which seemed to work! i.e. I'm no longer getting the "No worthy mechs" error message (which was the original issue), which seems to indicate that GSSAPI is now working. My actual integration test is now failing at a new spot which I haven't taken the time to solve, so I can't definitively say it's working, but the error I'm receiving indicates it's simply a configuration error:
This seems good to merge, mod the fact that it seems to not work with |
I think I've fixed the linking error in CI, so I'm expecting this to finally go green! I'll look into your issue right after lunch, Sean! You're running on macOS, right? |
Yep, macOS. |
Whew, ok, got it working on macOS too. I don't think anyone's tried to cross-compile and statically link libsasl2 in like ten years. @sploiselle if you've still got that machine around, could you retest? I had to change enough things to get cross compilation working that it's possible I broke something. |
(If this goes green, though, I'm going to merge, to keep things moving!) |
We weren't working hard enough before to statically link in GSSAPI support. That is now fixed with the latest version of rdkafka, with help from sasl2-sys and krb5-src. We are rapidly becoming the Rust authentication experts.
commit 51fc3293fb41a08985df608a495b4924e0f07cb7 Merge: 259562a8 eac03e22 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 20:03:23 2020 -0400 Merge pull request #2875 from MaterializeInc/dependabot/cargo/thiserror-1.0.16 build(deps): bump thiserror from 1.0.15 to 1.0.16 commit 259562a8d117930df42183f48debd15c6889e73f Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Mon May 4 18:41:53 2020 -0400 Update CREATE SOURCE for Kinesis sources (#2886) Updates to creating Kinesis clients via credentials found by a chain provider, only using static access_key and secret_access_key strings if provided. commit 698509a648f864d3838a8ed8f4342a3e1d591715 Merge: 785ae3b4 21b32051 Author: Sean Loiselle <sean@materialize.io> Date: Mon May 4 18:40:11 2020 -0400 Merge pull request #2897 from sploiselle/auth-release-notes release notes: Kafka SSL & krb5 auth commit 785ae3b458074b20be8632eb72081112268d3d5c Merge: 6fdc4164 28a24856 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 18:15:20 2020 -0400 Merge pull request #2894 from benesch/ci-test ci: improve names of ssl/krb5 testdrive jobs commit 21b32051983ae7e48409d454ea13c7b19eaf964a Author: Sean Loiselle <himself@seanloiselle.com> Date: Mon May 4 18:11:49 2020 -0400 release notes: Kafka SSL & krb5 auth commit 6fdc4164dcd065cfd46135e28234aefc2be08e2e Merge: 39706dbf 4ec327eb Author: Sean Loiselle <sean@materialize.io> Date: Mon May 4 18:05:50 2020 -0400 Merge pull request #2887 from sploiselle/kafka-auth-docs docs: kafka auth docs commit 4ec327eb5cb00008cad690e2140ecffc300be6c4 Author: Sean Loiselle <himself@seanloiselle.com> Date: Mon May 4 14:04:58 2020 -0400 docs: kafka auth docs commit 39706dbf5402f7edbe3046c4cec2d69f8f13ee39 Merge: e81ff1fa c0bc03b7 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Mon May 4 18:02:11 2020 -0400 Merge pull request #2857 from justinj/generate_series Basic support for generate_series commit c0bc03b704c52768d0bec705469c4023e6cd90e3 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Mon May 4 11:37:06 2020 -0400 Add generate_series to docs commit 9865f126f4ff8162e1281dffc783794d2293b80e Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 30 22:20:19 2020 -0400 Basic support for generate_series This change required generalizing FlatMapUnary to take multiple arguments. This commit does not add support for either of the three-argument versions of generate_series. commit e81ff1fa3dbb78366d33bbb247b12504c5a66b23 Merge: 7f5948c8 172a8042 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 17:37:26 2020 -0400 Merge pull request #2893 from benesch/from-misparse sql: don't wildly misparse trailing comma in SELECT list commit 28a24856d3824ff9569410caab11aff4ee5876f4 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 17:26:43 2020 -0400 ci: improve names of ssl/krb5 testdrive jobs These are Kafka specific, so mark them as such in the CI configuration, per @sploiselle's suggestion. Also, add a bunch of missing dependencies to the deploy job. commit 7f5948c846479d9cce737efde995884f6c616e77 Merge: 08baf3d9 f9d4f8ed Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 17:21:57 2020 -0400 Merge pull request #2888 from benesch/test-ssl test/ssl: reorganize into dedicated directory commit 08baf3d9cd4abe232693342698c306aeb69876d6 Author: Brennan Vincent <brennan@materialize.io> Date: Mon May 4 16:50:19 2020 -0400 Don't hang the client on parse errors (#2882) commit 172a80427e561fcd7d8a7616a7f00d9b7f002ea2 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 16:40:57 2020 -0400 sql: don't wildly misparse trailing comma in SELECT list A query like SELECT 1, 2, FROM foo WHERE blah would be erroneously parsed as SELECT 1, 2, "from" AS foo WHERE blah which was baffling to both users and developers. Fix this by disallowing "FROM" as an identifier in any expression anywhere. This is technically a backwards-incompatible change, but almost surely no one was relying upon this behavior. Fix #2517. commit d75a3c5b3c8425d23bb7e68eefacf26460e92aaa Merge: 8f09ed9c 0c643fd0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 16:18:02 2020 -0400 Merge pull request #2890 from benesch/push-unstable ci: push "unstable" tags for Docker images commit 0c643fd07383d8d80a55ca85e9750b48a9a7f7a0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 16:12:48 2020 -0400 ci: push "unstable" tags for Docker images The "unstable" tag will refer to the latest image built from the tip of master. commit 8f09ed9c2a2dc1cf94e76040dc1f35ed0c42968a Merge: 4933aff5 5c771b1d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 16:05:18 2020 -0400 Merge pull request #2889 from benesch/passthru-buildkite-commit ci: pass through BUILDKITE_COMMIT environment variable commit 5c771b1da5e1a1126cbd78936a611b1339d19513 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 15:56:29 2020 -0400 ci: pass through BUILDKITE_COMMIT environment variable The test job was not properly triggering the deploy job due to this missing environment variable. commit f9d4f8edfb95d62c90e7056baa5dce1fe6440773 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 15:47:52 2020 -0400 test/ssl: reduce test script to essentials We only need to test that the connection is working properly. We can leave the other validations to the non-SSL tests. Should speed things up a bit. commit a9b5a33925f51463e4a98c6b8bf6394853f0b2a0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 13:32:06 2020 -0400 test/ssl: simplify compose configurations commit 4933aff5b1a859a22254155a2065d393939e6539 Author: Frank McSherry <fmcsherry@me.com> Date: Mon May 4 15:33:12 2020 -0400 Announce names of catalog entries (#2884) Catalog entries found on start up were not announced as they are in catalog_transact. This led to a mz_catalog_names view that did not match the registered catalog entries. Fixes #2883 commit 757dd525de459b5655a04a1039848fda852404b4 Author: Brennan Vincent <brennan@materialize.io> Date: Mon May 4 15:26:31 2020 -0400 Write materialized version and other relevant info to log file (#2879) commit c9f5835328e9ccb335f24f1c3828726be1743fe0 Merge: b8b5d278 ce4a27df Author: Brandon W Maister <bwm@materialize.io> Date: Mon May 4 14:40:42 2020 -0400 Merge pull request #2885 from quodlibetor/dashboard-docs Improve dashboard documentation commit ce4a27df249e7d8ed4cd3c776239d6af950df4f0 Author: Brandon W Maister <bwm@materialize.io> Date: Fri May 1 14:07:18 2020 -0400 Document changing the user docker runs as when mounting prometheus data commit 78afea66b6a162a8293a0e9e05fad7fd77c08c8e Author: Brandon W Maister <bwm@materialize.io> Date: Fri May 1 14:06:46 2020 -0400 Make monitoring docs explain why it's not designed for production commit b8b5d278b704b9b15ede5fc81025f03d3e8997f6 Merge: d329bc3a 48460d95 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 13:59:55 2020 -0400 Merge pull request #2881 from benesch/krb5-bin test/krb5: wrangle kerberos configuration commit 0ab99d61fd5b150ddc78a035689e4eb4519b2ffe Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 13:07:04 2020 -0400 testdrive: move SSL tests to dedicated directory This way they mirror the krb5 tests and keep all the SSL stuff together. commit 48460d95f6f66dd04b140a80fb38a8a9ef1b7891 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 12:36:51 2020 -0400 ci: rebuild docker-compose containers if necessary It's crazy that this isn't the docker-compose default, but it isn't. commit e13e4508049ecff0e5bce42413bfd64b329e0279 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 12:07:36 2020 -0400 test/krb5: have kdc log to stderr This is conventional for Docker containers. commit d9287f8272e8077b3004624667fbf227cfe7acc6 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 11:10:06 2020 -0400 test/krb5: use more conventional "kafka" service name Using "broker" is a strange Confluent-ism. commit 3c8ca49d313d4c2810db163ed950a09fe51d6ff0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 10:43:34 2020 -0400 test/krb5: cull unnecessary krb5 configurations commit 58ac67ab803f1d7ccbd47ca00904ba91aba3368d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 10:34:38 2020 -0400 test/krb5: ditch krb5.local network The default Docker Compose networking works ok. commit 837fd91df5cbf4814bf3becd2072c6cbdb6a63b9 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 10:29:51 2020 -0400 test/krb5: ditch explicit hostname configuration The default hostnames actually work just fine now. commit 9dfcf03eaafe3eb966608e3ad166d4ed98125e1c Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 10:26:58 2020 -0400 test/krb5: disable zookeeper client hostname canonicalization This means we're not reliant on the reverse DNS lookup of the zookeeper IP, which Docker Compose really wants to be in charge of. commit c362e95b443c93bc23e68943fd375a327685c51c Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 10:08:05 2020 -0400 test/krb5: cull some unnecessary environment variables commit 19c49417f7c0d76cf04df9d1ca3a2c21c587c477 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:45:17 2020 -0400 test/krb5: correct wait-for-it commit c20814f441344651aee12a14c6d70a6831a45808 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:43:23 2020 -0400 testdrive: move krb5 test to test/krb5 It's nice to have all the krb5 stuff in one place. It's not like int-krb5.td is really useful outside of this test harness anyway. commit 74cad884bb6522c8d9629ac1e220d6b1c866a6fd Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:38:23 2020 -0400 test/krb5: remove tmp/mzdata volumes We're only interested in a basic test of krb5 support, so we don't need to bother with more advanced testdrive features that require shared volumes. commit 93e3423abe9923765a34811a61da036580e102c0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:35:24 2020 -0400 test/krb5: use single JAAS config commit 95abe7a917ceab9e9c163be928696b1435b09197 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:29:52 2020 -0400 test/krb5: use standard mzcompose.yml naming Since we have a dedicated directory for the krb5 test (unlike the SSL test at the moment), we can get all the standard mzcompose niceties if we use the standard mzcompose.yml name. commit d329bc3a4bee047f4be5880364ecfdc6e300f6ed Merge: f6f54db2 aaa23f60 Author: Brandon W Maister <bwm@materialize.io> Date: Mon May 4 11:16:55 2020 -0400 Merge pull request #2838 from quodlibetor/dashboard-logging dashboard: Add a program that prefixes logs with the name of their service commit f6f54db2d408fd2f37bdc4067c7240c76d984e75 Merge: 5b8c9d99 41103168 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 11:13:51 2020 -0400 Merge pull request #2871 from benesch/krb5-bin materialized,testdrive: include kinit and kdestroy in Docker images commit 5b8c9d99d9777e4876fa461f6c837f213865df8d Merge: 9099a940 a4d77849 Author: Brandon W Maister <bwm@materialize.io> Date: Mon May 4 11:11:44 2020 -0400 Merge pull request #2859 from quodlibetor/release-updates Update release checklist commit 9099a94006d5c256b3397b07bc4a40053fff917c Author: Brennan Vincent <brennan@materialize.io> Date: Mon May 4 10:53:44 2020 -0400 Use RTEs to handle I/O errors in file sources (#2841) commit 623244ae7a24005beace833a5b157e19ae1ff0f5 Merge: 94077c5c ae2c0a1d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 09:55:09 2020 -0400 Merge pull request #2864 from benesch/deb-fixups deb: various fixups commit eac03e227469f4a186e5b1f076953d66ab855b40 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon May 4 13:52:14 2020 +0000 build(deps): bump thiserror from 1.0.15 to 1.0.16 Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.15 to 1.0.16. - [Release notes](https://github.com/dtolnay/thiserror/releases) - [Commits](https://github.com/dtolnay/thiserror/compare/1.0.15...1.0.16) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 94077c5cfcaa72aadc775181ec8020dab221e6ba Merge: 91aa59fd 35892b66 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 09:50:36 2020 -0400 Merge pull request #2878 from MaterializeInc/dependabot/cargo/serde_json-1.0.52 build(deps): bump serde_json from 1.0.51 to 1.0.52 commit 91aa59fdf55eb1bae0a2c7ff7df662c87f48aff5 Merge: 8b4ec9d1 8da772f8 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 09:50:27 2020 -0400 Merge pull request #2876 from MaterializeInc/dependabot/cargo/failure-0.1.8 build(deps): bump failure from 0.1.7 to 0.1.8 commit 8b4ec9d1e759cb70b3e872d4513d86a3a1505c98 Merge: 33d34a4b 5aa20f03 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 09:50:16 2020 -0400 Merge pull request #2874 from MaterializeInc/dependabot/cargo/tokio-0.2.20 build(deps): bump tokio from 0.2.19 to 0.2.20 commit 35892b668eeae3824dabb1c08efe65bf53e4ffc7 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon May 4 10:55:00 2020 +0000 build(deps): bump serde_json from 1.0.51 to 1.0.52 Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.51 to 1.0.52. - [Release notes](https://github.com/serde-rs/json/releases) - [Commits](https://github.com/serde-rs/json/compare/v1.0.51...v1.0.52) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 8da772f810df3c19363f087e06f7062cb0f12ea7 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon May 4 10:53:37 2020 +0000 build(deps): bump failure from 0.1.7 to 0.1.8 Bumps [failure](https://github.com/rust-lang-nursery/failure) from 0.1.7 to 0.1.8. - [Release notes](https://github.com/rust-lang-nursery/failure/releases) - [Changelog](https://github.com/rust-lang-nursery/failure/blob/master/RELEASES.md) - [Commits](https://github.com/rust-lang-nursery/failure/commits) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 5aa20f03811534a0723641ec9c55c047974e1782 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon May 4 10:52:30 2020 +0000 build(deps): bump tokio from 0.2.19 to 0.2.20 Bumps [tokio](https://github.com/tokio-rs/tokio) from 0.2.19 to 0.2.20. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](https://github.com/tokio-rs/tokio/compare/tokio-0.2.19...tokio-0.2.20) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 411031682d886ac164d47be1155ffb10625ba2f9 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun May 3 23:13:42 2020 -0400 materialized,testdrive: include kinit and kdestroy in Docker images materialized now statically links libkrb5, but an important part of Kerberos is the kinit binary, which obtains the initial ticket-granting ticket and is invoked automatically by librdkafka. This binary is normally supplied by the krb5-user package, or the equivalent on non-Debian systems, but since we're statically linking libkrb5 we need to provide our own statically-linked copy of kinit. It is in theory possible to use the distribution's kinit with our copy of libkrb5, but this can lead to all sorts of subtle problems if the distribution's kinit is from a significantly different version of libkrb5 than Materialize was linked against. For good measure, this commit includes kdestroy, since it is the complement of kinit. Additional binaries can be added if they prove to be necessary, but for now we'll limit it to these two to reduce the size of the resulting Docker image. (Each binary is ~2MB in size.) Fix #2869. commit 33d34a4b064059605160a2b813f2a51f72f40720 Merge: e6db87a0 271a34c2 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 01:09:58 2020 -0400 Merge pull request #2873 from benesch/sql-func-yml www/data: use YAML instead of JSON for SQL function data commit 271a34c2ad7de84a2f88178238b79d4fce5ff440 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon May 4 00:24:45 2020 -0400 www/data: use YAML instead of JSON for SQL function data I find YAML to be a good bit more human-readable and editable when multiline strings are involved. commit e6db87a04c1ddbefff82842282c3a7308695ae2c Merge: 62e9ac14 f459897f Author: Sean Loiselle <sean@materialize.io> Date: Mon May 4 00:24:12 2020 -0400 Merge pull request #2870 from sploiselle/kerberos-integration-test integration: test krb5 code commit f459897fdc6a9cfee1ff8770a53f735cd5f5d2ae Author: Sean Loiselle <himself@seanloiselle.com> Date: Sun May 3 22:42:03 2020 -0400 integration: test krb5 code commit 62e9ac14a01feed47c7595784f40657cef3f3150 Merge: af29b80a cb817279 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun May 3 19:59:02 2020 -0400 Merge pull request #2865 from benesch/mzbuild-no-pull mzbuild: don't pass --pull to docker build commit af29b80a867d7576ca42083420d956008f1dcb50 Merge: 2e22333f 3862af3e Author: Andi Wang <5522362+wangandi@users.noreply.github.com> Date: Sun May 3 11:57:04 2020 -0400 Merge pull request #2854 from wangandi/upsertavrofix Fix support for avro record in upsert. commit 2e22333f2cdd08a255ec993be872c0afa4a12d89 Merge: 1717197f 9d2ead38 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun May 3 02:16:36 2020 -0400 Merge pull request #2867 from benesch/ccsr-doc-bug ccsr: fix broken documentation comment commit 1717197f43937ccd76d5721781b6326d866ba9aa Author: Brandon W Maister <bwm@materialize.io> Date: Sat May 2 22:49:11 2020 -0400 Update LICENSE and materialized version post 0.2.1 (#2863) commit 9d2ead38a5dacdcfd12c1fc8a24bda77c430dfcb Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 22:35:59 2020 -0400 ccsr: fix broken documentation comment commit 43d745176b2890391bfa9d206b155801c01bf969 Merge: 073f6577 98b2fd2d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 22:31:12 2020 -0400 Merge pull request #2866 from benesch/remove-ore-macros ore: remove some ancient, dead macros commit 98b2fd2d8a268f23a889845761ccfaf826be7d7a Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 19:21:51 2020 -0400 ore: remove some ancient, dead macros These were used once, long ago, but have been dead code for months at least. They're not particularly idiomatic Rust, so just remove them. commit cb8172795784212742f053cb022f6669858b5108 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 19:03:10 2020 -0400 mzbuild: don't pass --pull to docker build If building an image where there is a FROM directive for an image that is not available on Docker Hub, `docker build --pull` will fail the entire build. The point of `--pull` was to pick up changes to base images on Docker Hub, but that's against the philosophy of mzbuild anyway, where images are immutable. Instead, we can refer to Ubuntu's immutable tags (bionic-20200403 instead of bionic, for example) and drop `--pull`. If we need to update to a newer version of Ubuntu, we simply update the version in the FROM line, and a normal `docker build`, without `--pull`, will fetch the new version. commit ae2c0a1d6f58eb43e21591a8b84dcf8c9e98ff51 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 18:49:32 2020 -0400 deb: quiet user creation adduser is very admonishing about the name we've chosen--a name we've chosen specifically to avoid the possibility of conflicting with an existing username--so jsut tell it to be quiet with the `--quiet` flag. commit bcc89981ca3d508ab67b08c95563435045f0ec77 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 18:46:31 2020 -0400 deb: fix systemd service thread configuration Explicitly specifying the desired number of threads is now required. commit f00a137e9c7731b098e954c18dacf5f990632f23 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 18:37:06 2020 -0400 deb: don't fail if systemd is missing This is modeled after how the nginx package handled systemd before it switched to dh_helper. [0] (Ideally we would use dh_helper, but that's not possible with cargo-deb.) [0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=713853;filename=0001-Switch-to-dh-systemd-making-packaging-easier-and-mor.patch;msg=5 commit d4d61c06d64585aba4ffc80aaf3b782357dc1b18 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 18:08:48 2020 -0400 dist: move into misc These scripts are not important enough to warrant a top-level directory. commit 92c9af3a4a05d55b302f29a377d82a3e7874732b Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 18:04:33 2020 -0400 ci: build correct debian package for release Building the stable Debian package requires changing some metadata for cargo-deb. commit a4d77849028f278d95d774a01d37fab9a0a94366 Author: Brandon W Maister <bwm@materialize.io> Date: Sat May 2 16:51:50 2020 -0400 release: Move "Update master dev version" to just after the fork All commits after the release has been tagged are destined for the *next* release unless backported, so the materialized version strings should reflect that. commit 9b430cebc2bac809028c37f93d67198984540195 Author: Brandon W Maister <bwm@materialize.io> Date: Sat May 2 16:24:51 2020 -0400 release: We no longer need to manually create the deb package commit d094bace68690342ac4969e9e448ebfc0c30fd85 Author: Brandon W Maister <bwm@materialize.io> Date: Sat May 2 12:41:18 2020 -0400 release: Convert "Final Release" artifacts into headings Mostly aesthetic, this also allows linking directly to headings commit 4838095e9e3d5a33d71a1257a10622111ff0fa0e Author: Brandon W Maister <bwm@materialize.io> Date: Wed Apr 22 11:14:34 2020 -0400 Update release checklist * Add creating a release issue to the beginning * Make it easier to follow by breaking out the steps more granularly * Hide a fairly large diff by default commit 073f6577f0b435006275bedf09f422ed9d681bf4 Merge: 3dab7bfe 3a318dc5 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 12:53:25 2020 -0400 Merge pull request #2861 from benesch/no-rustls ccsr: remove dependency on rustls commit 3a318dc5e43864188bd8d2607b800fc96f842d54 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 00:24:39 2020 -0400 ccsr: remove dependency on rustls See #2799. We're stuck with OpenSSL for librdkafka, and best not to have two crypto libraries in play. commit 3dab7bfe0cbd5804cf2d493b3100c0c44a68dd0c Merge: 9974fcfd 132ee70d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 10:53:09 2020 -0400 Merge pull request #2775 from benesch/batch-execute materialized: use new multi-statement execute support commit 132ee70d47501ada7aaf7b27b9307957d5e1a3f6 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun Apr 26 22:31:05 2020 -0400 pgwire: minorly clean up multi-statement execution commit c982817ec571a3687768768fbb3abfeb4c353292 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat Apr 25 19:26:04 2020 -0400 materialized: use new multi-statement execute support Use the new support for multi-statement execution requests to clean up some of our test code. commit 9974fcfd14d531813a1934fbb306e65b962a7027 Merge: efce79ee 316f2036 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sat May 2 00:28:26 2020 -0400 Merge pull request #2840 from benesch/mzbuild-push ci: ensure all mzbuild images are pushed commit efce79eec2cd94795e17c3d2be11031e18128ab5 Merge: 602c6bdb 515979b3 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 23:13:38 2020 -0400 Merge pull request #2858 from benesch/ccsr-refactor ccsr: tidy up commit 515979b3fa294d390e4d7eaf283bcf7fa6b2bc07 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 15:09:47 2020 -0400 *: remove serde snake-case renaming This is no longer necessary, and leaving it around is just confusing. It's gotten cargo-culted in several places. commit ee75625ebe1bf3ee5a421e8ff6235be46f277e89 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 15:07:31 2020 -0400 ccsr: fill in all missing docs commit 419ace58f067f007f47d808f81a7923e126f602b Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 14:32:47 2020 -0400 ccsr: capitalize acronyms according to Rust style Rust capitalizes PEM as "Pem" and so on. commit 7a84a6f8884480bc66eda9ae0a41f9d84ed771d1 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 14:07:02 2020 -0400 ccsr: document all TLS methods Also avoid encoding URLs directly in the rustdoc. This will happen automatically when using bin/doc. commit 602c6bdbfbe97dde15bc1a5cd0513676b28ca862 Merge: 39ec6783 48cc82a7 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 22:38:25 2020 -0400 Merge pull request #2831 from benesch/release-notes-021 doc/user: add release notes for v0.2.1 commit 39ec678306318e3e234729d6e3d591354a75beef Merge: 4dc6ae2b d14abc9f Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 22:34:41 2020 -0400 Merge pull request #2855 from ruchirK/rk-fix-sinks dataflow: improve sink responsiveness and perf commit 3862af3e284002b7a75bace1087d121b09a730c2 Author: Andi Wang <andi@materialize.io> Date: Fri May 1 13:05:41 2020 -0400 Disable support for union of record + null in avro. Fix support for avro record in upsert. commit d14abc9fe8ed261453e854ba1005367a625a1ec9 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 17:43:37 2020 -0400 make kafka producer flush on drop commit 4dc6ae2bedfd6b4ef853f7c205fc1d56e25e9a33 Merge: c1e8fa1b 02548620 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 16:47:11 2020 -0400 Merge pull request #2850 from ruchirK/rk-2522 catalog: suppress log output for placeholder sink entries commit f7e89adcf5b44d51ed1cd79b4b1a325a0ef29e48 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 16:40:59 2020 -0400 Reintroduce flush w/ 0ms timeout commit 48cc82a773faf9f219a3c9d96bdcc5aac3736b07 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 17:32:55 2020 -0400 doc/user: add release notes for v0.2.1 commit df6a36a29eab1898726e7937b173a52ef6ea2bf4 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 16:12:11 2020 -0400 Remove flush entirely commit f4b9662e0f749b49ca8cee5a014bfeda031f7b9a Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 14:06:03 2020 -0400 ccsr: document ClientConfig methods commit 89fb922cb6b1c7fb45058656ade20a4ab065ec30 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 13:56:15 2020 -0400 ccsr: make config builder the only interface It is unusual in Rust to have an object with a builder where the builder has a build method that construts the object, and the object has a new method that takes the builder. (If there is a way to create the object with a default config, then a new/default method that does not take a config is typical, but that's not the case here.) This patch makes ccsr::ClientConfig::build the only means of creating a ccsr::Client. It also adjusts the method to take ClientConfig by value, as is conventional, to avoid clones in the common case. On the rare occasion that the ClientConfig needs to be preserved, it can simply be cloned by the caller. commit 58cba2388e96c6765c9223cc726881f556a9e704 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 13:00:14 2020 -0400 ccsr: split into modules The main file was getting rather large. Split it into modules to make it more manageable. This commit is pure code movement. commit 95da515de5464a92e7f8b48b5dd1c01b4b7d15ba Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri May 1 12:49:49 2020 -0400 ccsr: delete synchronous client The synchronous client was always meant to be temporary. Port the last remaining user of the sync client over to the async client. commit d88b7e7663c9c9f3079cdf166614caf78f479aab Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 15:20:32 2020 -0400 Update comments + downsize max buffer a bit commit c0aaaf551a400f8edffa65759ecc4b8d42ef7c17 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 14:24:52 2020 -0400 dataflow: improve sink responsiveness and perf commit aaa23f603415e4fb95099e978b6719b57c0ce1dc Author: Brandon W Maister <bwm@materialize.io> Date: Fri May 1 13:43:37 2020 -0400 dashboards: Reduce prometheus sql poll interval to 10s When the system is under load these can contribute to it, and they start timing out. commit 3e99f046383e1dbbfb1193cbb3dccaad362397a7 Author: Brandon W Maister <bwm@materialize.io> Date: Thu Apr 30 15:05:37 2020 -0400 dashboard-logs: Printing causes problems, so write to a file instead By default in ubuntu docker containers writing to /dev/stdout is similar to printing, but it doesn't share the same fileno as the actual stdout stream, which seems to mean that it doesn't have problems locking stdout when sharing file descriptors with supervisord. Additionally, because of the fact that FIFO files block their owning process if there is not both a reader and a writer attached, we need to open the same file in both modes, even though we're only reading from it. commit 6aa01b99154e35b3879434595fe0976992829813 Author: Brandon W Maister <bwm@materialize.io> Date: Wed Apr 29 23:16:12 2020 -0400 dashboard: Add a program that prefixes logs with the name of their service Having a bunch of services in a docker container is bad for a bunch of reasons, one of which is docker isn't really designed to handle that kind of logging. This works around that, to make it easier for us to debug. commit c1e8fa1b06d8bc894d08b609bb7a1a054cb8cbab Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Fri May 1 11:47:50 2020 -0400 Fix get_shard_ids bug (#2853) Insert all shard id strings to the set individually commit b023f1c17b865bc1bc1967a665850a5d1feb38f0 Merge: 443986e9 4b7cac91 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Fri May 1 07:23:15 2020 -0400 Merge pull request #2849 from justinj/fix-demand Fix panic in demand analysis commit 025486204f7dd6653be57ad5681e8efcf9bae43e Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri May 1 00:15:25 2020 -0400 catalog: suppress log output for placeholder sink entries Fixes #2522 commit 4b7cac9166c60a14bf91217ddd04c32a311ffba4 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 30 22:56:05 2020 -0400 Fix panic in demand analysis Previously, we would inform the input to a FlatMapUnary that we expected it to produce even the column we ourselves were producing, which understandably caused it anguish. This change only asks for the columns the input is capable of producing. I couldn't find an existing place that set-returning functions were tested, so I added an SLT file. commit 443986e9ee1e142c022b6f6e8d1e91cd6af2628c Merge: c81cdb50 1ecfa669 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 30 17:45:27 2020 -0400 Merge pull request #2844 from justinj/fix-timestamp-usage Fix TAIL's use of TIMESTAMP{,TZ} as timestamps commit c81cdb503621565580400163334b20e7afbc8c8f Author: Brandon W Maister <bwm@materialize.io> Date: Thu Apr 30 15:31:20 2020 -0400 dc.sh: support stopping a service that has been run multiple times (#2778) if you: $ dcsh run -d peeker $ dcsh run -d peeker that will result in two peekers running. Before now dc.sh would choke on that, with this change it stops them all. commit b552437effb2d37324851fd91f43c840799d022f Merge: b4dc283e 9d71e19b Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 30 15:26:26 2020 -0400 Merge pull request #2847 from benesch/ore-feature ore: add missing dependency on tokio's time feature commit b4dc283e69d4033b12034289dd8e81ba5eecc747 Author: Brandon W Maister <bwm@materialize.io> Date: Thu Apr 30 15:13:39 2020 -0400 Fix dangling reference to grafana mzcompose service (#2848) It was left over from the transition to single-image monitoring commit 1ecfa669fba8b478ff0868775be8f31f5bf5cec1 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 30 11:53:09 2020 -0400 Fix TAIL's use of TIMESTAMP{,TZ} as timestamps This was inadvertedly using seconds since epoch instead of milliseconds. This commit changes it to use the milliseconds. I'd like to add a test for this, but I'm still not sure how to do so in a nice way, so I'll just include the promise that I tested this out by hand (thought I did last time, I guess I confused myself somehow), unless there are better ideas! commit ddf7c4088df7bfadbaa2666a4caa7d0819464a3a Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Thu Apr 30 13:06:33 2020 -0400 Remove failure from aws-util, kinesis source code (#2828) Replaces failure usages with anyhow commit 7439f36f6e869b6c58b179f336ce6633ed7abc4c Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Thu Apr 30 12:45:47 2020 -0400 Add reset_kinesis to testdrive cleanup (#2832) reset_kinesis will remove any Kinesis streams created from the current run of testdrive iff the testdrive run completes. commit 9d71e19b7d0e8569446a439ad4b225538ce9ca46 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 30 12:22:17 2020 -0400 ore: add missing dependency on tokio's time feature This got missed because it's not necessary when building the whole workspace, but it *is* necessary when running just repr's miri tests. commit 256e0cc8547dd9436432aa428290a5f4755caf05 Merge: 5037a6e5 69134825 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 30 11:59:59 2020 -0400 Merge pull request #2836 from justinj/sql Some small cleanups to RelationalExprs commit 5037a6e511270624501950dc95390a8e18a293f7 Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Thu Apr 30 11:59:44 2020 -0400 Release notes: Kinesis multi-shard support (#2843) commit 711505c1475c896da539edde96358ab1ee4240ed Merge: e0642417 05b5c7f6 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 30 11:16:55 2020 -0400 Merge pull request #2816 from benesch/metabase-demo test/metabase: validate basic integration with materialize via API commit 691348258455e69b50bbd4d1b669605b6034721a Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Wed Apr 29 20:50:18 2020 -0400 Some small cleanups to RelationalExprs I was reading through this to get a feel for the code and added some comments and renamed some stuff as I went. commit e0642417e63bbc3884b03f298dc39f2eb337ff4a Author: Brennan Vincent <brennan@materialize.io> Date: Thu Apr 30 10:19:50 2020 -0400 Work around python3.6 timestamp parsing bug (#2842) commit 0cbbc2b676f52593ebcbfef91b62b22031b2b687 Merge: 569c43cf c84b9f1e Author: Arjun Narayan <arjunravinarayan@gmail.com> Date: Thu Apr 30 09:37:25 2020 -0400 Merge pull request #2833 from rjnn/fix_doc_link doc: fix broken link to monitoring dashboard commit 569c43cf58364edaddeaf084944b2bde8e6763db Author: Frank McSherry <fmcsherry@me.com> Date: Thu Apr 30 09:08:27 2020 -0400 Extract arrangement frontier state (#2826) This PR extracts from the coordinator logic related to managing the frontier state of arrangements. These methods were previously member methods of the coordinator, rather than restricted to the index they act on. This is part of a progressive thinning out of tasks from the coordinator and in to comprehensible modules. commit 316f203675b1cfc2f0c8d827ec0f6fee07d46afe Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 30 00:11:59 2020 -0400 ci: ensure all mzbuild images are pushed If pushing an image fails (e.g. because the Docker Hub permissions are incorrect), that push won't be retried on subsequent CI builds, because the image will be available locally and the push will get skipped. Rework the build jobs so that all images are checked for existence on Docker Hub and pushed if they do not exist. commit 05b5c7f6fee81d226164e2cfd30e79bacacea945 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun Apr 12 13:40:53 2020 -0400 test/metabase: validate basic integration with materialize via API Add some smoke testing of Materialize and Metabase by using the Metabase API. The new test installs the Materialize connector in Metabase, creates a simple database with a simple table, and verifies that Metabase correctly reports this table's columns. Fix #2083. commit a30c0fe8c2be5477c8759438a5a499d60059b9d4 Merge: 8202cd5a c8ac3d9d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 30 00:19:55 2020 -0400 Merge pull request #2839 from benesch/testdrive-flakiness ci: run docker-compose down before docker-compose up commit c8ac3d9d91698390e3d8cb9e8eb3bbd1d9c6066d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 23:35:12 2020 -0400 ci: run docker-compose down before docker-compose up Sometimes build cancellations cause Docker Compose to fail to clean up after itself, and those lingering containers and volumes can interfere with the next build. commit 8202cd5a64ffcadbfc089a35b369375dbeb89fd2 Merge: 687b1df7 1e2fefeb Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 23:12:27 2020 -0400 Merge pull request #2829 from benesch/sql-ast-cleanup sql,sql-parser: assorted cleanups commit 687b1df7adfb8ae5bf6a5b393afabcaca56599dd Merge: 1f012b9c ace7bb62 Author: Sean Loiselle <sean@materialize.io> Date: Wed Apr 29 22:42:26 2020 -0400 Merge pull request #2820 from sploiselle/client-ssl-support-tests integration: test Kafka SSL auth commit 1e2fefeba35cb41af67be021ace82383c5ef50b8 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 17:22:32 2020 -0400 sql: avoid unnecessary else branch indentation commit a28cb782155f06bd676fdf89d221c33538125566 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 17:14:04 2020 -0400 sql: limit places where we parse wildcards Wildcards are really special in SQL syntax, and can only appear unqualifed in a select list, like SELECT * FROM ... or inside of an aggregate function, like: count(*) Our parser previously permitted wildcards anywhere in an expression tree, which didn't make much sense. Adjust it so it only allows wildcards in the two places mentione dabove. commit 1c3189fc0edf666177acac88fb5eea1b97103e2c Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Wed Apr 29 16:13:43 2020 -0400 sql-parser: remove distinction between single and compound identifier Compound identifiers are fully general; there's no reason to special case single-element compound identifiers. commit ace7bb6226a1cf69dd9d7bba3162e4ea8564e77d Author: Sean Loiselle <himself@seanloiselle.com> Date: Wed Apr 22 15:07:52 2020 -0400 integration: test Kafka SSL auth commit c84b9f1e1d53a599372a96abe40fd9e0fea26bc8 Author: Arjun Narayan <arjun@materialize.io> Date: Wed Apr 29 18:40:26 2020 -0400 docs: typo thanks Frank! commit 81aa7e68407fd3522b9f2c5c6d607ed9fe1fdab0 Author: Arjun Narayan <arjun@materialize.io> Date: Wed Apr 29 18:26:50 2020 -0400 doc: fix broken link to monitoring dashboard when rendered, it renders into a link under materialize.io, which is incorrect. commit 1f012b9c8d59a2564485c6d87cc7ede860772097 Merge: 2412b418 8ad2ea57 Author: Brandon W Maister <bwm@materialize.io> Date: Wed Apr 29 18:02:26 2020 -0400 Merge pull request #2780 from quodlibetor/single-image-monitoring Add materialize/dashboard docker image commit 8ad2ea574cefa099796ae4088cc3c719d564df1f Author: Brandon W Maister <bwm@materialize.io> Date: Wed Apr 29 12:40:44 2020 -0400 dashboard: Change prometheus retention to max out at 1GB Or 15 days, whichever is smaller. 15 days is the default, this just makes it explicit. commit 7a3dcd363b4b4f3149a9225d97981df59f93c923 Author: Brandon W Maister <bwm@materialize.io> Date: Sun Apr 26 17:09:44 2020 -0400 Add dashboard to all our demos, replacing the old setup in chbench This uses the new mzbuild download-files pre-image plugin to build simple-monitor, and uses `mzbuild: simple-monitor` everywhere. commit ba0363a0879451b30e455373a2575e8078a979b7 Author: Brandon W Maister <bwm@materialize.io> Date: Fri Apr 24 17:31:58 2020 -0400 Provide an easy to use, pleasant monitoring experience This builds a new simple-monitor Docker repo which can be run with only one parameter, and gets you the most important metrics easily. It also makes it easy for us to provide updates to users. commit 79d9cf634f805f3bd83469983946035edb049f99 Author: Brandon W Maister <bwm@materialize.io> Date: Sun Apr 26 12:50:42 2020 -0400 mzbuild: support passing --build-args to docker-build I want this so that for some cases configuration doesn't need to live in both Dockerfile and build scripts, the mzbuild.yml file can become the single source of truth of all versions in the build. commit 2412b418250e37d52ae1201e4e067a05d2b5819b Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Wed Apr 29 17:17:57 2020 -0400 Rename kinesis-util -> aws-util (#2827) commit 915c518264f77cf7ae2f0f023f0bdfe19ab0df1e Merge: d9406649 2ff02661 Author: Andi Wang <5522362+wangandi@users.noreply.github.com> Date: Wed Apr 29 16:25:47 2020 -0400 Merge pull request #2818 from wangandi/upsertreleasenotes Add missing avg to docs. Add release note for upsert. commit 2ff02661e1ef6d7f2be177ba68083ac683fe13e9 Author: Andi Wang <andi@materialize.io> Date: Wed Apr 29 11:11:52 2020 -0400 Add missing avg. Add release notes for upsert. commit d9406649f9b93633fc042561fa83a0b241b33bf9 Author: Frank McSherry <fmcsherry@me.com> Date: Wed Apr 29 14:48:12 2020 -0400 Pivot frontier structures to antichains (#2822) This tracks a differential dataflow change that exposes more Antichain<T> and AntichainRef<T> types in public APIs where things that were formerly Vec<T> and &[T] should actually be antichains. I think most of these changes are superficial, and lead us to more semantically clear things that we hand around. commit 43eadc68dd04ec0923f06e695287a1cf485ba829 Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Wed Apr 29 13:52:45 2020 -0400 Move get_shard_iterator to kinesis-utils (#2819) commit 5750582631386bedcfaa34429e015161d3d945f7 Merge: 83f2152a d1e0346a Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Wed Apr 29 12:14:41 2020 -0400 Merge pull request #2762 from justinj/tail-as-of Support TAIL ... AS OF ... commit d1e0346ad901137d63455009c843e39369766375 Author: Justin Jaffray <justin.jaffray@gmail.com> Date: Thu Apr 23 09:46:27 2020 -0400 Support TAIL ... AS OF ... Closes #2721. This commit permits the inclusion of an AS OF argument to TAIL, which causes TAIL to emit updates after that timestamp. It also allows WITH SNAPSHOT, which, if set, will move the timestamps of all events prior to the given timestamp up to the timestamp. commit 83f2152ae4f0df2ba942e24e948e22ad2b0b98d5 Merge: 53c6608d e9f8c2ae Author: Sean Loiselle <sean@materialize.io> Date: Wed Apr 29 10:46:10 2020 -0400 Merge pull request #2744 from sploiselle/client-ssl-support Support SSL auth for Kafka clusters commit e9f8c2ae424f98b1f40eecc15c284a222419a318 Author: Sean Loiselle <himself@seanloiselle.com> Date: Tue Apr 28 10:55:37 2020 -0400 config: address issues with license commit 1836ef29d3de259ea3399307e4378ceb672b54fe Author: Sean Loiselle <himself@seanloiselle.com> Date: Wed Apr 22 15:07:25 2020 -0400 integration: Kafka SSL auth commit 53c6608d596824e18591aed2e4909b5d042bd177 Author: Brennan Vincent <brennan@materialize.io> Date: Wed Apr 29 10:15:00 2020 -0400 fix typo (#2817) commit 179d9f7b3cfb7210c9d688321962d0b53eecb913 Merge: ec762547 1747673e Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Tue Apr 28 23:57:33 2020 -0400 Merge pull request #2815 from MaterializeInc/pyactivate pyactivate: reinstall venv iv dependencies are missing commit 1747673e7f8cb78450f3be94d7000a509dbf0291 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Tue Apr 28 22:20:13 2020 -0400 pyactivate: reinstall venv if dependencies are missing This prevents wedged virtual envs on Debian/Ubuntu when the python-venv is not installed. This fixes part of #2752. commit ec762547b397122e000b73ce4c1941a1f8232473 Merge: 5c10925d 04b93941 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Tue Apr 28 21:31:40 2020 -0400 Merge pull request #2801 from benesch/bump-sasl deps: bump librdkafka for proper gssapi support commit 04b93941c081dd60795823d0b3bed26e7941524d Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Tue Apr 28 00:47:50 2020 -0400 deps: bump librdkafka for proper gssapi support We weren't working hard enough before to statically link in GSSAPI support. That is now fixed with the latest version of rdkafka, with help from sasl2-sys and krb5-src. We are rapidly becoming the Rust authentication experts. commit 5c10925d3a3f3351e6875bfee3652b08e3232246 Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Tue Apr 28 19:32:45 2020 -0400 Pull get_shard_ids into new kinesis-util crate (#2810) Creates a new kinesis-util crate for common implementations of wrappers around AWS Kinesis APIs. commit 28ba1368e7829210411da6ed72e0653f30ee99c6 Author: Jessica Laughlin <JLDLaughlin@users.noreply.github.com> Date: Tue Apr 28 15:32:51 2020 -0400 Rate limit Timestamper hitting Kinesis ListShards API (#2807) Actually hit the ListShards API every 100 iterations of Timestamper's update(), use a cached list of Shard ids otherwise. commit 003951827d610bfe89760a5aeb20da66047e8cc9 Merge: e15fefff 0200a063 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Tue Apr 28 10:30:32 2020 -0400 Merge pull request #2761 from benesch/testdrive-auto-topic-creation testdrive: disable automatic topic creation commit e15fefff3ec7677e26a4361adb1500c8c06c5594 Author: Brennan Vincent <brennan@materialize.io> Date: Tue Apr 28 09:14:36 2020 -0400 Quick check that a file exists when creating a source interactively (#2459) commit 26c261753621a7fe971672b11a9039f74f480fd5 Author: Frank McSherry <fmcsherry@me.com> Date: Mon Apr 27 21:26:00 2020 -0400 implement as_of for sources (#2797) This PR updates source creation to respect the as_of component of a DataflowDesc. This was previously used for index imports, but not for sources. This may not have been a problem, as it was not clear that you could create a DataflowDesc that had an as_of set and used bare sources, but it is becoming possible so we should fix it up. commit 33dc9a89996244034487b7dbbbd66e2d9d429369 Author: Frank McSherry <fmcsherry@me.com> Date: Mon Apr 27 21:25:15 2020 -0400 Consolidate sink updates (#2798) Sinks were previously not consolidating their inputs, which lead to things like TAIL producing updates like (in #2767): \N Diff: -1 at 1587733550550 \N Diff: 1 at 1587733550550 Presumably we would also see these in other sinks, and committing them to output is not necessary. commit 47ab3ebe0160ef61aaa79c4be5a885912dd66240 Author: Brennan Vincent <brennan@materialize.io> Date: Mon Apr 27 16:36:32 2020 -0400 Add script to delete materialized-unstable binaries older than 14 days. (#2794) commit 0c8d6db1a3380a0df0cad0b8aceb7c3ce955d735 Merge: 6e6600c5 5d342da5 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 16:02:52 2020 -0400 Merge pull request #2782 from benesch/mzbuild-deleted-files mzbuild: don't panic on unstaged deletions commit 6e6600c534acfdc24fd65c3565fbc0d5b9b98594 Author: Frank McSherry <fmcsherry@me.com> Date: Mon Apr 27 15:44:30 2020 -0400 use count_total where appropriate (#2796) commit de5ef9dae98c0cd7b9754507d3327f4f56b6f76f Author: Brennan Vincent <brennan@materialize.io> Date: Mon Apr 27 12:18:34 2020 -0400 fix `publish_wait_for_secs` spelling (#2793) commit 7e773a88ad0837d311ac87faaa3b8c0f318325e2 Author: Brennan Vincent <brennan@materialize.io> Date: Mon Apr 27 12:04:05 2020 -0400 Don't force Bintray usernames to the form user@materialize (#2792) commit 089e7c09d89e5ceaf875d3419c7f36b162e712e8 Merge: 18dac8d6 b3ce2dbf Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 12:02:53 2020 -0400 Merge pull request #2785 from MaterializeInc/dependabot/cargo/criterion-0.3.2 build(deps): bump criterion from 0.3.1 to 0.3.2 commit b3ce2dbf8644bea0878a24fa805cc49d3b33a315 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 11:01:07 2020 +0000 build(deps): bump criterion from 0.3.1 to 0.3.2 Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.3.1 to 0.3.2. - [Release notes](https://github.com/bheisler/criterion.rs/releases) - [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md) - [Commits](https://github.com/bheisler/criterion.rs/compare/0.3.1...0.3.2) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 18dac8d667c00fbe57fe0f172dbe3c58fd067dd8 Author: Brennan Vincent <brennan@materialize.io> Date: Mon Apr 27 11:30:00 2020 -0400 disable materialized-unstable uploads. (#2791) commit 36e906380fe446c577ae662550185737c420ef3c Merge: b42c0c27 c1fd0c2c Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 11:29:32 2020 -0400 Merge pull request #2788 from MaterializeInc/dependabot/cargo/tokio-0.2.19 build(deps): bump tokio from 0.2.18 to 0.2.19 commit b42c0c276fbe2f657ee08737d7ff59def98a7cfa Merge: 82f6d447 84113dc5 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 11:29:12 2020 -0400 Merge pull request #2789 from MaterializeInc/dependabot/cargo/tracing-subscriber-0.2.5 build(deps): bump tracing-subscriber from 0.2.4 to 0.2.5 commit 82f6d44744527ac6bb61c34134e184a202bdf599 Merge: 20654a22 c7bda008 Author: Natacha Crooks <nacrooks@gmail.com> Date: Mon Apr 27 11:20:56 2020 -0400 Merge pull request #2743 from nacrooks/kafkabug Fixing issues with multi-partition Kafka streams commit c1fd0c2c443f44982ee8217ef4de4e7c73eb9929 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 14:59:30 2020 +0000 build(deps): bump tokio from 0.2.18 to 0.2.19 Bumps [tokio](https://github.com/tokio-rs/tokio) from 0.2.18 to 0.2.19. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](https://github.com/tokio-rs/tokio/compare/tokio-0.2.18...tokio-0.2.19) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 20654a22e887800dc66e9e783722f770807150e5 Merge: 429fe2de b8bf950e Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 10:57:39 2020 -0400 Merge pull request #2787 from MaterializeInc/dependabot/cargo/structopt-0.3.14 build(deps): bump structopt from 0.3.13 to 0.3.14 commit 429fe2deefe17c46487731ab6b9ef15f0ab3b04e Merge: bf76b2c6 d6d1577f Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 10:33:35 2020 -0400 Merge pull request #2786 from MaterializeInc/dependabot/cargo/derivative-2.1.1 build(deps): bump derivative from 2.1.0 to 2.1.1 commit 84113dc5680d33b5f7e81964e22ea767e36b7bda Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 14:33:18 2020 +0000 build(deps): bump tracing-subscriber from 0.2.4 to 0.2.5 Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.4 to 0.2.5. - [Release notes](https://github.com/tokio-rs/tracing/releases) - [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.4...tracing-subscriber/-0.2.5) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit bf76b2c6b8f2f90e56c70772e739fe4d31767038 Merge: 7b8b102c 40cb41d5 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 10:31:10 2020 -0400 Merge pull request #2784 from MaterializeInc/dependabot/cargo/smallvec-1.4.0 build(deps): bump smallvec from 1.3.0 to 1.4.0 commit c7bda0088b5888ddbe0b36cfc43d34d6fc1275e8 Author: Natacha Crooks <nacrooks@gmail.com> Date: Thu Apr 23 10:36:37 2020 -0400 Fix liveness and safety issues with multipartition Fixing corner-case where start_offset > current number of entries in topic commit b8bf950e84a3393699523ea3e4523f590bbb6c5a Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 14:10:49 2020 +0000 build(deps): bump structopt from 0.3.13 to 0.3.14 Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.13 to 0.3.14. - [Release notes](https://github.com/TeXitoi/structopt/releases) - [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md) - [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.13...v0.3.14) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit d6d1577f54764171eef2b6ee3809f80c55e5c963 Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 14:07:14 2020 +0000 build(deps): bump derivative from 2.1.0 to 2.1.1 Bumps [derivative](https://github.com/mcarton/rust-derivative) from 2.1.0 to 2.1.1. - [Release notes](https://github.com/mcarton/rust-derivative/releases) - [Changelog](https://github.com/mcarton/rust-derivative/blob/master/CHANGELOG.md) - [Commits](https://github.com/mcarton/rust-derivative/commits) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 40cb41d57401bd6eb74272c50b140b9bfe4ce59d Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 14:06:54 2020 +0000 build(deps): bump smallvec from 1.3.0 to 1.4.0 Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.3.0 to 1.4.0. - [Release notes](https://github.com/servo/rust-smallvec/releases) - [Commits](https://github.com/servo/rust-smallvec/compare/v1.3.0...v1.4.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 7b8b102cf7180db83163b9a4e9fc8ab44b05b2fb Merge: 7fd948ce acd08f0b Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 10:03:53 2020 -0400 Merge pull request #2783 from MaterializeInc/dependabot/cargo/rdkafka-c414a78 build(deps): bump rdkafka from `5a5f284` to `c414a78` commit 7fd948ce6e34daa43943d2b07af8c63c2265400b Merge: acca70a0 f990d872 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Mon Apr 27 10:03:19 2020 -0400 Merge pull request #2781 from benesch/chbench-schema-registry chbench: remove custom schema-registry container commit acd08f0b6d146a0fd4e666028ddd2d80f7179d0b Author: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Date: Mon Apr 27 11:00:20 2020 +0000 build(deps): bump rdkafka from `5a5f284` to `c414a78` Bumps [rdkafka](https://github.com/fede1024/rust-rdkafka) from `5a5f284` to `c414a78`. - [Release notes](https://github.com/fede1024/rust-rdkafka/releases) - [Commits](https://github.com/fede1024/rust-rdkafka/compare/5a5f284b9d5cbe8ad2d44f268f51aa91a3a87614...c414a780f47fbfc9fa783cf47d0f09759f504edc) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> commit 5d342da52404372eb0fe5736ba643046929b67b0 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun Apr 26 21:41:20 2020 -0400 mzbuild: don't panic on unstaged deletions Discovered by @quodlibetor. See the comment within for the full story. commit f990d8723e5266b8c77cb22a4e421c36859aee1a Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Sun Apr 26 20:34:19 2020 -0400 chbench: remove custom schema-registry container flush-tables is now run by the chbench container. Plus this avoids depending on a weirdly flaky `apt-get install` command; something about the Confluent images breaks APT. commit acca70a020ac92b66db4004adcd41a5241b34657 Author: Brennan Vincent <brennan@materialize.io> Date: Sat Apr 25 19:05:59 2020 -0400 Allow multiple statements in one SQL query (#2773) commit b9071eaa736edaecb128ea43ce9efe5e85fd669a Merge: 6db5fae7 cde443ca Author: Brandon W Maister <bwm@materialize.io> Date: Sat Apr 25 14:52:29 2020 -0400 Merge pull request #2769 from quodlibetor/dcsh-demo-load-creates-materialized-views dc.sh and peeker now support creating materialized sources commit 6db5fae7e248da087bb0787aeef26b9228cdc4d7 Author: Brennan Vincent <brennan@materialize.io> Date: Sat Apr 25 11:39:30 2020 -0400 bump rusqlite version (#2774) commit 0200a0630e40412e0443f2764e6bf871c0fe6934 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 21:29:14 2020 -0400 coord: don't loop forever on erroring metadata requests The timestamper could wedge forever in a Kafka metadata retrieval loop. This would prevent any sources from getting timestamped, leading to the system wedging. (This presented in CI as flakes in testdrive.) In particular, if a topic is deleted from Kafka before the timestamper is instructed to drop the source, the metadata requests on the topic will fail forever. If auto topic creation is enabled, as it previously was in CI, we would accidentally recreate the topic and break out of the loop; but with that setting disabled, we get stuck forever. Instead of assuming that every metadata request will succeed eventually, bubble the error up so that it can be handled appropriately. commit 1765126b016a7b962d879d21c3714b930be965fe Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 23 16:41:46 2020 -0400 testdrive: disable automatic topic creation Testdrive and Materialize often race to create the same topic. We want testdrive to win, as it issues actual create topic requests with specific partition counts, while Materialize is accidentally creating the topic via Kafka's auto-topic creation feature. Auto-topic creation is usually disabled in production, so it's actually more realistic to test with it off, anyway. This exposed we were forgetting to create topics in a few testdrive scripts. The hypothesis is that this is a cause of one the flakes described in commit 2abbdafc8b5c36ecd595253ed228194f68296786 Merge: 0a2d3d35 ef953446 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 16:08:25 2020 -0400 Merge pull request #2764 from benesch/source-config dataflow: rework source configuration commit 0a2d3d3568ddfbaf64bdc00bfe82ac59a970b7ce Merge: 66f1ba1d e350041c Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 16:06:26 2020 -0400 Merge pull request #2768 from benesch/deemphasize-pgcompat doc: de-emphasize PostgreSQL compatiblitiy commit e350041cd76f696f283f84fd20a53770c301a9b2 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 12:09:49 2020 -0400 doc: de-emphasize PostgreSQL compatiblitiy We are decidedly not compatible with pgcli, so strike that from the docs. Also tone down the language about PostgreSQL compatibility to cut ourselves some slack. commit ef95344631b8e231d8dd4c79c6f3164369b9ca77 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Thu Apr 23 19:43:09 2020 -0400 dataflow: rework source configuration Put a few more parameters into the shared source configuration, so more logic can be shared between source configurations. This doesn't change any sematics. commit 66f1ba1d48e54fa20e3fe560c85d7009a4e74aa7 Merge: d86a94e0 a520ffd1 Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 13:51:57 2020 -0400 Merge pull request #2771 from benesch/mzimage-spec mzimage: add spec convenience subcommand commit cde443cae6ba77f73468da3861ab17677572c1b4 Author: Brandon W Maister <bwm@materialize.io> Date: Fri Apr 24 12:02:30 2020 -0400 dcsh demo-load --up to handle starting up demos commit a520ffd1eb449f3e08beb40b3c5ef4e1020d724e Author: Nikhil Benesch <nikhil.benesch@gmail.com> Date: Fri Apr 24 13:49:02 2020 -0400 mzimage: add spec convenience subcommand Add a subcommand that prints out an image spec, like materialize/testdrive:mzbuild-U6WKRP7LGOLURBHYKVZK65KCY7MOXHZ4 so that folks don't need to manually construct one from the output of `mzimage describe`. commit d86a94e055ddb20485bea9bbaff23fd3b6481a21 Author: Brennan Vincent <brennan@materialize.io> Date: Fri Apr 24 13:04:24 2020 -0400 Simplify timestamps (#2766) commit 70dd366b7b6597446f8b64a96c089d864033ab97 Author: Brandon W Maister <bwm@materialize.io> Date: Fri Apr 24 11:31:57 2020 -0400 runv: quote arguments to sub-processes commit 68beb2618200e48283d2d4db9209e3fd69840d96 Author: Brandon W Maister <bwm@materialize.io> Date: Fri Apr 24 11:30:40 2020 -0400 Make dc.sh demo-load create materialized sources commit 6b3ecea370517e8b585e1be7e69344c484ef281f Author: Brandon W Maister <bwm@materialize.io> Date: Fri Apr 24 11:04:12 2020 -0400 peeker: allow configuring sources to be materialized commit cd5f97d32f424739f5c12ef95c9d9de0aa4992eb Merge: 3248fc33 7927db5b Author: Natacha Crooks <nacrooks@gmail.com> Date: Fri Apr 24 11:39:39 2020 -0400 Merge pull request #2765 from nacrooks/tailing Fixing testdrive failure on file sources commit 3248fc3350f09ebf4766b6445cca0fdf7cd2a2e2 Author: Ruchir Khaitan <ruchir0001@gmail.com> Date: Fri Apr 24 10:09:34 2020 -0400 Add support for Smith fuzzer (#2754) Touches #1722 commit 7927db5b3a90e5288822f73f3c7f4781ae72a206 Author: Natacha Crooks <nacrooks@gmail.com> Date: Fri Apr 24 01:02:15 2020 -0400 Removing stale comment commit 6337f3db6536a4407222e67573f8a79bfc5f55d1 Author: Natacha Crooks <nacrooks@gmail.com> Date: Thu Apr 23 23:46:31 2020 -0400 Missing field commit c13ee0ed5a7b612dcb574b0c7f524aa518e497e8 Author: Natacha Crooks <nacrooks@gmail.com> Date: Thu Apr 23 23:35:56 2020 -0400 Missing lib file commit 9452dc58e1867daabb9f307d5facbb265d72fcf7 Author: Natacha Crooks <nacrooks@gmail.com> Date: Thu Apr 23 23:24:05 2020 -0400 Making persistence of RT timestamp an optional feature (default off) commit 01b28f98f36bb1318f23d970f75fbcb60fb2a65c Author: Natacha Crooks <nacrooks@gmail.com> Date: Thu Apr 23 20:34:04 2020 -0400 Adds mandatory activation of source after every timestamp update. commit ce05fe6306e7ec7adeb2f5d2ebab699246c3a78f Merge: 5d7d1545 b2874b27 Author: Jamie Brandon <jamie@scattered-thoughts.net> Date: Thu Apr 23 14:37:50 2020 -0700 Merge pull request #2644 from jamii/datum-array Add a LIST type commit b2874b270fb53630307e2049844376419404fc7d Merge: a8416e2a 5d7d1545 Author: Jamie Brandon <jamie@scattered-thoughts.net> Date: Thu Apr 23 13:26:03 2020 -0700 Merge branch 'master' into datum-array commit 5d7d1545e6be7f84dcf8aab9233c40d51330b55e Merge: e7e0539b 169ac3e7 Author: Brandon W Maister <bwm@materialize.io> Date: Thu Apr 23 15:40:54 2020 -0400 Merge pull request #2745 from quodlibetor/kafka-stream-tracking Add "messages distance behind kafka" prometheus metrics commit 169ac3e781160d294bffae2ff9bc4a6711668f0b Author: Brandon W Maister <bwm@materialize.io> Date: Wed Apr 22 22:00:17 2020 -0400 Delete leftover commented-out c…
We weren't working hard enough before to statically link in GSSAPI
support. That is now fixed with the latest version of rdkafka, with help
from sasl2-sys and krb5-src. We are rapidly becoming the Rust
authentication experts.
@sploiselle do you still have that Kerberos testing rig lying around? Can you verify that this actually works? So far I've just gotten it to build, which was a nightmare and a half.
This change is