This repository has been archived by the owner on Dec 16, 2022. It is now read-only.
Slack sync upstream 2019 11 09.r0 #142
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
The fake db client framework now allows for queries to return different results based on each call. This is required for future tests. Also added more logging in a few places for improved troubleshooting. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Make stream validation more robust. It wasn't catching all mismatches. Clean up target streams before migration in case some were left over during previous failed cutover. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
* Remove obsolete comments. These are talking about the serving graph, which no longer exists. Instead of storing serving state of each tablet in topo, we now have vtgate directly query serving state of every tablet. Signed-off-by: Anthony Yeh <enisoc@planetscale.com> * Make DemoteMaster idempotent. Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
…Cancel Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: tanjunchen <2799194073@qq.com>
Signed-off-by: wh <wanghao.seed@gmail.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
Signed-off-by: Saif Alharthi <saif@saifalharthi.me>
Unit tests for wrangler version of TabletExternallyReparented
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: Andres Taylor <antaylor@squareup.com>
Split tests into unit and endtoend
Less output from Maven
Signed-off-by: deepthi <deepthi@planetscale.com>
When grabbing a connection from the transaction pool, we should use the queryserver-config-txpool-timeout. BEGIN statements should not use the queryserver-config-txpool-timeout when executing the BEGIN query. Signed-off-by: dleibovic <dleibovic@etsy.com>
unit tests for shard watch
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
Added support for CHECK in Parser Grammar
…r-java-directory update the license header in java/ tools/ travis/ tests.go files
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
…rd 0 Signed-off-by: deepthi <deepthi@planetscale.com>
split unit and unit_race between different shards, reduce load on shard 0
Signed-off-by: Andres Taylor <antaylor@squareup.com>
change test to handle new mastership rules Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
vreplication: vdiff
Fixes vitessio#5206 The conn schemaname var can get set through a new connection or a ComInitDB packet. However, a use statement bypasses this mechanism and gets handled at the vtgate level. This causes problems because schemaName in the connection disagrees with TargetString in vtgate. To fix this, the new scheme calls a new handler function: ComInitDB every time schemaName is initialized or changed in the connection. This way, the TargetString remains the authoritative source for the current db name. I couldn't write a test for this specific behavior change because the mysql go client does not implement support for ComInitDB. But I've verified the things generally work as expected. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
mysql: handle db name correctly
…ipt-osx Fix usage of arch to work on OSX
Signed-off-by: Mark Solters <msolters@gmail.com>
At startup, we already avoid asking mysqld for its port if we were given the port in DB configs. However, we would forget this port when we later trigger a recheck of the MySQL port. It's important to trust the port that was given to us because it might be different from the port that mysqld is actually serving on, such as in the case that the host we were given is actually a proxy. Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
The IsFunctional support function was mostly unused. The only place left was in the vstreamer. It was protecting from a vstreamer using a lookup vindex because vstreamer doesn't have a VCursor. This is now addressed by the lookup vindexes checking for VCursor being nil. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Bind var initialization was duplicated in different parts of the primitive. It's now unified and done after all the vindexes are processed. Also, the primary vindex code was flattening the vindex values because it assumed only one column. The code has been rewritten to not assume this. This prepares us for the next step, which is to implement MapNew that will accept multi-column vindex values. Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Signed-off-by: Morgan Tocker <tocker@gmail.com>
VStreams should respect FieldEventMode
Stop nagging about where go is installed
vindex: region_experimental
Signed-off-by: Morgan Tocker <tocker@gmail.com>
Add GitHub action for local example
Relates to vitessio#5418 and several other support requests. Small fix to GitHub action. Signed-off-by: Morgan Tocker <tocker@gmail.com>
Signed-off-by: Ricardo Lorenzo <rlorenzo@slack-corp.com>
Add stronger protection against running as root
AWS max retries option added
rafael
approved these changes
Dec 11, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR merges the latest upstream from vitessio/vitess to our fork. We chose to pull these changes up to this particular commit so that we could take advantage of the changes introduced by this PR: vitessio#5354
Relase notes: https://github.com/vitessio/vitess/releases/tag/v4.0.0
JIRA: https://jira.tinyspeck.com/browse/VITESS-424
Testing
Jenkins build: https://jenkins.tinyspeck.com/job/slack-vitess/376/
Tag:
slack-vitess-2019.11.09.r0Benchmarking results (performed on
loadtest)Dev rollout notes
Link to #deploys-vitess thread with rollout notes: https://slack-pde.slack.com/archives/CQGR39RA5/p1574710112018800
Changes
Relevant vitess release notes: https://github.com/vitessio/vitess/releases
To view all commits introduced by this PR, please refer to commits.txt.
git log origin..slack-sync-upstream-2019-11-09.r0 --oneline | tail -1git log origin..slack-sync-upstream-2019-11-09.r0 --oneline | head -1git log cf7fe76d3...9b5de395f --name-only > commits.txtFor all PRs included in this deploy, please refer to this page or this list of PRs.
Noteworthy changes
vitessio/vitess#5000: check replication lag on state change before starting query service
vitessio/vitess#5143: Orc basic auth with vttablet flags
vitessio/vitess#5151: make TabletExternallyReparented more robust
vitessio/vitess#5160: Point in time recovery (part 1)
vitessio/vitess#5143: Orc basic auth with vttablet flags
vitessio/vitess#5203: Add startup argument to stop tablet from reparent
vitessio/vitess#5216: vstream: best effort option for field events
vitessio/vitess#5226: PlannedReparentShard: Allow retrying PRS to the existing master
vitessio/vitess#5316: InitTablet should not update master alias on shard record
vitessio/vitess#5360: vttablet-up and vtgate-up should block until ready
vitessio/vitess#5363: EmergencyReparentShard does not need to update shard master
vitessio/vitess#5376: Fix more known-recoverable problems
vitessio/vitess#5391: Make reparents more robust
Slack changes:
@rafael vitessio/vitess#5197: Redact password from error log
@demmer vitessio/vitess#5139: vtexplain support select distinct and other expressions
@RicardoLorenzo vitessio/vitess#5354: AWS max retries option added
Non-test/documentation/kube related changes:
vitessio/vitess#4978: consistent lookup: handle insert ignore
vitessio/vitess#5000: check replication lag on state change before starting query service
vitessio/vitess#5103: Allow setting key/value in a metadata stored in the topology
vitessio/vitess#5109: Switch to go modules
vitessio/vitess#5118: Make it possible to send span context over the MySQL end point
vitessio/vitess#5120: make vtgate query_plans endpoint return json; also fix comment typos
vitessio/vitess#5130: vreplication: improved rowstreamer
vitessio/vitess#5136: Only override session.TargetString if it has not been set
vitessio/vitess#5139: vtexplain support select distinct and other expressions
vitessio/vitess#5140: Subtraction operation implementation
vitessio/vitess#5141: Execute "ALTER VSCHEMA" statements without specifying keyspace in the target connection
vitessio/vitess#5143: Orc basic auth with vttablet flags
vitessio/vitess#5145: Delete vitess metadata by setting empty key
vitessio/vitess#5147: vtbackup: Make initial backup idempotent again
vitessio/vitess#5149: Handle case where mysqld replies to Initial Handshake Packet with an ERR packet
vitessio/vitess#5150: Handle more situations with empty results and no grouping keys
vitessio/vitess#5151: make TabletExternallyReparented more robust
vitessio/vitess#5153: vtbackup: Stop slave before trying to change master address
vitessio/vitess#5157: Don't try to reparent to yourself
vitessio/vitess#5159:vreplication: refactor table_plan_builder
vitessio/vitess#5160: Point in time recovery (part 1)
vitessio/vitess#5163: vreplication: ability to exclude tables
vitessio/vitess#5165: RefreshMaster on both source and target before turning off query service on source during splitting
vitessio/vitess#5177: xtrabackup: Make sure all files are closed before writing MANIFEST
vitessio/vitess#5178: migrater: identify shard migration differently
vitessio/vitess#5179: vreplication: more VReplicationExec constructs
vitessio/vitess#5184:Track number of times pool reached zero available slots
vitessio/vitess#5186:Multiplication operation expression
vitessio/vitess#5192: vtbackup: Retry fetching master position
vitessio/vitess#5193: xtrabackup: Add a timeout on closing backup files
vitessio/vitess#5194: vreplication: migrate source streams to target shards on MigrateWrites
vitessio/vitess#5197: Redact password from error log
vitessio/vitess#5198: Enable selecting consistent snapshot during resharding workflows via vtctld ui
vitessio/vitess#5203: Add startup argument to stop tablet from reparent
vitessio/vitess#5210: TER should not demote current master if it is run with current master
vitessio/vitess#5213: Added exclude_tables flag in workflow and pass it down to vtworker
vitessio/vitess#5216: vstream: best effort option for field events
vitessio/vitess#5219: Add support for COM_RESET_CONNECTION
vitessio/vitess#5225: Do not do mysql_flavor detection for external/unmanaged tablets
vitessio/vitess#5226: PlannedReparentShard: Allow retrying PRS to the existing master
vitessio/vitess#5236: Reparent: add ability to watch shard data
vitessio#5238): Transaction BEGIN statements should use the query timeout rather than… the transaction pool timeout
vitessio/vitess#5244: topology_watcher: Allow tablets to reuse old tablet addresses
vitessio/vitess#5251: Simplify replication lag selection logic
vitessio/vitess#5254: Don't abort restore if master is unreachable
vitessio/vitess#5259: Make StatementType a type alias instead of an int
vitessio/vitess#5262: VtCompose - Local Vitess Cluster (Docker) Auto-Setup
vitessio/vitess#5264: tabletmanager: Keep tablet and shard in sync.
vitessio/vitess#5268: treat the type of 'dual' table as 'TypeReference'
vitessio/vitess#5271: Fix invocation of deferred function for loading schema
vitessio/vitess#5277: add support for DEFAULT in insert statements for sequences to the query planner
vitessio/vitess#5280: Move vtCompose into own directory to avoid conflicting main() functions
vitessio/vitess#5290: MultiSplitDiff support in workflow
vitessio/vitess#5293: Make DemoteMaster idempotent
vitessio/vitess#5301: Added support for CHECK in Parser Grammar
vitessio/vitess#5312:split unit and unit_race between different shards, reduce load on shard 0
vitessio/vitess#5314:Allow vtbackup to run as a daemon
vitessio/vitess#5315:migrater: handle tables based on the vschema info
vitessio/vitess#5316: InitTablet should not update master alias on shard record
vitessio/vitess#5321: Add read-only security_policy to block admin HTTP endpoints
vitessio/vitess#5323: Cat error log when mysqld fails to start
vitessio/vitess#5324: Added foreign_key_checks to ignored query set
vitessio/vitess#5325: migrater: reverse_replication and other solidification
vitessio/vitess#5336: Add MYSQL8 new keywords
vitessio/vitess#5338: Add division operator for sql parser
vitessio/vitess#5347: migrater: option to cancel a migration
vitessio/vitess#5348: Clean up temp dir after xtrabackup restore
vitessio/vitess#5349: examples/local/401_teardown start from any point vitessio#5349
vitessio/vitess#5351: compute total size of files to be backed up and use that for xtrabackup
vitessio/vitess#5360: vttablet-up and vtgate-up should block until ready
vitessio/vitess#5361: Examples: search for mysqld instead of mysqld_safe
vitessio/vitess#5363: EmergencyReparentShard does not need to update shard master
vitessio/vitess#5366: Implement a small subset of config file refactor
vitessio/vitess#5367: vreplication: vdiff
vitessio/vitess#5376: Fix more known-recoverable problems
vitessio/vitess#5379: Don't set RestorePosition in local_metadata
vitessio/vitess#5380: Change local_metadata.value to MEDIUMBLOB
vitessio/vitess#5383: Don't recheck mysqld port if it was provided explicitly
vitessio/vitess#5384: Fix usage of arch to work on OSX
vitessio/vitess#5391: Make reparents more robust
vitessio/vitess#5398: mysql: handle db name correctly
vitessio/vitess#5400: [JAVA] Vitess JDBC release 4.0
vitessio/vitess#5404: VStreams should respect FieldEventMode
vitessio/vitess#5407: Stop nagging about where go is installed
vitessio/vitess#5419: Add stronger protection against running as root