Skip to content
This repository has been archived by the owner on Dec 8, 2021. It is now read-only.

dep: update uuid dependency to latest google/uuid #452

Merged
merged 7 commits into from
Nov 6, 2020

Conversation

lance6716
Copy link
Contributor

What problem does this PR solve?

after hiding lighting's go.mod and embedded it into tidb, an error checkpoint tool will raise error like

$ ./tools/check/check-errdoc.sh
Generate errors.toml failed: exit status 2, output:
# github.com/pingcap/tidb-lightning/lightning/backend
../../../../pkg/mod/github.com/pingcap/tidb-lightning@v4.0.9-0.20201104053247-8013216a3446+incompatible/lightning/backend/local.go:486:26: multiple-value uuid.NewV4() in single-value context

What is changed and how it works?

update satori/go.uuid to latest

Check List

Tests

  • pass original test

Side effects

Related changes

@lance6716 lance6716 requested review from glorv, 3pointer and kennytm and removed request for glorv and 3pointer November 5, 2020 08:44
@glorv
Copy link
Contributor

glorv commented Nov 5, 2020

@lance6716 please fix the failed unit test

@lance6716
Copy link
Contributor Author

@glorv CI pass

@glorv
Copy link
Contributor

glorv commented Nov 5, 2020

Shall we move to github.com/google/uuid, the same library as tidb?

@lance6716
Copy link
Contributor Author

Shall we move to github.com/google/uuid, the same library as tidb?

there're some uuid.NewV5 usage like

engineUUID := uuid.NewV5(engineNamespace, tag)

and through a glance I think github.com/google/uuid doesn't support version 5.

@glorv
Copy link
Contributor

glorv commented Nov 5, 2020

@lance6716 uuid.V5 is not a big issue, that just generate a uniq sign to each engine. Yout can change to any other implement in google/uuid you like

@lance6716 lance6716 added the status/WIP Work in progress label Nov 5, 2020
@lance6716 lance6716 added status/PTAL This PR is ready for review. Add this label back after committing new changes and removed status/WIP Work in progress labels Nov 5, 2020
@lance6716
Copy link
Contributor Author

CI passed @glorv

@lance6716 lance6716 changed the title dep: update satori/go.uuid to latest dep: update uuid dependency to latest google/uuid Nov 5, 2020
Copy link
Collaborator

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

lightning/backend/backend.go Outdated Show resolved Hide resolved
lightning/backend/backend_test.go Outdated Show resolved Hide resolved
lightning/backend/importer.go Outdated Show resolved Hide resolved
lightning/backend/importer_test.go Outdated Show resolved Hide resolved
lightning/backend/local.go Outdated Show resolved Hide resolved
lightning/restore/restore_test.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kennytm kennytm added status/LGT1 One reviewer already commented LGTM (LGTM1) and removed status/PTAL This PR is ready for review. Add this label back after committing new changes labels Nov 6, 2020
@glorv
Copy link
Contributor

glorv commented Nov 6, 2020

LGTM

@glorv glorv removed the status/LGT1 One reviewer already commented LGTM (LGTM1) label Nov 6, 2020
@glorv glorv added the status/LGT2 Two reviewers already commented LGTM, ready for merge (LGTM2) label Nov 6, 2020
@glorv glorv merged commit a1ac978 into pingcap:master Nov 6, 2020
@lance6716 lance6716 deleted the update-dep branch November 6, 2020 04:18
lance6716 added a commit to lance6716/tidb-lightning that referenced this pull request Nov 17, 2020
add notes

save work

save work

fix unit test

remove tidbMgr in RestoreController

remove some comments

remove some comments

change logger in SQLWithRetry

revert replace log.Logger to *zap.Logger

dep: update uuid dependency to latest google/uuid (pingcap#452)

* dep: update satori/go.uuid to latest

* fix tests

* change to google/uuid

* fix build

* try fix test

* get familiar with google/uuid

* address comment

tidb-lightning-ctl: change default of -d to 'noop://' (pingcap#453)

also add noop:// to supported storage types (to represent an empty store)

replace tab to space

try another port to fix CI

remove some comment

*: more glue

restore: fix the bug that gc life time ttl does not take effect (pingcap#448)

* fix gc ttl loop

* resolve comment and add tests

fix CI

report info to host TiDB

config: filter out all system schemas by default (pingcap#459)

backend: fix auto random default value for primary key (pingcap#457)

* fix auto generate auto random primary key column

* fix default for auto random primary key

* fix test

* use prev row id for auto random and add a test

* replace chunck with session opt

* fix

* fix

mydumper: fix parquet data parser (pingcap#435)

* fix parquet

* reorder imports

* fix test

* use empty collation

* fix a error and add more test cases

* add pointer type tests

* resolve comments

Co-authored-by: kennytm <kennytm@gmail.com>

address comment

backend/local: use range properties to optimize region range estimate (pingcap#422)

* use range propreties to estimate region range

* post-restore: add optional level for post-restore operations (pingcap#421)

* add optional level for opst-restore operations

* trim leading and suffix '"

* use UnmarshalTOML to unmarshal post restore op level

* resolve comments and fix unit test

* backend/local: do not retry epochNotMatch error when ingest sst (pingcap#419)

* do not retry epochNotMatch error when ingest sst

* add retry ingest for 'Raft raft: proposal dropped' error in ingest

* change some retryable error log level from Error to Warn

* fix nextKey

* add a comment for nextKey

* fix comment and add a unit test

* wrap time.Sleep in select

Co-authored-by: kennytm <kennytm@gmail.com>

* update

* use range properties to optimze region range estimate

* update pebble

* change the default value for batch-size

* add unit tests and reslove comments

* add a comment to range properties test

* add a comment

* add a test for range property with pebble

* rename const variable

Co-authored-by: kennytm <kennytm@gmail.com>

fix pd service id is empty (pingcap#460)

fix s3 parquet reader (pingcap#461)

Co-authored-by: Neil Shen <overvenus@gmail.com>

fix service gc ttl again (pingcap#465)

address comment

mydumper: verify file routing config (pingcap#470)

* fix file routing

* remove useless line

* remove redundant if check

rename a method in interface

save work

try fix CI

could work

change ctx usage

try fix CI

try fix CI

refine function interface

refine some fucntion interface

debug CI

address comment

config: allow four byte-size config to be specified using human-readable units ("100 GiB") (pingcap#471)

* Makefile: add `make finish-prepare` action

* config: accept human-readable size for most byte-related config

e.g. allow `region-split-size = '96M'` in additional to `= 100663296`

(known issue: these values' precisions will be truncated to 53 bits
instead of supporting all 63 bits)

* restore: reduce chance of spurious errors from TestGcTTLManagerSingle

Co-authored-by: glorv <glorvs@163.com>

remove debug log

test: change double type syntax (pingcap#474)

address comment

checkpoint: add glue checkpoint

resolve cycle import

expose Retry

refine

change interface to cope with TiDB

fix SQL string

fix SQL

adjust interface to embedded in TiDB

could import now

reduce TLS

restore: add `glue.Glue` interface and other function (pingcap#456)

* save my work

* add notes

* save work

* save work

* fix unit test

* remove tidbMgr in RestoreController

* remove some comments

* remove some comments

* change logger in SQLWithRetry

* revert replace log.Logger to *zap.Logger

* replace tab to space

* try another port to fix CI

* remove some comment

* *: more glue

* report info to host TiDB

* fix CI

* address comment

* address comment

* rename a method in interface

* save work

* try fix CI

* could work

* change ctx usage

* try fix CI

* try fix CI

* refine function interface

* refine some fucntion interface

* debug CI

* address comment

* remove debug log

* address comment

modify code

add comment

refine some code
lance6716 added a commit to lance6716/tidb-lightning that referenced this pull request Nov 17, 2020
add notes

save work

save work

fix unit test

remove tidbMgr in RestoreController

remove some comments

remove some comments

change logger in SQLWithRetry

revert replace log.Logger to *zap.Logger

dep: update uuid dependency to latest google/uuid (pingcap#452)

* dep: update satori/go.uuid to latest

* fix tests

* change to google/uuid

* fix build

* try fix test

* get familiar with google/uuid

* address comment

tidb-lightning-ctl: change default of -d to 'noop://' (pingcap#453)

also add noop:// to supported storage types (to represent an empty store)

replace tab to space

try another port to fix CI

remove some comment

*: more glue

restore: fix the bug that gc life time ttl does not take effect (pingcap#448)

* fix gc ttl loop

* resolve comment and add tests

fix CI

report info to host TiDB

config: filter out all system schemas by default (pingcap#459)

backend: fix auto random default value for primary key (pingcap#457)

* fix auto generate auto random primary key column

* fix default for auto random primary key

* fix test

* use prev row id for auto random and add a test

* replace chunck with session opt

* fix

* fix

mydumper: fix parquet data parser (pingcap#435)

* fix parquet

* reorder imports

* fix test

* use empty collation

* fix a error and add more test cases

* add pointer type tests

* resolve comments

Co-authored-by: kennytm <kennytm@gmail.com>

address comment

backend/local: use range properties to optimize region range estimate (pingcap#422)

* use range propreties to estimate region range

* post-restore: add optional level for post-restore operations (pingcap#421)

* add optional level for opst-restore operations

* trim leading and suffix '"

* use UnmarshalTOML to unmarshal post restore op level

* resolve comments and fix unit test

* backend/local: do not retry epochNotMatch error when ingest sst (pingcap#419)

* do not retry epochNotMatch error when ingest sst

* add retry ingest for 'Raft raft: proposal dropped' error in ingest

* change some retryable error log level from Error to Warn

* fix nextKey

* add a comment for nextKey

* fix comment and add a unit test

* wrap time.Sleep in select

Co-authored-by: kennytm <kennytm@gmail.com>

* update

* use range properties to optimze region range estimate

* update pebble

* change the default value for batch-size

* add unit tests and reslove comments

* add a comment to range properties test

* add a comment

* add a test for range property with pebble

* rename const variable

Co-authored-by: kennytm <kennytm@gmail.com>

fix pd service id is empty (pingcap#460)

fix s3 parquet reader (pingcap#461)

Co-authored-by: Neil Shen <overvenus@gmail.com>

fix service gc ttl again (pingcap#465)

address comment

mydumper: verify file routing config (pingcap#470)

* fix file routing

* remove useless line

* remove redundant if check

rename a method in interface

save work

try fix CI

could work

change ctx usage

try fix CI

try fix CI

refine function interface

refine some fucntion interface

debug CI

address comment

config: allow four byte-size config to be specified using human-readable units ("100 GiB") (pingcap#471)

* Makefile: add `make finish-prepare` action

* config: accept human-readable size for most byte-related config

e.g. allow `region-split-size = '96M'` in additional to `= 100663296`

(known issue: these values' precisions will be truncated to 53 bits
instead of supporting all 63 bits)

* restore: reduce chance of spurious errors from TestGcTTLManagerSingle

Co-authored-by: glorv <glorvs@163.com>

remove debug log

test: change double type syntax (pingcap#474)

address comment

checkpoint: add glue checkpoint

resolve cycle import

expose Retry

refine

change interface to cope with TiDB

fix SQL string

fix SQL

adjust interface to embedded in TiDB

could import now

reduce TLS

restore: add `glue.Glue` interface and other function (pingcap#456)

* save my work

* add notes

* save work

* save work

* fix unit test

* remove tidbMgr in RestoreController

* remove some comments

* remove some comments

* change logger in SQLWithRetry

* revert replace log.Logger to *zap.Logger

* replace tab to space

* try another port to fix CI

* remove some comment

* *: more glue

* report info to host TiDB

* fix CI

* address comment

* address comment

* rename a method in interface

* save work

* try fix CI

* could work

* change ctx usage

* try fix CI

* try fix CI

* refine function interface

* refine some fucntion interface

* debug CI

* address comment

* remove debug log

* address comment

modify code

add comment

refine some code
glorv pushed a commit that referenced this pull request Nov 23, 2020
* save my work

add notes

save work

save work

fix unit test

remove tidbMgr in RestoreController

remove some comments

remove some comments

change logger in SQLWithRetry

revert replace log.Logger to *zap.Logger

dep: update uuid dependency to latest google/uuid (#452)

* dep: update satori/go.uuid to latest

* fix tests

* change to google/uuid

* fix build

* try fix test

* get familiar with google/uuid

* address comment

tidb-lightning-ctl: change default of -d to 'noop://' (#453)

also add noop:// to supported storage types (to represent an empty store)

replace tab to space

try another port to fix CI

remove some comment

*: more glue

restore: fix the bug that gc life time ttl does not take effect (#448)

* fix gc ttl loop

* resolve comment and add tests

fix CI

report info to host TiDB

config: filter out all system schemas by default (#459)

backend: fix auto random default value for primary key (#457)

* fix auto generate auto random primary key column

* fix default for auto random primary key

* fix test

* use prev row id for auto random and add a test

* replace chunck with session opt

* fix

* fix

mydumper: fix parquet data parser (#435)

* fix parquet

* reorder imports

* fix test

* use empty collation

* fix a error and add more test cases

* add pointer type tests

* resolve comments

Co-authored-by: kennytm <kennytm@gmail.com>

address comment

backend/local: use range properties to optimize region range estimate (#422)

* use range propreties to estimate region range

* post-restore: add optional level for post-restore operations (#421)

* add optional level for opst-restore operations

* trim leading and suffix '"

* use UnmarshalTOML to unmarshal post restore op level

* resolve comments and fix unit test

* backend/local: do not retry epochNotMatch error when ingest sst (#419)

* do not retry epochNotMatch error when ingest sst

* add retry ingest for 'Raft raft: proposal dropped' error in ingest

* change some retryable error log level from Error to Warn

* fix nextKey

* add a comment for nextKey

* fix comment and add a unit test

* wrap time.Sleep in select

Co-authored-by: kennytm <kennytm@gmail.com>

* update

* use range properties to optimze region range estimate

* update pebble

* change the default value for batch-size

* add unit tests and reslove comments

* add a comment to range properties test

* add a comment

* add a test for range property with pebble

* rename const variable

Co-authored-by: kennytm <kennytm@gmail.com>

fix pd service id is empty (#460)

fix s3 parquet reader (#461)

Co-authored-by: Neil Shen <overvenus@gmail.com>

fix service gc ttl again (#465)

address comment

mydumper: verify file routing config (#470)

* fix file routing

* remove useless line

* remove redundant if check

rename a method in interface

save work

try fix CI

could work

change ctx usage

try fix CI

try fix CI

refine function interface

refine some fucntion interface

debug CI

address comment

config: allow four byte-size config to be specified using human-readable units ("100 GiB") (#471)

* Makefile: add `make finish-prepare` action

* config: accept human-readable size for most byte-related config

e.g. allow `region-split-size = '96M'` in additional to `= 100663296`

(known issue: these values' precisions will be truncated to 53 bits
instead of supporting all 63 bits)

* restore: reduce chance of spurious errors from TestGcTTLManagerSingle

Co-authored-by: glorv <glorvs@163.com>

remove debug log

test: change double type syntax (#474)

address comment

checkpoint: add glue checkpoint

resolve cycle import

expose Retry

refine

change interface to cope with TiDB

fix SQL string

fix SQL

adjust interface to embedded in TiDB

could import now

reduce TLS

restore: add `glue.Glue` interface and other function (#456)

* save my work

* add notes

* save work

* save work

* fix unit test

* remove tidbMgr in RestoreController

* remove some comments

* remove some comments

* change logger in SQLWithRetry

* revert replace log.Logger to *zap.Logger

* replace tab to space

* try another port to fix CI

* remove some comment

* *: more glue

* report info to host TiDB

* fix CI

* address comment

* address comment

* rename a method in interface

* save work

* try fix CI

* could work

* change ctx usage

* try fix CI

* try fix CI

* refine function interface

* refine some fucntion interface

* debug CI

* address comment

* remove debug log

* address comment

modify code

add comment

refine some code

* address comment

* add some comments

* fix CI and change CREATE TABLE
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
status/LGT2 Two reviewers already commented LGTM, ready for merge (LGTM2)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants