Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve test stability #3989

Closed
spolitov opened this issue Mar 17, 2020 · 0 comments
Closed

Improve test stability #3989

spolitov opened this issue Mar 17, 2020 · 0 comments
Assignees
Labels
area/docdb YugabyteDB core features

Comments

@spolitov
Copy link
Contributor

spolitov commented Mar 17, 2020

No description provided.

@spolitov spolitov added the area/docdb YugabyteDB core features label Mar 17, 2020
@spolitov spolitov self-assigned this Mar 17, 2020
@spolitov spolitov added this to To Do in YBase features via automation Mar 17, 2020
spolitov added a commit that referenced this issue Mar 18, 2020
…est.PreserveLogs

Summary:
We have check that there are were several log segments at end of the QLTransactionTest.PreserveLogs.
It is ok to have this number smaller that half of transactions, we should just check that there are multiple of them.
To make sure that test did his check.

This diff relaxes requirement, to avoid test flakiness.

Test Plan: ybd --gtest_filter QLTransactionTest.PreserveLogs -n 100 -- -p 8

Reviewers: bogdan

Reviewed By: bogdan

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8145
spolitov added a commit that referenced this issue Mar 19, 2020
Summary:
During SealTxnTest.NumBatchesDisable we perform cluster restart right after writing data.
Sometimes it causes transaction to expire.
While it is expected, we could avoid handling this scenario in this particular test by disabling transaction timeouts.

Test Plan: ybd debug --gtest_filter SealTxnTest.NumBatchesDisable -n 100 -- -p 8

Reviewers: bogdan

Reviewed By: bogdan

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8151
spolitov added a commit that referenced this issue Mar 19, 2020
…t.MultiBankAccountSnapshot

Summary:
The test fails because of frequent read restart during read.
Use UNION to fetch all balances in one shot, to decrease read restart probability.
Increased test timeout to avoid test flakiness.

Test Plan: ybd --cxx-test pgwrapper_pg_libpq-test --gtest_filter PgLibPqTest.MultiBankAccountSnapshot --clang -n 100

Reviewers: bogdan, alex

Reviewed By: bogdan, alex

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8150
spolitov added a commit that referenced this issue Mar 19, 2020
Summary:
We restart cluster in this test, so write could time out.
We could ignore this failure, since we count only successfully committed transactions.

Test Plan: ybd --gtest_filter SnapshotTxnTest.RemoteBootstrapOnStart -n 100 -- -p 8

Reviewers: bogdan

Reviewed By: bogdan

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8154
spolitov added a commit that referenced this issue Mar 25, 2020
…shutdown

Summary:
In YSQL layer we could run YBCPgGetCatalogMasterVersion and close client immediately.
It results in heap-use-after-free error:

```
[ts-1]     #0 0x7f006c05b5a5 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/opt/yb-build/thirdparty/yugabyte-db-thirdparty-v20191209181439-7fc63d1583-centos/installed/asan/libcxx/lib/libc++.so.1+0x2065a5)
[ts-1]     #1 0x7f007d8cc059 in yb::HostPort::HostPort(yb::HostPort const&) /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/util/net/net_util.h:49:7
[ts-1]     #2 0x7f007d8f61b7 in yb::client::YBClient::Data::leader_master_hostport() const /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/client/client-internal.cc:1825:10
[ts-1]     #3 0x7f007d9306a5 in yb::Status yb::client::YBClient::Data::SyncLeaderMasterRpc<yb::master::GetYsqlCatalogConfigRequestPB, yb::master::GetYsqlCatalogConfigResponsePB>(std::__1::chrono::time_point<yb::CoarseMonoClock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >, yb::client::YBClient*, yb::master::GetYsqlCatalogConfigRequestPB const&, yb::master::GetYsqlCatalogConfigResponsePB*, int*, char const*, std::__1::function<yb::Status (yb::master::MasterServiceProxy*, yb::master::GetYsqlCatalogConfigRequestPB const&, yb::master::GetYsqlCatalogConfigResponsePB*, yb::rpc::RpcController*)> const&) /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/client/client-internal.cc:192:40
[ts-1]     #4 0x7f007d8abc33 in yb::client::YBClient::GetYsqlCatalogMasterVersion(unsigned long*) /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/client/client.cc:654:3
[ts-1]     #5 0x7f00800a14f5 in yb::pggate::PgSession::GetCatalogMasterVersion(unsigned long*) /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/yql/pggate/pg_session.cc:457:19
[ts-1]     #6 0x7f008007be7c in yb::pggate::PgApiImpl::GetCatalogMasterVersion(unsigned long*) /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/yql/pggate/pggate.cc:342:23
[ts-1]     #7 0x7f0080067f27 in YBCPgGetCatalogMasterVersion /nfusr/centos-gcp-cloud/jenkins-slave-150/jenkins/jenkins-github-yugabyte-db-centos-master-clang-asan-758/build/asan-clang-dynamic-ninja/../../src/yb/yql/pggate/ybc_pggate.cc:179:29
```

Fixed by forcing YBClient to wait for all sync operations to complete.

Test Plan: ybd asan --gtest_filter PgLibPqTest.TxnConflictsForColocatedTables

Reviewers: dmitry

Reviewed By: dmitry

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8178
spolitov added a commit that referenced this issue Apr 6, 2020
…llisionTest

Summary:
This diff fixes concurrent access to metadata cache in tablet.
That is being overwritten during alter table, and frequently accessed during other operations.

Test Plan: ybd tsan --gtest_filter CppCassandraDriverTest.TestCreateIdxTripleCollisionTest

Reviewers: amitanand

Reviewed By: amitanand

Subscribers: bogdan, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8243
spolitov added a commit that referenced this issue Apr 17, 2020
Summary:
It is possible that during shutdown tablet peer was already reset, while snapshot coordinator is still alive.
In this case it calls CatalogManager::IsLeader, that has null pointer to tablet peer.

Added check for this case.

Test Plan: ybd --java-test org.yb.loadtester.TestClusterFullMove#testClusterFullMove -n 20

Reviewers: dmitry, timur

Reviewed By: timur

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D8321
spolitov added a commit that referenced this issue Jul 11, 2020
Summary:
This diff fixes 2 issues related to OperationTracker::WaitForAllToFinish.
1) MonoDelta was wrong sanity check during multiplication by zero, that is used in WaitForAllToFinish.
   Fixed by updating logic in MonoDelta.

2) New operations could arrive into operation tracker while WaitForAllToFinish performs sleep, so it will wait until deadline.
   Fixed by introducing condition variable and waiting on it.

Test Plan: ybd asan --gtest_filter BackupTxnTest.Restart

Reviewers: timur, bogdan

Reviewed By: bogdan

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D8875
YBase features automation moved this from Backlog to Done Oct 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/docdb YugabyteDB core features
Projects
Development

No branches or pull requests

2 participants