Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests under TSAN: yb-master: Memory usage exceeded configured limit #4688

Open
ttyusupov opened this issue Jun 4, 2020 · 0 comments
Open

Tests under TSAN: yb-master: Memory usage exceeded configured limit #4688

ttyusupov opened this issue Jun 4, 2020 · 0 comments
Labels
kind/failing-test Tests and testing infra
Projects
YBase features

Comments

@ttyusupov
Copy link
Contributor

ttyusupov commented Jun 4, 2020 •
edited by daniel-yb

There are number of tests failing under TSAN with yb-master: Memory usage exceeded configured limit, terminating the process: Non-shared RSS (2744156160 bytes, 2617 MiB) has exceeded the configured limit (2684354560 bytes, 2560 MiB):

find ~/jobs -type f \( -name "*.log" -o -name "*.txt.gz" \) -print0 | xargs -0 -I{} zgrep --with-filename "Memory usage exceeded configured limit" {} | tee ~/mem-usage-exceeded.txt

jenkins@jenkins-master ~$ cat ~/mem-usage-exceeded.txt | pcregrep --om-separator=": " -o1 -o2 ".*/jobs/([^/]+)/.*/archive/.*/([^/:]*):" | sort | uniq -c | sort -rh
    170 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestAuthorizationEnforcement-output.txt.gz
     51 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.jedis.TestYBJedis-output.txt.gz
     29 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestIndex-output.txt.gz
     26 github-yugabyte-db-centos-master-clang-tsan: CppCassandraDriverTest_Rejection.log
     25 github-yugabyte-db-centos-master-clang-tsan: org.yb.client.TestYBClient-output.txt.gz
     22 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestSelect-output.txt.gz
     20 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestAuthentication-output.txt.gz
     19 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.BinaryValuesCommandsTest-output.txt.gz
     19 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.jedis.TestReadFromFollowers-output.txt.gz
     18 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestRoles-output.txt.gz
     17 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.PipeliningTest-output.txt.gz
     17 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.SimpleQueryTest-output.txt.gz
     16 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.JedisPoolTest-output.txt.gz
     16 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.AllKindOfValuesCommandsTest-output.txt.gz
     16 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBindVariable-output.txt.gz
     15 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTransaction-output.txt.gz
     15 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestKeyspace-output.txt.gz
     15 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestInsertJson-output.txt.gz
     15 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.jedis.TestYBJedisTSRevRangeByTime-output.txt.gz
     14 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestCreateTableWithProperties-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.ShardedJedisTest-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.PublishSubscribeCommandsTest-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestVarIntDataType-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTimestampDataType-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBatchRequest-output.txt.gz
     13 github-yugabyte-db-centos-master-clang-tsan: CppCassandraDriverTest_ConcurrentIndexUpdate.log
     12 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.ProtocolTest-output.txt.gz
     12 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.StringValuesCommandsTest-output.txt.gz
     12 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTableTTL-output.txt.gz
     12 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestDecimalDataType-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.HashesCommandsTest-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTTLSemantics-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTimeuuidFunctions-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestStaticColumn-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestOrderedColumns-output.txt.gz
     11 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestInsertValues-output.txt.gz
     10 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.SortedSetCommandsTest-output.txt.gz
     10 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestSystemTables-output.txt.gz
      9 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestDelete-output.txt.gz
      8 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestUserDefinedTypes-output.txt.gz
      7 github-yugabyte-db-phabricator: org.yb.cql.TestTransactionStatusTable-output.txt.gz
      7 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestSelectAggr-output.txt.gz
      7 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestFloatLiterals-output.txt.gz
      7 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestConditionalDml-output.txt.gz
      7 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBasicStatements-output.txt.gz
      7 github-yugabyte-db-centos-master-clang-tsan: org.yb.client.TestYBTable-output.txt.gz
      6 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestUsingTimestamp-output.txt.gz
      6 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTransactionStatusTable-output.txt.gz
      6 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestReturnsClause-output.txt.gz
      6 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestCollectionTypes-output.txt.gz
      6 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.driver.core.policies.TestLoadBalancingPolicy-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.JedisTest-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.SetCommandsTest-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestUpdate-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestExplainPlan-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestConsistencyLevels-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestCollectionExpressions-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBooleanDataType-output.txt.gz
      5 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBlobFunctions-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.utils.JedisURIHelperTest-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.ConnectionTest-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.ConnectionCloseTest-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.VariadicCommandsTest-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTuple-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestKeyspaceProperties-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestJsonIndex-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestFrozenType-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestCreateTable-output.txt.gz
      4 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestArith-output.txt.gz
      3 github-yugabyte-db-phabricator: CppCassandraDriverTest_ConcurrentIndexUpdate.log
      3 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.ListCommandsTest-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.HostAndPortTest-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestFullMoveWithHeartBeatDelay-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestClusterExpandShrink-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestTruncate-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestPrepareExecute-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestPagingSelect-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestDateTime-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestAlterKeyspace-output.txt.gz
      3 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.driver.core.TestPartitionMetadata-output.txt.gz
      2 github-yugabyte-db-phabricator: org.yb.cql.TestIndex-output.txt.gz
      2 github-yugabyte-db-phabricator: CppCassandraDriverTest_Rejection.log
      2 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.TupleTest-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.utils.JedisClusterCRC16Test-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.KeyMergeUtilTest-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.ConnectionHandlingCommandsTest-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.collections.SetFromListTest-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestSparkLocality-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestMasterLeaderDecommission-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestClusterWithHighLoadAndSlowSync-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestCassandraSparkWordCount-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestWithMasterLatency-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestUseKeyspace-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestSelectIf-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestMessageCompression-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestBlobDataType-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestAlterTable-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: org.yb.clientent.TestReadReplicas-output.txt.gz
      2 github-yugabyte-db-centos-master-clang-tsan: com.yugabyte.jedis.TestYBJedisCluster-output.txt.gz
      1 github-yugabyte-db-ph-mb-tst: org.yb.loadtester.TestFullMoveWithHeartBeatDelay-output.txt.gz
      1 github-yugabyte-db-ph-mb-tst: CppCassandraDriverTest_Rejection.log
      1 github-yugabyte-db-phabricator: org.yb.cql.TestSelect-output.txt.gz
      1 github-yugabyte-db-phabricator: org.yb.cql.TestAuthentication-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.commands.ControlCommandsTest-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: redis.clients.jedis.tests.BuilderFactoryTest-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestRFChange-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.loadtester.TestClusterFullMove-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestUuidBuiltIn-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestSelectNoHash-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestJson-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestHideRedis-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.cql.TestEditCQLNodes-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.client.TestQLType-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: org.yb.client.TestMasterFailover-output.txt.gz
      1 github-yugabyte-db-centos-master-clang-tsan: CppCassandraDriverTest_TestTableCreateUniqueIndexCovered.log
      1 github-yugabyte-db-centos-master-clang-tsan: CppCassandraDriverTest_TestTableCreateIndex.log
@ttyusupov ttyusupov added the kind/failing-test Tests and testing infra label Jun 4, 2020
@bmatican bmatican added this to To Do in YBase features Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/failing-test Tests and testing infra
Projects
YBase features
  
Backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ttyusupov