Skip to content
Permalink
Browse files
Bug #31576731 INNODB_FT_TOTAL_CACHE_SIZE NOT CAPPED WHEN SET TO 32000000
Problem:
Server throws OOM error when we execute twitter load with SELECTs
for UPDATE + UPDATES, and SELECT queries on tables with full-text
index.
FTS cache->total_memory store count of total memory allocated to FTS
cache (for all fulltext indexes on a table).
For each word in fts cache, we store doc-id & word position in a
node->ilist.
we increment cache->total_memory with size of doc-id & word position
whereas we allocate ilist in chuck of 16, 32 ,64 bytes or 1.2 times
of last size.
When we wil insert huge amount of data into the FTS aux index tables
then collectively these small chucks for each token become huge
unaccounted memory allocated for FTS cache.

Fix:
Incremented cache->total_memory by size of chunk allocated to
node->ilist.

RB: 25286
Reviewed by : Rahul Agarkar <rahul.agarkar@oracle.com>

mysql/mysql-server@7ab5707
  • Loading branch information
sachinagarwal1111 authored and dr-m committed Jul 22, 2021
1 parent c4295b9 commit 236f825
Showing 1 changed file with 8 additions and 6 deletions.
@@ -1,7 +1,7 @@
/*****************************************************************************
Copyright (c) 2011, 2018, Oracle and/or its affiliates. All Rights Reserved.
Copyright (c) 2016, 2020, MariaDB Corporation.
Copyright (c) 2011, 2021, Oracle and/or its affiliates.
Copyright (c) 2016, 2021, MariaDB Corporation.
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -1299,6 +1299,9 @@ fts_cache_node_add_positions(
ptr = ilist + node->ilist_size;

node->ilist_size_alloc = new_size;
if (cache) {
cache->total_size += new_size;
}
}

ptr_start = ptr;
@@ -1325,17 +1328,16 @@ fts_cache_node_add_positions(
if (node->ilist_size > 0) {
memcpy(ilist, node->ilist, node->ilist_size);
ut_free(node->ilist);
if (cache) {
cache->total_size -= node->ilist_size;
}
}

node->ilist = ilist;
}

node->ilist_size += enc_len;

if (cache) {
cache->total_size += enc_len;
}

if (node->first_doc_id == FTS_NULL_DOC_ID) {
node->first_doc_id = doc_id;
}

0 comments on commit 236f825

Please sign in to comment.