ERL-720: Memory leak in ETS set with write_concurrency option

[OTP-Maintainer]

Original reporter: kjellwinblad
Affected version: Not Specified
Fixed in version: OTP-21.1
Component: erts
Migrated from: https://bugs.erlang.org/browse/ERL-720

---

There seems to be a memory leak in the ETS table type set when the write_concurrency option turned on. This memory leak occur when there are concurrent insert/2, delete/2 and select_count/2 operations.

*How to reproduce*

1. Apply the attached git patch file (Also available here https://github.com/kjellwinblad/otp/commit/cb09dae130ee3b7096e3639889db911bd5f3b1e5)
2. In a terminal run:
3. {{export ERL_TOP=`pwd`}}
4. {{./otp_build tests}}
5. {{cd release/tests/test_server}}
6. {{ts:install().}}
7. {{ts:run(stdlib, ets_SUITE, test_concurrent_insert_remove_select, [batch]).}}

The test that step 7 runs fails which indicates that the set table with the write_concurrency option has a memory leak. The ordered_set table type does not seem to have this problem and the memory leak does not seem to occur if one one removes one of the tree sub processes that are started by the test (one do inserts, one do deletes and one selects).

*Configuration*
OTP: from master branch with commit id 869537a9bf799c8d12fc46c2b413e532d6e3b10c
OS: Linux
Processor: Intel i7
compiler:

$ gcc -v      
Target: x86_64-pc-linux-gnu
Thread model: posix
gcc version 8.2.0 (GCC)

Let me know if you need more information.

[OTP-Maintainer]

sverker said:

Thanks for the exemplary bug report, Kjell.

Fortunately this is not an actual memory leak but rather a bug in the stats reported by erlang:memory.
Here is a a preliminary fix:

{code:java}
diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c
index b988a19..b648965 100644
--- a/erts/emulator/beam/erl_db_hash.c
+++ b/erts/emulator/beam/erl_db_hash.c
@@ -162,6 +162,7 @@ static ERTS_INLINE int link_fixdel(DbTableHash* tb,
         if (NFIXED(tb) <= fixated_by_me) {
             erts_db_free(ERTS_ALC_T_DB_FIX_DEL, (DbTable*)tb,
                          fixd, sizeof(FixedDeletion));
+            ERTS_ETS_MISC_MEM_ADD(-sizeof(FixedDeletion));
             return 0; /* raced by unfixer */
         }
         exp_next = was_next;

{code}

[OTP-Maintainer]

sverker said:

Fix merged to maint (21.1) and master (22.0).