byte size for ets record?

[redink]

Hi,
I am developing a cache via ets, one feature is single record memory usage limit. That is , if a record inserted into ets table need too large memory, I will delete it . Conversely , retain it .
operation process :

ets:info(T, memory) --> ets:insert(T, RecordA) --> ets:info(T, memory)

But, I think this method is not smart. So, I finding another way, get the single record memory usage without insert ets table. I found your repo through erts_debug:flat_size/1 from Erlang-MailList.

Soon, I got a question, let me show you my test code:

1> A = ets:new(a, [named_table, public]). %% create an ets table
a
2> D = {[{} || _ <- lists:seq(1, 100)], <<"handling">>, [self() || _ <- lists:seq(1, 10)], [{<<"1234567890">>, {}} || _ <- lists:seq(1, 1000)]}. %% prepare data
...
3> erlang_term:byte_size(D)/8.  %% get the data words (64-bits)
12332.0
4> ets:info(A, memory).   %% get the ets table memory before insert data
305
5> ets:insert(A, D).
true
6> ets:info(A, memory). %% get the ets table memory after insert data
12640
7> v(6) - v(4).
12335

Indeed, I got two different result via above test code. I can not understand why. Would you help me please ? Thanks very much.

[okeuday]

Part of the difference could be sharing the term so that a size is only counted once (i.e., not a flat size), but the main difference should be how the shell reports the wrong size for binaries. It is better if you test this in a compiled module. Either way, due to terms being used in ways where the size only counts once, it is possible the exact size value may differ slightly. The erlang_term module is giving the upper limit on the memory for storing the term (when not used in the shell, since the shell does not report the correct sizes). So, the erlang_term module can help you manage caching, but the real situation in the Erlang VM with the many memory pools is more complex.

[redink]

Thanks very much for reply.

since the shell does not report the correct sizes

Yes, I tested this in a compiled module :

$ cat test_for_ets_record.erl 
-module(test_for_ets_record).

-compile(export_all).

start() ->
    A = ets:new(a, [named_table, public]),
    D = {[{} || _ <- lists:seq(1, 100)], <<"handling">>, [self() || _ <- lists:seq(1, 10)], [{<<"1234567890">>, {}} || _ <- lists:seq(1, 1000)]},
    io:format(" ** data words size ~p~n", [erlang_term:byte_size(D)/8]),
    io:format(" ** before insert ~p~n", [ets:info(A, memory)]),
    ets:insert(A, D),
    io:format(" ** after insert ~p~n", [ets:info(A, memory)]).

the result is :

1> test_for_ets_record:start().
 ** data words size 10329.0
 ** before insert 305
 ** after insert 10637
ok

As you can see, the shell reported wrong sizes, but the difference is remains.
I guess, the difference due to management of ets memory and it is more complex.

Thanks very much again !

[okeuday]

No problem.