serious memory bug on multiple update a table in a single transaction [CORE1575]

[firebird-automations]

Submitted by: Daniel Bauten (danielbauten)

Is related to CORE381
Relate to CORE1477

Attachments:
MemoryBug.zip

Votes: 1

Dear all,

I found a serious bug leading Firebird in allocating a very large amount of RAM - in my example (with 80.000 records) it crashes after allocating 1.7 GB of RAM.

You can reproduce this bug by using the attached database "MemoryBug.fdb" by execute the following two queries WITHOUT commiting after the first query.

Query A:

update mytable
set CIS = 0;

==> Firebird uses (with my DefaultDBCachePages) approx. 90 MB of RAM.

Query B:

update mytable
set CIS = 1;

==> use of RAM for the Firebird servers grows in a few seconds up to 1.7 GB + and crashes.

Note 1: It seems that updating records several times in the SAME transaction leads in this bug. If one commits after the first query, every works fine.

Note 2: Using a stored procedure including the two above statements DOESN'T lead in growing memory that much (just up to about 180 MB in my case).

Note 3: Using tables with less fields as in my example seems to work correctly. Therefore: Use my example database to reproduce the bug!

Note 4: I guess the bug reported by Eugene Kuznetsov in CORE1477 is the same.

It is essential for the further development of my project to have a solution for this. So please, let me know, when this bug will be fixed.

Thanks in advance.

Daniel Bauten

Commits: e0db5f3 1a943d1

[firebird-automations]

Commented by: Daniel Bauten (danielbauten)

A small database with one table ('MYTABLE') containing about 80.000 records.

Just use the following two queries WITHOUT commiting after the first one to "see" the bug reported in CORE1575.

update mytable
set CIS = 0;

update mytable
set CIS = 1;

[firebird-automations]

Modified by: Daniel Bauten (danielbauten)

Attachment: MemoryBug.zip [ 10642 ]

[firebird-automations]

Modified by: @dyemanov

assignee: Dmitry Yemanov [ dimitr ]

[firebird-automations]

Modified by: @dyemanov

Link: This issue is related to CORE381 [ CORE381 ]

[firebird-automations]

Modified by: @dyemanov

Link: This issue relate to CORE1477 [ CORE1477 ]

[firebird-automations]

Commented by: Daniel Bauten (danielbauten)

Dear Dmitry,

three years ago for „CORE381" you wrote that this is a design pitfall...

As I mentioned this bug leads in several problems for my actual project. Can you tell me when this will be fixed?

Thanks in advance for your help, Dmitry.

Daniel

[firebird-automations]

Commented by: @dyemanov

Second update in the same transaction transforms the delta record to a full record version, hence requiring more I/O. This is by design and unlikely to change. The big memory consumption is another (although related) issue which hopefully can be optimized. I have a draft solution ready, but it requires some more work.

[firebird-automations]

Commented by: Daniel Bauten (danielbauten)

I see. Then I will check the memory consumption for "my database" after CORE1477 is fixed. As you have a draft solution, is it possible to fix the big memory consumption (i.e. CORE1477) in Firebird 2.1 Final?

[firebird-automations]

Modified by: @dyemanov

Target: 2.5.0 [ 10221 ]

[firebird-automations]

Modified by: @pcisar

Workflow: jira [ 13390 ] => Firebird [ 14058 ]

[firebird-automations]

Modified by: @dyemanov

status: Open [ 1 ] => Open [ 1 ]

Fix Version: 2.5 Beta 1 [ 10251 ]

[firebird-automations]

Modified by: @dyemanov

status: Open [ 1 ] => In Progress [ 3 ]

[firebird-automations]

Modified by: @dyemanov

status: In Progress [ 3 ] => Open [ 1 ]

[firebird-automations]

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

[firebird-automations]

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

[firebird-automations]

Modified by: @pavel-zotov

QA Status: No test