Firebird 1.5.3 SS crashes under load [CORE538]

[firebird-automations]

Submitted by: skaushanskiy (skaushanskiy)

Votes: 2

SFID: 1439774#⁠
Submitted By: skaushanskiy

When running over 50 concurrent long running read/write
transactions FB will crash periodically.

My Setup:
Firebird 1.5.3 SuperServer
GDS32.dll installed by instclient.exe i g
Apache2 2.0.52
PHP 5.1.2 with php_interbase.dll
Windows XP Pro SP2

Hardware:
3.0GHZ P4 HT
1GB Ram

I have posted this problem on the yahoo
firebird-support group, subject: Firebird Unstable
under load. Please refer to this thread for details.

This happens on Win XP Pro SP2, Win 2k Server SP4 and
Win 2003 Server SP1.

I have a minidump with heap saved from Visual Studio 7.
This dump was produced by running fbserver.exe -a
without the guardian. This crashed
and activated the VS7 JIT Debugger. Let me know who to
email the dump, its ~3MB with bzip2 so I cannot include
it in this post.
Here is the backtrace:
-------------------------------------
fbserver.exe!LCK_release(tdbb * tdbb=0x0336fcf0, lck *
lock=0x013930b0) Line 836 C++
fbserver.exe!CMP_shutdown_database(tdbb *
tdbb=0x0139233c) Line 2414 + 0x7 C++
fbserver.exe!CMP_fini(tdbb * tdbb=0x0336fcf0) Line
729 + 0x6 C++
fbserver.exe!shutdown_database(dbb * dbb=0x055a7494,
unsigned char release_pools='') Line 6031 C++
fbserver.exe!purge_attachment(tdbb * tdbb=0x0336fe44,
long * user_status=0x044ae5a0, att *
attachment=0x03f6765c, unsigned char force_flag=0)
Line 6579 + 0x8 C++
fbserver.exe!isc_detach_database(long *
user_status=0x00000009, why_hndl * * handle=0x0041b9c5)
Line 1678 + 0x18 C++
fbserver.exe!ThreadPriorityScheduler::InternalGet()
Line 86 C++
fbserver.exe!put_specific(thdd *
new_context=0x0051709c) Line 1813 + 0xa C++
fbserver.exe!THD_put_specific(thdd *
new_context=0x7c809590) Line 1259 + 0x8 C++
kernel32.dll!FindAtomW() + 0x94
kernel32.dll!CreateFileMappingA() + 0xb6
-------------------------------------

When running with the guardian, the firbird.log has:

MACHINE (Client) Thu Feb 23 23:29:55 2006
C:\Firebird\bin\fbserver.exe: terminated abnormally
(3221225477)

when running FB as an application, and:

MACHINE (Client) Thu Feb 23 12:46:25 2006
C:\Firebird\bin\fbserver.exe: terminated abnormally
(4294967295)

when running FB as a service, but a dump is not
produced and the JIT debugger does not activate.

Let me know if you need additional information.

thanks,

stan

Commits: 116cd97

[firebird-automations]

Commented by: Alice F. Bird (firebirds)

Date: 2006-03-13 19:44
Sender: skaushanskiy
Logged In: YES
user_id=1453243

Hi Dmitry,

thanks for the response.

It happens intermittently by often under heavy load:
1 crash in 5 minutes.

My metadata is stable there are no create's or drop's.

One more thing, I tried the same setup but with Firebird
running under Linux, no crashes. It only happens on windows.

I would love to try fixes.
should I email you the dump file?

thanks again,

-stan

[firebird-automations]

Commented by: Alice F. Bird (firebirds)

Date: 2006-03-10 06:16
Sender: dimitr
Logged In: YES
user_id=61270

Is your metadata stable or do you create/drop tables during
the load? How often does the crash happen, i.e. is it
reproducable on your side? Would you mind trying the
presumably fixed build when it's available?

[firebird-automations]

Commented by: skaushanskiy (skaushanskiy)

It happens often under heavy load
at least a couple of times an hour

I am very willing to try fixes

this issue affects FB 1.5.2, FB 1.5.3, FB 2.0 RC1, RC2, and RC3

this never happens under Linux

[firebird-automations]

Modified by: skaushanskiy (skaushanskiy)

Version: 2.0 RC3 [ 10040 ]

Version: 2.0 RC2 [ 10032 ]

Version: 2.0 RC1 [ 10031 ]

Version: 1.5.2 [ 10027 ]

Version: 1.5.3 [ 10028 ]

SF_ID: 1439774 =>

[firebird-automations]

Commented by: Manabu Suzuki (manabu)

Hi Skaushanskiy and Dimitr,

This problem can be reproduced on my test environment.
I'm running FB 1.5.3 SS on Dell PowerEdge 830 with 1GB memory.
My program is written in Java and run on JRockit-R26.4.0.
The program opens a connection, attachs an event handler for an event that is repeatedly raised when another program inserts a record into a certain table, and performs the following operations each time the event is received:

1. Begin transaction.
2. Execute SELECT statement with a PreparedStatement to a certain table.
3. Execute INSERT statement with a PreparedStatement to another table.
4. Commit transaction.

During the operation, the connection is never disconnected intentionally.
The error (4294967295) occurs before 1000 events are raised.

To avoid this problem, I periodically reconnect a connection.

I hope this information helps to make Firebird better.

[firebird-automations]

Commented by: skaushanskiy (skaushanskiy)

Monabu,

Yours is a different problem because I am not using any events, only inserts and updates
through stored procedures.

[firebird-automations]

Commented by: liuyan (lucas.liuyan)

Hi All,

I came across the same problem,even though in the 2.0 the problem was still exist.
Because of this problem,we can't publish our software.
Do you have the suggesion how to avoid this problem.

Regards
Lucas

[firebird-automations]

Commented by: skaushanskiy (skaushanskiy)

a couple of things you can do will make this a lot less likely to happen:

1. do not update the same record more than once in the same transaction.
2. do not insert and update the same record in the same transaction
3. use read-only transactions for selects

Also, the guardian will restart firebird when it crashes, and firebird should pick
up any incoming work. So if you implement a "retry" mechanism in your application,
this problem can be "hidden".

[firebird-automations]

Commented by: @hvlad

Does someone can reproduce it with FB 2.0.1 ?
If yes i'll be glad to get reproducible test case

[firebird-automations]

Modified by: @pcisar

Workflow: jira [ 10562 ] => Firebird [ 14885 ]

[firebird-automations]

Commented by: @dyemanov

Two causes of this issue has been independently resolved by Nickolay in v2.0 and Alex in v2.1.
I'm backporting the solution into v1.5.6 and resolving this ticket.

[firebird-automations]

Modified by: @dyemanov

status: Open [ 1 ] => Resolved [ 5 ]

resolution: Fixed [ 1 ]

Fix Version: 1.5.6 [ 10225 ]

Fix Version: 2.1.0 [ 10041 ]

Fix Version: 2.0.0 [ 10091 ]

[firebird-automations]

Modified by: @pcisar

status: Resolved [ 5 ] => Closed [ 6 ]

[firebird-automations]

Modified by: @pavel-zotov

QA Status: No test

[firebird-automations]

Modified by: @pavel-zotov

status: Closed [ 6 ] => Closed [ 6 ]

QA Status: No test => Cannot be tested