Problem with possibly corrupted (?) index

[cpyrgas]

This is the last problem I am fighting with, in Joe's app. The attached (empty) dbf file is one that is used in the app and the accompanying (also included) .cdx file is being generated (for the empty dbf) on the fly on app startup. The problem is that the first operation made on this particular file/index fails (returns false actually), see below:

dbf.zip

FUNCTION Start() AS VOID
? DbUseArea(,"DBFCDX","c:\test\test.dbf",,TRUE) // true
? DbAppend() // false
? DbCloseArea() // false

there's also a handled exception, see below. This problem happens only the first time the test runs. In the second run, all works well (same happens in the real app). Also after packing or zapping or simply reindexing the empty dbf, then there is not problem at all. So it looks like there's some problem/incompatibility only with the initial generation of this particular index file.

Problem is that when I try to generate the index in a test app, then it always works correctly, it's only when it is being created from the original large app that it later fails. I guess there's some setting from the app that I am missing, but haven't figured out what that is yet.

Anyway, Robert, can you tell by the exception message or by debugging with the index file provided what the problem is with it? Is it worth me trying to further track this down and try to create a repro sample also for producing this index?

This is the handled exception when calling DBAppend(), which fails:

HANDLED exception:

XSharp.RDD.RddError


Callstack : 
System.Void XSharp.RDD.DBF._dbfError(System.Exception ex, System.UInt32 iSubCode, System.UInt32 iGenCode, System.String strFunction, System.String strMessage, System.UInt32 iSeverity, System.Boolean lThrow)() 
XSharp.RDD.CDX.CdxLeafPage.Boolean XSharp.RDD.CDX.CdxLeafPage.Write()() 
CdxAction XSharp.RDD.CDX.CdxLeafPage.Insert(System.Int32 nPos, System.Int32 recno, System.Byte[] key)() 
CdxAction XSharp.RDD.CDX.CdxTag.InsertKey(XSharp.RDD.CDX.CdxAction action)() 
XSharp.RDD.CDX.CdxTag.CdxAction XSharp.RDD.CDX.CdxTag.DoAction(XSharp.RDD.CDX.CdxAction action)() 
XSharp.RDD.CDX.CdxTag.Boolean XSharp.RDD.CDX.CdxTag._keyUpdate(System.Int32 recordNo, System.Boolean lNewRecord)() 
Boolean XSharp.RDD.CDX.CdxTag.GoCold()() 
Boolean XSharp.RDD.CDX.CdxOrderBag.GoCold()() 
Boolean XSharp.RDD.CDX.CdxOrderBagList.GoCold()() 
XSharp.RDD.DBFCDX.Boolean XSharp.RDD.DBFCDX.GoCold()() 
XSharp.RDD.DBFCDX.Boolean XSharp.RDD.DBF.Append(System.Boolean lReleaseLock)() 
Boolean XSharp.CoreDb+<>c__DisplayClass18_0.<Append>b__0()() 
static Object XSharp.CoreDb.Do(System.Func`1[System.Object] action)() 
static Boolean XSharp.CoreDb.Append(System.Boolean lReleaseLocks)() 
static Boolean XSharp.RT.Functions.DbAppend(XSharp.__Usual[] Xs$Args)() 
static System.Void xRuntime.Exe.Functions.Start()()   :  C:\VIDE\Projects\Project3\Applications\xRuntime\Prg\xRntime.prg  :  7
``

[RobertvanderHulst]

I'll have a look at this tomorrow.
This should not be too difficult to find I guess.
I suspect that this is an error in code that I added in 2.8 to validate the integrity of CDX files. Maybe this code fails when there is only one key in the page ?
Robert

[cpyrgas]

No hurry, was just stuck with it and gave up trying to further narrow down, so reported what I had so far.
I don't think it's about one key, because if you then zap the dbf and try again the error does not happen again.
Also if before you run it for the first time, you reindex the (empty) dbf, again the problem does not happen.

It only happens the very first time you use this dbf/index without touching it all before. Then the problem disappears forever.

[RobertvanderHulst]

Chris,
What I found so far is that the DBF file is empty, but the CDX file contains an index entry for record 1.
So these two files do not match.
The Append fails, because after appending the first record in the DBF I am running some validation code (that I introduced in X# 2.8) that compares the keys and values in the index page. This code detects 2 keys in the index page with both the same key value and both the same record number (1). This is not valid. So it aborts the write.

If I open the same DBF and CDX in VO (make sure that the app also includes the Deleted() function, because there is a for condition on! DELETED() ) then VO will update the index and the page in the index will have 2 entries for record 1. So VO ignores the problem in the index.

So the question now is: where is this code created ? How is it possible that the CDX has 1 record and the DBF has no records ?

[cpyrgas]

Robert,
This is generated at program startup, there is a routine that checks for missing indexes and creates them if they do not exist. Very interesting this about a record in the dbf, maybe there's another file used as a dbf source for some reason. Will look further into this and get back to you, thanks for the info.

[wriedmann]

Hi Robert,
I can confirm that VO 2.8 has some erratic behavior when creating an index with an empty DBF - sometimes it returns an error in the index creation. I have never been able to understand the circumstances, but I have to admit that I have not invested any time in it.

[cpyrgas]

Wolfgang, this is about an X# app, seems there's a small problem in index creation under circumstances.

Robert, I just figured it out, has to do with an order condition, with both a FOR and an EvalBlock supplied, apparently this creates a bad index. In the real app, those are both doing useful things, but problem can be reproduced also with dummy ones, see below. Need to go out now, but later I will upload a new sample that does not depend on the DBServer class:

FUNCTION Start() AS VOID
LOCAL oServer AS DbServer
LOCAL cDbf AS STRING

cDbf := "c:\test\newtest"
RddSetDefault("DBFCDX")

FErase(cDbf + ".cdx")
DbCreate(cDbf , {{"FLD","C",10,0}})

oServer := DbServer{cDbf}
oServer:SetOrderCondition( ".T.",,,,{||1+2})
? oServer:CreateOrder("TEST" , cDbf, "Upper(FLD)")
oServer:Close()

oServer := DbServer{cDbf,TRUE,FALSE,"DBFCDX"}
? oServer:Append() // returns FALSE
oServer:Close()

[cpyrgas]

Here's a sample without DBServer. If the file is opened in shared mode, then both DBAppend() and DBCloseArea() return FALSE. If opened exclusive, then only DBCLoseArea() returns false.

VO returns TRUE in all cases and the index is correctly generated.

FUNCTION Start() AS VOID
LOCAL cDbf AS STRING

cDbf := "c:\test\newtest"
RddSetDefault("DBFCDX")

FErase(cDbf + ".cdx")
DbCreate(cDbf , {{"FLD","C",10,0}})

DbUseArea(TRUE,,cDbf)
OrdCondSet( ".T.",,,,{||1+2})
? DbCreateOrder("TEST" , cDbf, "Upper(FLD)")
DbCloseArea()

? DbUseArea(TRUE,,cDbf,,TRUE) // if opened exclusive, DBAppend() returns TRUE, but DBCloseArea() returns again FALSE
? DbAppend()  // false
? DbCloseArea() // false

[cpyrgas]

Confirmed fixed.