Suggestion: Add TObject.Classname to leakage report

[hpw4711]

Hi,

as using D2007 with multiple packages & package plugin-ins and did run in an object free leakage issue as:

A memory block has been leaked. The size is: 1668

This block was allocated by thread 0x1174, and the stack trace (return addresses) at the time was:
20003DAE [sys\system.pas][System][@getmem][2654]
2000673B [sys\system.pas][System][TObject.NewInstance][8807]
20006B3E [sys\system.pas][System][@ClassCreate][9472]
63636502
20003DAE [sys\system.pas][System][@getmem][2654]
20007AA9 [sys\system.pas][System][@NewAnsiString][12429]
20007AEC [sys\system.pas][System][@LStrFromPCharLen][12476]
63661004
6368103E
636BA2A1
636BFF2B

I would like is, that the @CreateClass shows also the ClassName .. as from TObject.Classname

Did also compiled the new FastMM_FullDebugMode.dll using JCL and used it without success ...

otherwise fishing into the dark leaking :D

Hp

[pleriche]

Hi,

Have you tested the application with FastMM5? Does it report the class of the object? If not, could you perhaps send me a small test case that reproduces the issue?

Thanks.

[hpw4711]

Hi Pierre,

1. The app is a Delphi 2007 and almost not compatible to Unicode as a 3-6 month work. :D
2. Even to create small app would be a Hugh work.

A simple way would be, to intercept TObject (on BPL) and then adds the ClassName or to create an own THpWObject with related procedures what notes the create & free and counts the beans..

May al this happens already on the detector DLL as [sys\system.pas][System][@ClassCreate][9472] after creation to use the ClassName :D

[pleriche]

I suspect what is happening here is that you're allocating an object in a library using a shared instance of FastMM. When the leak report is generated the library is already unloaded so FastMM can no longer determine the class of the object.

Unfortunately I do not see an easy way to determine and store the class name when the object is created, since when the initial allocation is done using GetMem that information is not readily available. The class pointer will likely be on the stack somewhere, and its exact location may vary by Delphi version.

[hpw4711]

When the leak report is generated the library is already unloaded so FastMM can no longer determine the class of the object.

YES, as the issue happens in a form (BPL) what gets closed (freed) earlier as step by step app shutdown

[hpw4711]

or one way would be: Store the ClassName and 'may' the Unit Name on object creation. So all data would be available on leakage detection time.

As on enhanced mode :D

[hpw4711]

Unfortunately I do not see an easy way to determine and store the class name when the object is created, since when the initial allocation is done using GetMem that information is not readily available. The class pointer will likely be on the stack somewhere, and its exact location may vary by Delphi version.

The best way would be to patch the TObject or TInterfaceObject as AfterConstruction & BeforeDestruction...

procedure TInterfacedObject.AfterConstruction;
begin
// Release the constructor's implicit refcount
InterlockedDecrement(FRefCount);
end;

procedure TInterfacedObject.BeforeDestruction;
begin
if RefCount <> 0 then
Error(reInvalidPtr);
end;

Ok, did some test using this:

TLibObject = class (TObject)
procedure AfterConstruction; override;
procedure BeforeDestruction; override;
End;

TSys_Reg_Base = class (TLibObject)
// TSys_Reg_Base = class (TObject)

Begin
OutputDebugString
(PChar
('TLibObject: Create ClassName: ' +
Self.ClassName +
', Length: ' +
IntToStr (Self.InstanceSize) +
', ' +
Lib_Str_AddrToString (Self)));
End;

procedure TLibObject.BeforeDestruction;

Begin
OutputDebugString
(PChar
('TLibObject: Destroy ClassName: ' +
Self.ClassName +
' ' +
Lib_Str_AddrToString (Self)));
End;

Results of modified Class:

TLibObject: Create ClassName: TSys_Reg_Base, Length: 68, $FEF0D8C0
TLibObject: Destroy ClassName: TSys_Reg_Base $FEF0D8C0

so to have all used TObject with my TLibObject as a simple way and afterwords sort and count the create vs destroy

Hp

[the-Arioch]

YES, as the issue happens in a form (BPL)

does the BPL itself get unloaded then?

because what it seems is that the class itself was unloaded, so any attempt to run ClassName function would result in unpredictable crash inside FastMM4

you can sort of check it yourself

ShowMessage(IntToHex(NativeInt(pointer(MyObj.ClassName)), 2 * SizeOf(Pointer)))

Then you can compare the addres with the loaded DLLs addresses (Delphi IDE / Debug Windows / Modules/ Sort y base address, or SysInternals Process Explorer, or Bagel's Program Memory Map, or any other process memory walker). That pointer might lead to nowhere...

Notice: Delphi 2007 has wrong declaration for NativeInt, albeit this would not make this code behave wrong

Recently i had to override ALL objects constructors, in Delphi 2007, because of our legacy app had very broken VBScript interface and internal design, leading occasionally of stale pointers and double-frees. The idea of the design was automatically creating iDispatch wrappers for ANY Delphi objects, but they did never care that the object can be freed fopr zillion reasons, and the memory the interfaced wrapper references would be occupied by some other object or maybe not an object at all. So in the end i had to make every TObject mantain the list of its wrappers ever created, if any, with bi-directional 'notifications' about being disposed.

Trying to get O.ClassType was producing pointers to VERY random places in memory, when that happenned. Trying to calculate O.ClassName would've effected in unpredictable crashes like triggering guard pages. Raymond Chen blogged extensively against probing unknown pointers even using native Win32 API.

I would certainly avoid FORCING memory manager from getting any class names, if it could not get them usual way. Frankly, i feel frightened at the very idea it ever tries getting classnames, for ANY stale objects at all. In my experience D2007 is very fragile (it even does not support debugging when running in Win64!) and i had many cases IDE freezing just trying to calculate ClassName of a perfectly valid object. But maybe FastMM4 implements its own custom, ultra-careful ways of RTTI parsing, dunno.

or one way would be: Store the ClassName and 'may' the Unit Name on object creation

Since Delphi 2007 keeps classname as ShortString (check the declaration of TObject.ClassName) in const-memory, this would mean creating yet another LongString instance and then copying the text into it. I would expect this functionality to be very bad for both speed and memory fragmentation.

In my specific case every object gets extra 8 bytes allocated - and even this visibly effects HeapMM behavior (it tries to have several pools with small-size slots, and the "minimal" pool is almost never used now, while before the patch very many of simple TObject descendants fit there. Maybe i should rebuild the DLL with some constants tweaked...). However those extra bytes are only initialized to plain int32 values when allocated. Only very few objects (about 4K objects per 2-5 minutes of work) that DO get the COM-wrappers carry the extra load (creating TObjectList and linking it to the target object). The overhead does not look noticeable. But turning EVERY single object creation into allocati nand copying of LongString... I shiver at the very thought of it.

All you really can do going this way would be having some global map pointer-to-string (like a sorted TClassList together with mirroring TStringList (itself one slow ugly beast, but there is no TList<string> in D2007, not TDictionary<TClass, string>)) and then EVERY created object should before creating start checking whether its class was registered and if not - then add it to the registry.

Notice a little hen and egg problem here, the registry objects before they are created should register their own classes in the registry objects that they themselves are and that were not created yet. Do you feel power to solve the riddle?

---

If you ever would try folowing this rabbit hole - it would be rather entertaining in its own kind, even if i believe is practically useless - forget about GetMem, pleriche is very correct here: when it is finally gets called it is too late already. You MIGHT try reading the stack, determining the caller, then trying read stale data from dirty memory or dirty CPU registers, etc. But this would be truly fragile, depending upon both RTL version, compiler version and compiler flags. You would have to make VERY ABI-specific fragile memory walking there. If you dive that low level you would be much better suited cracking into TObject itself creating methods instead of GetMem, at least in D2007.

Another aspect, the TObject.Create does NOT have to ever call GetMem, and even if it chooses to do, it can position the object data ANYWHERE within the allocated memory. Granted, usually nobody bothers to change the defaults, but it certainly IS allowed. But if FastMM4 would dare hijacking objects creation it would be required to recognize and adapt to those customizations automagically. This is NOT possible.

Your project in the end can afford this hackery, because it would only ever be compiled with very narrow set of ABI-defining factors (compiler version, RTL version, compiler options), most probably with only one single set. Since you did not already upgrade from Dephi 2007 to FPC or Delphi XE - i expect you would never do it. Hence YOU can implement a small set of these one-purpose hacks locally. But no extrnal library with broad ABI targets can do so.

[the-Arioch]

The class pointer will likely be on the stack somewhere, and its exact location may vary by Delphi version.

I would definitely not count on that. TObject.Create sequence uses very tight methods, and i would expect even very few registers 386-arch has would be enough to almost always have Self var in some register. AMD64-arch (lacking in D2007) having 3x amount of register variables should be 100% stack-free w.r.t. Self.

AFAIR there is just not enough registry pressure in stock RTL to ever offload Self-obj or Self-class there. Even if it occasionally would, i'd expect it to depend upon compiler options, thus to be extremely fragile, beyond usable.

It could've been different with 16-bit x86 code though due to much more limited mod-r/m :-)

[hpw4711]

As in the old days requested and did skipped from the Delphi Managers, as the developer should build there own RTL & VCL as in Delphi 1, May this is in fpc possible out the box.

For now own I use my TLibObject = class (TObject) and can do much more and control in a different level.

BTW: the Class name as 256 bytes should be added in the leakage count :D

Hp