PS_NATIVESTRINGS: native string types, real Ansi script types and width-correct marshaling (fixes #296)#318
Draft
TetzkatLipHoka wants to merge 40 commits into
Draft
Conversation
With PS_USESSUPPORT, ProcessUses sets fModule to the name of the unit
being imported before invoking the OnUses callback. The regular failure
path (OnUses returning False) restores fModule and FParser afterwards,
but the exception handler did not. The stale fModule survives Cleanup:
Compile saves it as OldFileName on entry and restores it after the
System import, so every following Compile on the same instance treats
the main module as the previously failed unit and rejects any
'uses <that unit>;' with a bogus cross-reference error.
Reproduction (one compiler instance):
1. Compile a script whose OnUses handler raises (e.g. a failing
AddTypeS registration) -> correct error is reported.
2. Compile any script with 'uses <SameUnit>;' again
-> before: 'Cross-Reference error of <unit>'
-> after: compiles normally (or reports the real error again).
The exception handler now restores FParser and fModule exactly like the
regular failure path. MakeError stays first so the error keeps being
attributed to the failed unit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Imported (host) functions called through the Rtti.Invoke path (the default since XE2; PS_USECLASSICINVOKE was not affected) mishandled dynamic arrays in two ways: 1. By-value dynamic array parameters passed the address of the script's variable slot instead of the array pointer stored in it. The callee then interpreted the slot address as an array reference and crashed with an access violation (reproducible on Win32 and Win64). 2. Dynamic array results were only retrieved when a host RTTI type could be located whose name ends with the script type's exported name. Script types are usually not name-exported, so the lookup found nothing - and because the Invoke call sat inside the search loop, the imported function was then never called at all: the script silently received an empty array (and none of the function's side effects). The empty-result path additionally corrupted the result IFC record by setting res.dta to nil. The parameter case now passes the dereferenced array pointer. The result case selects a type info matching the managed kind of the element type (Invoke only needs it for the - universally identical - dynamic array result ABI and for releasing the TValue's reference; the element-wise transfer into the script slot goes through the script's own type record). The host-type name lookup remains as the path for nested container elements, and a failed lookup now fails the call instead of faking an empty result. Empty results clear the script slot properly via CopyArrayContents. Related: remobjects#198 (the FPC report covers the classic x86 path, but the same symptom class exists here on the default path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One line of the big compatibility expression was missing parentheses:
(((p1.basetype = btPchar) or (p1.BaseType = btString)) and
(p2.BaseType = btWideString) or (p2.BaseType = btUnicodeString)) or
'and' binds stronger than 'or', so the (p2 = btUnicodeString) term stood
alone: ANY destination type was considered assignment-compatible with a
UnicodeString source. Since script 'string' is btUnicodeString nowadays,
these all compiled without error and only failed at runtime (or
corrupted data):
var i: Integer; s: string; ... i := s;
var b: Boolean; s: string; ... b := s;
var d: Double; s: string; ... d := s;
With the added parentheses the term applies - as clearly intended - only
to btPchar/btString destinations. String-to-string, WideString and Char
conversions keep compiling as before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The cast import (SpecImport's CastProc) reported failed interface and
'as' casts via CMD_Err2 and then returned False. CMD_Err2 already
dispatches the error to the script's active exception handler and
resets the pending-error state - so when the external proc afterwards
returned False, the interpreter raised a second erCouldNotCallProc for
which the handler was already consumed. Net effect: a failing cast
could never be caught by try/except in a script, and the descriptive
'Cannot cast an interface/object' message was replaced by a generic
'Could not call proc'.
Return True after CMD_Err2 instead, exactly like the DefProc builtins
handle their reported errors (e.g. the string index checks). A failing
cast now surfaces as a regular catchable runtime error with its proper
message:
try
f := IMyFoo(u); // u does not implement IMyFoo
except
// reached now; previously the script died with 'Could not call proc'
end;
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Variant parameters holding an IDispatch reference were marshaled like every other variant: wrapped as VT_VARIANT or VT_BYREF pointing at a copy of the variant. Lenient automation servers (e.g. the Scripting.* classes) accept that form, but strict marshalers - most prominently .NET COM interop - reject it, and assigning an object to an IDispatch property only works with a plain dispatch argument. Such arguments are now passed as VT_DISPATCH with the interface pointer itself (deliberately WITHOUT VT_BYREF, which is known to crash several COM servers). The variant in Par keeps the reference alive for the duration of the Invoke call, so no AddRef is needed. Also replaces the repeated rgvarg[i] indexing with a PVariantArg local (FPC needs rgvarg^[i]) and the cleanup check with a VT_BYREF mask test so VT_DISPATCH entries are skipped correctly. Originally found and fixed in the TetzkatLipHoka fork during .NET interop testing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…riables Assigning a Variant that holds an interface reference to a script variable of an interface type only worked when the destination was exactly IDispatch; every other interface destination failed with a plain 'Type Mismatch'. Assignments between differently declared interface types blindly copied the reference without ever consulting QueryInterface. SetVariantValue's btInterface branch now: - accepts Variants holding varUnknown/varDispatch values (including by-ref variants) for ANY interface destination, casting via QueryInterface on the destination's GUID; empty/nil Variants assign nil. Destinations declared as IUnknown (or without a GUID) receive the reference as-is, preserving existing behaviour. - performs interface-to-interface assignments of differently declared types via QueryInterface as well; same-GUID (and nil) assignments stay plain reference copies. Failed casts raise a descriptive, script-catchable error that names the destination type (or its GUID when the type is unnamed) instead of the generic type mismatch. Pre-Delphi3 builds keep their previous paths. Note: the companion fix 'Make failing interface/object casts catchable' covers the explicit cast-import path (CastProc); together they make all interface conversion failures catchable with descriptive messages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Several failure paths inside LoadTypes/LoadProcs/LoadVars only return False without calling CMD_Err. LoadData then cleared the instance and returned False with ExceptionCode still erNoError, so hosts printing TIFErrorToString(ExceptionCode, ExceptionString) showed 'No Error' for a failed load. Each stage now reports a stage-specific erCustomError - but only when no more specific error is pending (ExEx = erNoError), so existing detailed errors like 'Out Of Range' or 'Unexpected End Of File' are preserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TPSCompileTimeClass.RegisterProperty freed the half-built property and returned silently when its type string referenced a type that was not (yet) registered. Scripts using the property then failed with a bare 'Unknown identifier' - very hard to trace back to a wrong registration order in an import unit. This is the actual root cause behind reports like issue remobjects#255 ('Delphi 11 case sensitive'). Raise EPSCompilerException naming the property and its type string instead, matching how AddTypeS and the other registration APIs report bad input. Inside OnUses the exception surfaces as a regular compiler error message. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Recent changes made the core units unbuildable on old Delphi versions (reported for Delphi 7 in issue remobjects#286). Three categories of fixes, all behavior-neutral on current compilers: - Exit(Value) needs Delphi 2009+: expanded to 'Result := ...; Exit;' in the for-loop code generation (EmitForEntryCheck/EmitForExitCheck). - High(UInt64)/High(tbtU64) is not evaluable as a constant expression on Delphi 7 (its UInt64 is a crippled signed alias): replaced with 'not UInt64(0)', which yields the same all-ones value everywhere. - 'tbtDouble(p^.textended)' is an invalid typecast on Delphi 7 (Extended -> Double must be a conversion): assign without the cast, which is the intended float conversion on every compiler. - SysUtils.StrToUInt64/StrToUInt64Def (XE4+) and SysUtils.UIntToStr (2009+) do not exist on old RTLs: uPSUtils now provides fallbacks, compiled only when SysUtils lacks them (checked via {$IF NOT DECLARED(...)}, so this adapts to any compiler and to FPC). The two qualified SysUtils.UIntToStr calls in uPSRuntime were unqualified so they resolve to whichever implementation exists. Note: pre-2010 compilers format UInt64 values above High(Int64) with a sign - an inherent limitation of their signed UInt64 alias. With this, all uPS* units (including the import units) compile on Delphi 7 again; Delphi 12 Win32/Win64 builds are unchanged. Fixes remobjects#286 (the current sources can be used on Delphi 7 again). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Set/array literals only accepted single elements; the common Pascal
notation with ranges failed to parse ('Closing square bracket
expected'):
if c in ['0'..'9'] then ...
if b in [1..5, 250..255] then ...
if f in [frBanana..frDate] then ...
ReadArray now expands constant '..' ranges into individual constant
elements while parsing. Rules match Delphi where applicable:
- both bounds must be compile-time ordinal constants of the same family
(char/char, integer/integer, or values of the identical enum type),
- ranges can be mixed freely with single elements,
- an inverted range ('z'..'a') contributes no elements,
- a range spanning 256 or more values is rejected (a set cannot hold
more), reported as a type mismatch.
Dynamic bounds stay unsupported (they would require runtime set
construction); the bounds must be constants, mirroring the previous
all-constant behaviour of set literals.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
raise: scripts could report custom errors only through the
RaiseException builtin and could not re-raise inside except blocks with
the usual Pascal syntax. The identifier RAISE is now handled as a
statement in ProcessSub:
raise; // re-raise: RaiseLastException
raise Exception.Create('message'); // RaiseException(erCustomError, msg)
The class name is parsed but not evaluated (scripts have no exception
objects), the message may be any string expression, and the strict
<ident>.Create(<expr>) grammar keeps the single-token-lookahead parser
deterministic. Both forms work as if/then one-liners and interact with
try/except/finally as expected; ExceptionType/ExceptionParam report
erCustomError and the message.
ParamCount/ParamStr: the host process' command line was not reachable
from scripts. Registered as DefProc indices 49/50 (45-48 are taken by
the StrToUInt64 family and UIntToStr), kept in sync in
DefineStandardProcedures, DefProc and RegisterStandardProcs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ordWithRTTI Importing host types previously meant hand-writing AddTypeS declarations that had to be kept in sync with the Delphi type. The compiler can now derive them from RTTI: Sender.AddType(TypeInfo(TFruits)); // set of TFruit Sender.AddType(TypeInfo(TIntArr)); // array of Integer Sender.AddType(TypeInfo(TMyEvent)); // procedure(...) of object Sender.AddType(TypeInfo(TFull)); // packed record (Delphi 2010+) AddType(PTypeInfo) covers the classic RTTI kinds - integers (all ord sizes), chars, all string kinds, floats, enums, Variant, Int64/UInt64 (UInt64 detected via its inverted MinInt64Value/MaxInt64Value pattern), sets, dynamic arrays and method pointers (signature reconstructed from the ParamList RTTI). Element types of sets/arrays are registered recursively; sets/dynarrays/methods are excluded under FPC where the TTypeData layouts differ. On Delphi 2010+ extended RTTI adds tkRecord via AddRecordWithRTTI and named static array types. Records must be packed: field offsets are verified one by one (including tail padding) and a padded layout is rejected with a clear error, because script records are packed and a mismatched layout would corrupt data on host interop. Fields with anonymous types (inline 'array[0..7] of X') have no RTTI and are rejected with a message naming the field; generic names such as TArray<System.Byte> are sanitized to valid script identifiers. TypInfo moves to the interface uses for PTypeInfo in the public signatures; the Rtti unit stays implementation-only (the protected helpers take untyped pointers). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t, PSBaseTypeToStr
Hosts that embed a script debugger (watch windows, tooltips, REPL-style
inspection) all end up writing the same code: turn an arbitrary script
value into readable text and parse an edited value back. This adds the
helpers used by the TetzkatLipHoka fork's debugger frame:
- PSVariantToString(IFC, AppendType, NoQuotes): renders every base type
including sets (element list), records (field list), static/dynamic
arrays (element list, capped with '..' after 50 chars), variants
(with their VarType), nested pointers and classes (via TStrings.Text
or a published Caption/Title/Text/Lines/Strings property).
AppendType prefixes the Delphi-style type name; NoQuotes returns
string values raw instead of quoted.
- StringToPSVariant: the reverse direction for editable watches -
parses text into ordinal/float/string/char/variant slots and into
class instances via the same published properties. btPChar slots are
deliberately not written: they hold raw pointers to memory the engine
does not own.
- PSBaseTypeToStr(TPSBaseType/PIFTypeRec): base type names, optionally
as Delphi names ('Integer' instead of 'S32'); the PIFTypeRec overload
renders composed types (record field lists, array element types,
class names).
- VarTypeToString: readable names for TVarType values.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # Source/uPSRuntime.pas
First step of the opt-in PS_NATIVESTRINGS mode (default: OFF, nothing changes without the define). With PS_NATIVESTRINGS, tbtChar/tbtPChar/tbtString follow the host compiler's native types (Unicode on Delphi 2009+). The practical win: host functions registered with tbtString/tbtChar parameters receive exactly the width the engine passes - the silent width mismatch that breaks legacy registrations against today's unicode script strings (see issue remobjects#296) disappears, because tbtString is the correct signature type again on every compiler version. This commit adds the type switch, the always-Ansi companion types (tbtAnsiChar/tbtAnsiString/tbtPAnsiChar) and four new base type ids: btPWideChar = 30 (PChar maps here under PS_NATIVESTRINGS) btAnsiChar = 31 \ real Ansi script types, needed because btAnsiString = 32 ) btChar/btString/btPChar follow the native btPAnsiChar = 33 / (wide) width under PS_NATIVESTRINGS Compiler and runtime support for the new base types follows in the next commits; until then the define is declared but not yet functional. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…array-of-const Three spots assumed tbtchar is always a single byte: - PSSetAnsiString widened an AnsiChar into a possibly-wide char slot without a conversion (E2010 with PS_NATIVESTRINGS), - the array-of-const builder stored wide chars through TVarRec.VChar (vtChar), truncating them; wide builds now pass vtWideChar/VWideChar, - the var-param readback mirrored the same assumption. All three compile to the previous code when PS_NATIVESTRINGS is off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Migrate all compiled-data containers from tbtString to tbtAnsiString so the bytecode stream stays a raw byte buffer when tbtString becomes a wide string under PS_NATIVESTRINGS: - TPSPascalCompiler.FOutput / GetOutput, TPSInternalProcedure.FData / Data - TPSExec.LoadData, TPSDebugExec.LoadData override - TPSScript.GetCompiled / SetCompiled / LoadExec (separate wide local for debug data, which stays tbtString) - IFPS3DataToText input - opcode appends use tbtAnsiChar casts (WriteByte, BlockWriteByte, FindMainProc master proc, empty-proc CM_R, CM_PO append); building these in a wide string and converting afterwards would turn bytes >127 into '?' - new PS_mi2d: PS_mi2s variant returning a byte string for binary buffers No behavior change when PS_NATIVESTRINGS is undefined (tbtAnsiString = tbtString = AnsiString there). Both modes compile warning-identical on Delphi 12 (DCC32). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Names, export declarations and string constants embedded in the bytecode are
tbtString values; their stream payload is Length(s) characters. Multiply all
raw reads/writes of such payloads by SizeOf(tbtChar) so they stay correct
when tbtChar is 2 bytes:
- compiler writers (MakeOutput/BlockWriteVariant): btChar constants,
btString constant payloads, attribute type names, class names, proc-ptr
decl bits, type/var export names, internal proc name + export decl,
external proc name + import decl
- runtime readers: attribute type name and btString/btPchar attribute
values, class FCN, proc-ptr FParamInfo, type/proc/var export names and
decls, ReadVariable inline string constants (terminator via tbtpchar)
- btChar constants take SizeOf(tbtChar) bytes; readers dispatch btchar into
the 1-byte or 2-byte group via {$IF SizeOf(tbtChar)}
- uPSDisassembly: string/char constant readers width-aware (s, c are now
tbt types)
btSet payloads keep byte semantics (Set_* operators address the string
container as PByteArray). No change with PS_NATIVESTRINGS undefined; both
modes compile on Delphi 12 and the OFF-mode probe suite passes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TPSPascalParser assumed 1-byte characters: FText was hard-typed PAnsiChar, token extraction moved Length bytes, and the keyword uppercaser walked the token as a byte string. With a wide tbtString the parser returned garbage tokens (every script failed with 'Unable to register type Boolean' from the very first AddTypeS). - FText: TbtPChar; all token/raw-text extractions move Length*SizeOf(tbtChar) - CheckReserved takes TbtString (keyword names no longer truncated through ShortString) - identifier uppercasing uses UpperCase for 2-byte tbtChar, the byte walk otherwise - char-class tests use CharInSet when available (silences WideChar-in-set warnings; same code on old compilers), incl. FastUpperCase/FastLowerCase No change with PS_NATIVESTRINGS undefined (SizeOf(tbtChar)=1 keeps all arithmetic identical). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With native (wide) tbtString, the script types AnsiChar/AnsiString/PAnsiChar would silently lose their Ansi semantics. Give them their own always-Ansi base types (btAnsiChar=31, btAnsiString=32, btPAnsiChar=33) and map script PChar to the new btPWideChar=30. Only active under PS_NATIVESTRINGS+UNICODE; otherwise DefineStandardTypes registers exactly the same types as before and the new base types are never emitted. Compiler: type registration incl. PWideChar and open arrays, IsCharType/ IsStringType, IsCompatibleType rules (PWideChar mirrors UnicodeString, Ansi family assignable across all string types), GetResultType (all-Ansi concat stays btAnsiString; comparisons; otIn accepts any char type against char sets and int against byte sets), const getters/setters, variant copy/finalize, ConvertToAnsiString + PreCalc folding (concat, all six comparisons, otIn), BlockWriteVariant/WriteVariant (btAnsiChar = 1 byte, btAnsiString = length x 1), set-of-AnsiChar, string indexing, Ord(), const cast folding. Runtime: LoadTypes/CalcSize/InitializeVariant/FinalizeVariant/ NeedFinalization for the four new types, attribute + inline-constant readers, PSGet*/PSSet*, CopyArrayContents (missing types here silently zero record fields!), SetVariantValue sources and destinations, comparison/concat operators, string builtins (Length/SetLength/StrGet/ StrSet/Copy/Delete/Insert/Low/High), IntPIFVariantToVariant, debug helpers, TVarRec builder, RTTI property stores. P-char ownership: btPAnsiChar/btPWideChar slots own their content as a managed string (the payload pointer IS the p-char value); they are finalized like strings and all writers assign managed strings. PSBorrowPCharOwnership/PSReownPCharOwnership are included for the call-marshaling layer. btPChar (14) deliberately stays the upstream raw pointer type - unchanged semantics for existing host code; under PS_NATIVESTRINGS scripts cannot create btPChar values (PChar maps to btPWideChar). Both modes compile on Delphi 12; OFF-mode probe suite unchanged-green; ON-mode script smoke tests (string/char/AnsiString/AnsiChar consts, concat, StrSet indexing, Ord) pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Marshal the new base types across all Delphi invoke paths: - InvokeCall.inc (Rtti, XE2+): TValue construction for by-value btAnsiChar/btAnsiString/btPAnsiChar/btPWideChar (nil p-char slots pass an empty terminated buffer), open-array element cases, managed result stores (a p-char result is copied into the slot as its managed string type), and var-param ownership handling: PSBorrowPCharOwnership before the call, PSReownPCharOwnership in a try..finally after it, so external code may overwrite the pointer or write through it without corrupting the slot's managed string. - x86.inc / x64.inc (classic paths): same by-value/var/result coverage, borrow/reown around the call, btAnsiString in the string-pointer and hidden-result groups. The x86 GetPtr 'tempstr' buffer is now explicitly TbtAnsiString - it is a raw byte buffer concatenated into the byte-string call stack and must never become wide (identical in OFF mode). btChar by-value reads SizeOf(tbtChar) bytes. EmptyPchar widened to two #0 chars since it also serves as the PWideChar nil replacement. - btPChar keeps its upstream raw-pointer marshaling everywhere. All 8 combos compile (DCC32/DCC64 x default/classic x on/off). Native-mode functional tests pass on all four ON combos: TestWideUp (29 checks: PWideChar/PAnsiChar/AnsiString params + results, stack spills), TestPChar (25 checks: ownership, aliasing, external reassignment, leak delta), TestRec (host record interop with AnsiChar fields and set of AnsiChar). OFF-mode probe suite unchanged-green on 32/64 bit, default and classic paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CreateOpenArray tagged btString elements as vtAnsiString unconditionally.
With a wide tbtString the callee then reinterpreted UTF-16 payloads as Ansi
data, so Format('%s', [...]) received empty/garbled strings (visible with
temporaries, e.g. [Trim(s)], on the classic invoke paths; the Rtti path has
its own builder). btString now maps to vtUnicodeString when tbtChar is
2 bytes, incl. the matching var-param readback and finalization.
Also add the missing element cases for the always-Ansi types: btAnsiString
(vtAnsiString, managed), btAnsiChar (vtChar), btPAnsiChar (vtPChar) -
previously they fell through the case and arrived as zeroed vtInteger.
No change with PS_NATIVESTRINGS undefined.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetProcAddress takes an ANSI name: convert explicitly via PAnsiChar(TbtAnsiString(s3)) instead of casting the tbtstring pointer - under PS_NATIVESTRINGS the raw cast reinterpreted UTF-16 data as an Ansi name (truncating it at the first high byte), and older compilers lack the PWideChar overload the old cast accidentally relied on. uPSC_dll locals follow tbtstring so DLL/function names survive without a wide->ansi->wide roundtrip. Verified with kernel32 calls (lstrlenA/W, lstrcmpW, MulDiv, delayload, UnloadDLL) on DCC32/DCC64, Rtti + classic invoke paths, PS_NATIVESTRINGS on; OFF mode and Delphi 7 unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bring the FPC-only invoke paths in line with the x86/x64/Rtti paths: - var-param type lists, by-value marshaling and result stores for btAnsiChar/btAnsiString/btPAnsiChar/btPWideChar; managed result copies for the owned p-char types (btPChar keeps its raw upstream semantics) - PSBorrowPCharOwnership before the call / PSReownPCharOwnership after it (arm64: in the existing try..finally; arm/ppc: after the call block) - EmptyPchar as two #0 chars, used as nil replacement for by-value p-char arguments (arm/ppc previously dereferenced nil pointers here) - arm64 by-value btChar and btChar results are SizeOf(tbtChar) wide Also fill marshaling gaps these backends had for existing types: by-value btWideChar/btWideString/btUnicodeString/btCurrency/btVariant/ btClass/btInterface (arm/ppc), btExtended (arm), btWideChar and btCurrency results, ppc btS64/btU64/btCurrency results via rtINT64 (the asm stores r3 high / r4 low - directly readable as a big-endian int64), and hidden-result registration for AnsiString/WideString/ UnicodeString results. These files only compile for FPC ARM/AArch64/PowerPC targets, which are not available here - the port is review-based against a tested implementation of the same model; the known-open items (btArray result semantics on arm/ppc, btSet by-value on arm) are unchanged. Please double-check on a real target before relying on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Free Pascal 3.2.2 could not compile uPSRuntime.pas:
- Inc()/Dec() on a Variant is a Delphi extension; FPC rejects it
('Operator is not overloaded'). Use := Variant +/- 1 instead, which is
semantically identical and compiles on both.
- ResultAsRegister was recently added to the interface section, but its
implementation sits inside the {$else} region of empty_methods_handler.
FPC on cpu64/arm/powerpc defines empty_methods_handler, so the whole
region - including the implementation - disappeared and the unit failed
with 'Forward declaration not solved'. Move the implementation in front
of the region; it is plain Pascal with no dependency on the handlers.
Repro: fpc 3.2.2 x86_64-linux fails on master with 4 errors + 1 fatal;
with this change all uPS* core units compile and a script compile/run
smoke test passes on Linux. No change for Delphi (verified DCC32/DCC64).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Under PS_NATIVESTRINGS on FPC, tbtString is a distinct string type
(type string), and FPC rejects the direct hard cast tbtString(Variant)
('Illegal type conversion'). Convert through the plain string type first;
the result is identical on Delphi.
With this, all uPS* core units compile with FPC 3.2.2 (x86_64-linux) both
without defines and with PS_NATIVESTRINGS, and the script probe suite
passes on Linux in both modes. (FPC_UNICODE remains uncompilable - that
mode already fails on upstream master with unrelated pre-existing errors.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TValueArrayToArrayOfConst converts a TValue holding a PChar/PWideChar/
PAnsiChar into a plain vtPointer TVarRec, so p-char elements in an
array-of-const arrived unusable (Format raised 'invalid or incompatible
with argument'). The classic x86/x64 paths were unaffected.
Track the element base types while building the TValue array and restore
the proper VTypes (vtPChar/vtPWideChar; the data slot is shared, only the
tag differs) after conversion.
Repro: MyFormat('<%s>', [p]) with p: PChar under PS_NATIVESTRINGS on the
Rtti invoke path; now passes on DCC32/DCC64, Rtti and classic, and the
OFF-mode array-of-const test stays green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two 1-byte leftovers clipped wide btChar values to their low byte (the high
byte then carried heap garbage):
- TPSTypeRec.CalcSize kept btChar in the 1-byte group; RealSize drives
record field offsets and every size-based copy.
- CopyArrayContents copied btChar elements 1 byte at a time; this is also
the copy used when values are boxed for array-of-const elements, so
Format('%s', ['x']) received a WideChar with a random high byte.
Both now use SizeOf(TbtChar); identical code with PS_NATIVESTRINGS off.
Found by the consolidated test suite (single-char literal element probe).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The classic x64 path passed by-value dynamic array parameters as the
ADDRESS of the variable slot on FPC ({$IFDEF FPC} FVar.Dta instead of
FVar.Dta^), a leftover from an FPC 2.4-era fix. On FPC 3.x x86_64-linux the
callee then reads garbage instead of the array (Length() returns random
values, element access crashes with an access violation).
A dynamic array IS a pointer on every current target: pass its value, like
the Delphi branch always did. The 2.4 behavior is kept behind
{$IF FPC_VERSION < 3}.
Repro (fails on master with the FPC-compilation fixes applied, passes with
this change; fpc 3.2.2 x86_64-linux):
host: function DynTake(const A: TArr): Integer; // TArr = array of string
script: SetLength(a, 2); a[0] := 'x'; a[1] := 'y'; n := DynTake(a);
Delphi Win64 classic path unchanged and re-verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PS_NATIVESTRINGS: native string types for the script engine
Fixes #296.
Draft note: this branch is stacked on the individual fix PRs (#300-#308, #313) plus the feature PRs (#309-#312) - its first ~25 commits are exactly those. Review the small PRs first; once they land, this PR reduces to the native-strings commits and I will rebase and mark it ready.
The problem
On Delphi 2009+ the script type
stringisbtUnicodeString, but the engine's owntbtStringis hard-wired toAnsiString. Host functions registered withtbtStringparameters therefore receive the wrong width - the root cause of #296 (Format witharray of constbroken in 32 bit, arguments empty in 64 bit) and of every "my imported function sees garbage strings" report.The fix (opt-in, default off)
A new define
PS_NATIVESTRINGS(see PascalScript.inc, default NOT defined):tbtChar/tbtPChar/tbtStringfollow the host compiler's native types (Unicode on Delphi 2009+). Host functions registered withtbtStringsignatures then match the width the engine passes - the Array of const broken in 32bit, Stack issue in 64bit #296 repro goes from 0/6 to 6/6 checks.AnsiChar/AnsiString/PAnsiCharkeep REAL Ansi semantics through three new base types (btAnsiChar=31,btAnsiString=32,btPAnsiChar=33; the number space was free), andPCharmaps to a new ownedbtPWideChar=30.TbtAnsiString); string payloads in the byte stream are written/read asLength * SizeOf(tbtChar).With the define off, behavior is unchanged - that was a hard rule for every commit; the consolidated test suite (#317) passes identically on the unmodified tree.
Verification
packedkeyword from Accept Delphi's 'packed' keyword in type declarations (fixes #246) #314), including p-char ownership with leak-delta checks, wide/ansi record interop, kernel32 imports and stack-spill marshaling.🤖 Generated with Claude Code