Large commit with very significant speedup of class/object method calls ...

[syberrus]

...and several more features:

1. Added hashmap.h - extremely fast low-level C hash to use for perl code needs (see below).

2. Speeding up stash cache, make gv_stash* use the cache and by this make all of perl core and XS modules use
   this cache, not only S_method_common.

   • Change PL_stashcache from HV* to SVMAP* (hashmap of SVMAP_ENT values)
   • The lowest-level function is now gv_stashent which receives precomputed hash64 value for class name
     and searches the cache. If nothing found, calls slow S_gv_stashpvn (former Perl_gv_stashpvn) and
     fills the cache.
   • gv_stashpvn is now a wrapper for gv_stashent, computing hash value.
   • add API functions to clear stash cache locally (1 class) and globally. Make use of them.
   • added one more flag to gv_stash* - GV_CACHE_ONLY which returns stash only if found in cache,
     otherwise NULL. Required for correct behaviour of rule "IO globs take precedence over class names".
   • use private hash function PERL_HASH64 for computing hash values for class names. This function is
     much faster than PERL_HASH function (especially for long keys, which is likely for full class names)
     and returns a 64bit value. It doesn't use randomization because stash cache is a private core data,
     not a user data.

3. Speeding up method cache and super method cache, make gv_fetchmeth_/gv_fetchmethod_ use the cache.

   • store method cache in meta->mro_method instead of package stash's HV itself.
   • rename meta->mro_super -> meta->mro_supermethod
   • meta->mro_method & meta->mro_supermethod are both SVMAP*
   • The lowest-level function for gv_fetchmeth* is now gv_fetchmeth_ent which receives precomputed
     hash value for method name.
   • gv_fetchmeth_pvn is now a wrapper for gv_fetchmeth_ent, computing hash64 value.
   • The lowest-level function for gv_fetchmethod* is now gv_fetchmethod_ent which receives precomputed
     hash value for method name. Method cannot contain '::', gv_fetchmethod_ent doesn't search for '::'.
   • gv_fetchmethod_pvn is now a wrapper for gv_fetchmethod_ent, searching for '::', changing method name
     (and possibly stash, flags) and computing hash64 value.
   • new flag for gv_fetchmethod_pvn - GV_METHOD_SIMPLE, which means gv_fetchmethod_pvn doesn't need to scan
     for '::' in method name.

4. Precomputing hash64 values for perl code.

   • added 2 more op types:
     • OP_METHOD_SUPER "$proto->SUPER::func()"
     • OP_METHOD_REDIR "$proto->OtherClass::func()" or "$proto->OtherClass::SUPER::func()"
   • added new op struct type - struct methop (typedef METHOP).
     It actually behaves like UNOP for OP_METHOD and like SVOP for OP_METHOD_* and fully compatible with
     them for backward compability. It contains additional fields:
     • hash for method name (for OP_METHOD__). Method name SV and targ are still in ->op_sv and ->op_targ
     • hash, sv and targ for class name (for all types)
     • hash, sv and targ for redirect class name (for OP_METHOD_REDIR)
   • ck_method detects const method name, fills hash and sv, and creates OP_METHOD_NAMED
   • ck_method detects const method name with SUPER keyword, fills hash and sv, and creates OP_METHOD_SUPER
   • ck_method detects const method name with class redirect (possibly with SUPER keyword),
     fills hash, sv, rclass hash and rclass sv and creates OP_METHOD_REDIR
   • ck_subr now detects if left operand is a const (MyClass->method()), and stores the data (class hash and sv)
     in underlying OP_METHOD_ op.
   • all of these infos are used in runtime to greatly speedup pp_method* calls

5. OOP context for XSUBs.
   Calculated stash HV of left operand (object/class/IO/etc) is now saved in PL_methstash for later use by XSUBS.
   This prevents double caclucation of stash's HV for such functions as UNIVERSAL::can and so on and speeds up them
   a lot.
   Moreover it has one more usage: pp_entersub sets PL_methstash to NULL if CV is called as function, and sets
   PL_methstash to object/class/IO/etc stash's HV if CV is called as class/object/etc method.
   This makes it possible for XS code to detect if it is called as funtion or method in case if user wants to implement
   separate behaviour for those cases.
   For example if (PL_methstash) { ... called as method, object/class/etc stash is in PL_methstash }
   else { ... called as function }
   Additionaly for XSUBs like UNIVERSAL::can which always want to interpret its calls as method calls (i.e. first arg is
   object/class/etc), there is a macro dMETHSTASH which defines variable 'HV* stash' and either sets it to PL_methstash
   (if called as method) or (if called as function) calculate stash in the same way as pp_method* would do in case of
   real method call.
   Note that PL_methstash is not stacked (to not introduce any overheat), that means you have to either use it
   before you made a new perl CV call or save it to a variable.

6. Bring B, B::Deparse and B::Concise in consistency with new ops.

7. misc:

• bugfix in Opcode.xs: _safe_call_sv (and therefore module Safe) had critical vulnerability
  which made it possible for code in $safe->reval("...") to hack your entire program.
• Remove "local's" unneccesary overheat. ("local GLOB = GLOB" - OK, "local GLOB = CODEREF" - OVERHEAT)
  {
  local *MyClass::func = sub {...}; # LINE A
  ...
  } # LINE B
  This example caused global method cache reset at both lines A and B because glob_assign_ref and leave_scope
  thought that GV's GP refcnt was 2 (because of saving to Save Stack).
  Issue has been fixed (added gp_flags and new GP flag LOCALIZED, if flag is set then 1 refcnt from save stack is
  not counted in gv_method_changed calls).
• bugfix in dump.c : sv_dump(stash) (where stash is HV* - someone's stash) crashes if stash has cached destructor -
  sv_dump thought that it was a stash and tried to dump CV* as HV*.
• UNIVERSAL::can (in universal.c) is rewritten to make use of PL_methstash and runs much faster now.

[seveas]

hi @syberrus,

The perl developers don't use pull requests. Patches should be sent as RT tickets. I'd also advise you to split up this huge patch into smaller, self-contained ones which will be easier to review and apply.

[syberrus]

Unfortunately i can't split. Because data types of caches have changed, this leaded to a lot of changes in a lot of files.
I've spent about 2 weeks testing it with all of perl core tests and a lot of cpan modules. Everything seems to work. Except for a couple of modules, but they have buggy tests. I'll post to perl 5 list with explanation of all the details and benchmarks a little later.

[kentfredric]

Given Perl developers now do use Pull Requests, maybe this should be resurrected.

[xenu]

I believe this was discussed here (alternative link).

It resulted in the following commits (from oldest to newest):

• 4e7ebec
• d283e87
• 808724c
• a3d47e0
• b46e009
• b55b14d
• e3384dc
• d648ffc
• 7d6c333
• 810bd8b
• c6afe66
• c290e18

[syberrus]

Yes, there are most of changes in those commits. Additionally there was step 2 (speedup next::method), and it is now in next::XS module (>10 times next::method/can speedup), written in C++. I wrote about this several years ago and proposed to add it to the perl core as completely rewritten mro.xs (which is wasting CPU time) вс, 21 июн. 2020 г. в 10:09, xenu <notifications@github.com>:

…

I believe this was discussed here <https://www.nntp.perl.org/group/perl.perl5.porters/2014/07/msg218321.html> (alternative link <https://markmail.org/message/myqdlqjk5bylzh7j>). It resulted in the following commits (from oldest to newest): - 4e7ebec <4e7ebec> - d283e87 <d283e87> - 808724c <808724c> - a3d47e0 <a3d47e0> - b46e009 <b46e009> - b55b14d <b55b14d> - e3384dc <e3384dc> - d648ffc <d648ffc> - 7d6c333 <7d6c333> - 810bd8b <810bd8b> - c6afe66 <c6afe66> - c290e18 <c290e18> — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6HYZWZERBVM6BYGAJUZZDRXWW4FANCNFSM4ASCLUSQ> .

-- Oleg Pronin, Crazy Panda LTD