Implement Direct Threaded VM described in #51. Improving ~49%. by KeenS · Pull Request #52 · k-takata/Onigmo

KeenS · 2015-05-25T15:55:05Z

Here are benchmark scores. Benchmark suite is derived form http://sljit.sourceforge.net/regex_perf.html.

	Master	This PR	Improve Rate
Twain	47 ms	47 ms	0%
^Twain	47 ms	47 ms	0%
Twain$	47 ms	47 ms	0%
Huck[a-zA-Z]+\|Finn[a-zA-Z]+	127 ms	127 ms	0%
a[^x]{20}b	1172 ms	889 ms	31%
Tom\|Sawyer\|Huckleberry\|Finn	151 ms	153 ms	-1%
.{0,3}(Tom\|Sawyer\|Huckleberry\|Finn)	497 ms	449 ms	10%
[a-zA-Z]+ing	4032 ms	2705 ms	49%
^[a-zA-Z]{0,4}ing[^a-zA-Z]	96 ms	98 ms	-2%
[a-zA-Z]+ing$	4175 ms	2797 ms	49%
^[a-zA-Z ]{5,}$	1770 ms	1623 ms	9%
^.{16,20}$	1757 ms	1637 ms	7%
([a-f](.[d-m].){0,2}[h-n]){2}	1849 ms	1670 ms	11%
([A-Za-z]awyer\|[A-Za-z]inn)[^a-zA-Z]	656 ms	607 ms	8%
"[^"]{0,30}[?!\.]"	115 ms	93 ms	24%
Tom.{10,25}river\|river.{10,25}Tom	260 ms	262 ms	-1%

Env:

$ uname -a
Linux Dynabook 3.19.0-18-generic #18-Ubuntu SMP Tue May 19 18:31:35 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

$ cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 37
model name  : Intel(R) Core(TM) i5 CPU       M 450  @ 2.40GHz
stepping    : 5
microcode   : 0x2
cpu MHz     : 1199.000
cache size  : 3072 KB
physical id : 0
siblings    : 4
core id     : 0
cpu cores   : 2
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4788.65
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model       : 37
model name  : Intel(R) Core(TM) i5 CPU       M 450  @ 2.40GHz
stepping    : 5
microcode   : 0x2
cpu MHz     : 1199.000
cache size  : 3072 KB
physical id : 0
siblings    : 4
core id     : 0
cpu cores   : 2
apicid      : 1
initial apicid  : 1
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4788.65
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 2
vendor_id   : GenuineIntel
cpu family  : 6
model       : 37
model name  : Intel(R) Core(TM) i5 CPU       M 450  @ 2.40GHz
stepping    : 5
microcode   : 0x2
cpu MHz     : 2133.000
cache size  : 3072 KB
physical id : 0
siblings    : 4
core id     : 2
cpu cores   : 2
apicid      : 4
initial apicid  : 4
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4788.65
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor   : 3
vendor_id   : GenuineIntel
cpu family  : 6
model       : 37
model name  : Intel(R) Core(TM) i5 CPU       M 450  @ 2.40GHz
stepping    : 5
microcode   : 0x2
cpu MHz     : 1199.000
cache size  : 3072 KB
physical id : 0
siblings    : 4
core id     : 2
cpu cores   : 2
apicid      : 5
initial apicid  : 5
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt lahf_lm ida arat dtherm tpr_shadow vnmi flexpriority ept vpid
bugs        :
bogomips    : 4788.65
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

$ cat /proc/meminfo
MemTotal:        7965524 kB
MemFree:         6007232 kB
MemAvailable:    6635616 kB
Buffers:          147344 kB
Cached:           791320 kB
SwapCached:            0 kB
Active:          1170092 kB
Inactive:         614612 kB
Active(anon):     848752 kB
Inactive(anon):   164980 kB
Active(file):     321340 kB
Inactive(file):   449632 kB
Unevictable:          64 kB
Mlocked:              64 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:               856 kB
Writeback:             0 kB
AnonPages:        846212 kB
Mapped:           257944 kB
Shmem:            167692 kB
Slab:              81000 kB
SReclaimable:      52676 kB
SUnreclaim:        28324 kB
KernelStack:        7824 kB
PageTables:        27924 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     3982760 kB
Committed_AS:    4154020 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      551880 kB
VmallocChunk:   34359178716 kB
HardwareCorrupted:     0 kB
AnonHugePages:    327680 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       99392 kB
DirectMap2M:     8079360 kB

$ gcc --version
gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

coveralls · 2015-05-25T15:59:36Z

Coverage decreased (-0.01%) to 79.77% when pulling d18ec9a on KeenS:master into a0e7f59 on k-takata:master.

KeenS · 2015-07-28T02:13:08Z

Note direct threading is enabled only when USE_DIRECT_THREAD_VM is defined. You need to choose how to enable the option. One way is making it a compile option, and another is auto detect environment like picrin.

k-takata · 2015-07-28T02:42:30Z

Thank you for the information.
I think this is very useful, but I haven't tried yet.

Merge Onigmo 58fa099ed1a34367de67fb3d06dd48d076839692 + k-takata/Onigmo#52 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52756 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Merge Onigmo 58fa099ed1a34367de67fb3d06dd48d076839692 + k-takata/Onigmo#52 git-svn-id: svn+ssh://svn.ruby-lang.org/ruby/trunk@52756 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

k-takata · 2016-10-02T15:45:24Z

Thank you and sorry for the very late response. Merged.

KeenS · 2016-10-03T01:14:22Z

🎉

k-takata · 2016-12-06T13:36:07Z

I read the article about YARV and Direct Threaded Code again.
http://magazine.rubyist.net/?0008-YarvManiacs#l8
Isn't your implementation Token Threaded Code instead of Direct Threaded Code?
I'm going to rename the definition USE_DIRECT_THREADED_VM to USE_TOKEN_THREADED_VM.

This is token threaded VM, not direct threaded VM. See: #52

PR #52 was actually a token threaded VM.

KeenS · 2016-12-07T02:47:48Z

To tell the truth, I'm not familiar with threaded vm but I think you are right. It seems I wrote token threaded vm code.

KeenS added 3 commits May 25, 2015 22:13

introduce Direct Threaded VM

76adf2e

fix a bug

099616a

tune a bit

d18ec9a

k-takata merged commit 919810e into k-takata:master Oct 2, 2016

k-takata mentioned this pull request Oct 11, 2016

Use Direct Threaded Code #51

Closed

k-takata added a commit that referenced this pull request Dec 6, 2016

Rename USE_DIRECT_THREADED_VM to USE_TOKEN_THREADED_VM

be63582

This is token threaded VM, not direct threaded VM. See: #52

k-takata added a commit that referenced this pull request Dec 6, 2016

Update HISTORY

55253f9

PR #52 was actually a token threaded VM.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Direct Threaded VM described in #51. Improving ~49%.#52

Implement Direct Threaded VM described in #51. Improving ~49%.#52
k-takata merged 3 commits into
k-takata:masterfrom
KeenS:master

KeenS commented May 25, 2015

Uh oh!

coveralls commented May 25, 2015

Uh oh!

KeenS commented Jul 28, 2015

Uh oh!

k-takata commented Jul 28, 2015

Uh oh!

k-takata commented Oct 2, 2016

Uh oh!

KeenS commented Oct 3, 2016

Uh oh!

k-takata commented Dec 6, 2016

Uh oh!

KeenS commented Dec 7, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

KeenS commented May 25, 2015

Uh oh!

coveralls commented May 25, 2015

Uh oh!

KeenS commented Jul 28, 2015

Uh oh!

k-takata commented Jul 28, 2015

Uh oh!

k-takata commented Oct 2, 2016

Uh oh!

KeenS commented Oct 3, 2016

Uh oh!

k-takata commented Dec 6, 2016

Uh oh!

KeenS commented Dec 7, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants