segmation fault occurs when many groups are used #24

Closed
k-takata opened this Issue Aug 4, 2013 · 3 comments

Comments

Projects
None yet
1 participant
@k-takata
Owner

k-takata commented Aug 4, 2013

see: https://bugs.ruby-lang.org/issues/8716

WindowsとOS Xで検証しました。

  • 再現手順 ruby 2.0.0p247 (2013-06-27) [x64-mingw32]
a="()"
(32767.times{a<<'()'}
eval "/#{a}/=~''"
  • 再現手順 ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin12.2.1]
a="()"
(1<<21).times{a<<'()'}
eval "/#{a}/=~''"

regexec.c:match_at()で呼ばれるSTACK_INITがサイズを考慮せずにxallocaしているため、スタックオーバーフローしています。

@k-takata k-takata closed this in b9fba1d Jan 21, 2014

sorbits added a commit to textmate/Onigmo that referenced this issue Feb 20, 2014

@k-takata

This comment has been minimized.

Show comment
Hide comment
@k-takata

k-takata Feb 21, 2014

Owner

It seems that b9fba1d causes random crashes with multithread settings.

Owner

k-takata commented Feb 21, 2014

It seems that b9fba1d causes random crashes with multithread settings.

@k-takata k-takata reopened this Feb 21, 2014

k-takata added a commit that referenced this issue Apr 9, 2014

additional fix for #24
Use xalloca() to allocate a stack.

@k-takata k-takata closed this in e62d2c5 Apr 12, 2014

k-takata added a commit that referenced this issue Apr 13, 2014

fix: segmation fault occurs when many groups are used (Close #24)
Allocate Onigmo's stack from a heap instead of a stack when many groups
are used.

see: https://bugs.ruby-lang.org/issues/8716
(cherry picked from commit b9fba1d)

Conflicts:

	regexec.c

k-takata added a commit that referenced this issue Apr 13, 2014

additional fix for #24
Use xalloca() to allocate a stack.
(cherry picked from commit a0efc0a)
@k-takata

This comment has been minimized.

Show comment
Hide comment
@k-takata

k-takata Apr 16, 2014

Owner

Oops, still SEGV occurs.
There were two causes with this issue.

  1. Using alloca() may cause stack overflow when allocating very large area. (Already fixed.)
  2. Group number is managed by MemNumType which is short int. Thus, the number become negative if the number of group exceeds 32767. It may cause access errors. (Not fixed yet.)

Should Onigmo support more than 32767 groups?

Owner

k-takata commented Apr 16, 2014

Oops, still SEGV occurs.
There were two causes with this issue.

  1. Using alloca() may cause stack overflow when allocating very large area. (Already fixed.)
  2. Group number is managed by MemNumType which is short int. Thus, the number become negative if the number of group exceeds 32767. It may cause access errors. (Not fixed yet.)

Should Onigmo support more than 32767 groups?

@k-takata k-takata reopened this Apr 16, 2014

@k-takata k-takata closed this in c7cda4e Apr 18, 2014

@k-takata

This comment has been minimized.

Show comment
Hide comment
@k-takata

k-takata Apr 18, 2014

Owner

Now the maximum number of capture groups is explicitly limited to 32767 (ONIG_MAX_CAPTURE_GROUP_NUM).

Owner

k-takata commented Apr 18, 2014

Now the maximum number of capture groups is explicitly limited to 32767 (ONIG_MAX_CAPTURE_GROUP_NUM).

k-takata added a commit that referenced this issue Apr 19, 2014

[bug] add checking the number of capture groups (fix #24)
The number of capture groups is handled with `MemNumType` which is
`short int`. Thus the maximum number is limited to 32767.
A check of the number is added in `scan_env_add_mem_entry()`.
Also a new error code `ONIGERR_TOO_MANY_CAPTURE_GROUPS` is added.
(cherry picked from commit c7cda4e)

k-takata added a commit to k-takata/bregonig that referenced this issue Sep 13, 2014

Ver.3.06
* Onigmo (Oniguruma-mod) 5.15.0 for bregonig.dll を使用。
  https://github.com/k-takata/Onigmo/tree/Onigmo-5.15.0_for_bregonig
  - Unicode 7.0 に対応
  - Oniguruma 5.9.5 をマージ
  - 大量のグループを使うと落ちる問題を修正
    k-takata/Onigmo#24
  - /\x{1ffc}/i =~ "\x1ff3" がマッチしない問題を修正
  - UTF-16/32 で /[a-c#]+\W/ =~ "def#" がマッチしない問題を修正
  - /(?i)\u0149\u0149/ =~ "\u0149\u0149" がマッチしない問題を修正
    k-takata/Onigmo#40
  - 文字クラスの中で /w を使い、/i オプションを指定したときの問題を修正
    k-takata/Onigmo#4
  - 文字プロパティが /i オプションを無視する問題を修正
    k-takata/Onigmo#41
  - "ab" =~ /(?!^a).*b/ がマッチしない問題を修正
    k-takata/Onigmo#44

k-takata added a commit to k-takata/bregonig that referenced this issue Sep 13, 2014

Ver.2.09
* Oniguruma 5.9.5 改変版2 for bregonig.dll V2 を使用。
  https://github.com/k-takata/Onigmo/tree/onig-5.9.5-mod2_for_bregonig-v2
  - ベースバージョンを Oniguruma 5.9.4 から 5.9.5 に変更。
  - 大量のグループを使うと落ちる問題を修正
    k-takata/Onigmo#24
  - UTF-16/32 で /[a-c#]+\W/ =~ "def#" がマッチしない問題を修正
  - /(?i)\u0149\u0149/ =~ "\u0149\u0149" がマッチしない問題を修正
    k-takata/Onigmo#40
  - "ab" =~ /(?!^a).*b/ がマッチしない問題を修正
    k-takata/Onigmo#44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment