Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

segmation fault occurs when many groups are used #24

Closed
k-takata opened this issue Aug 4, 2013 · 3 comments
Closed

segmation fault occurs when many groups are used #24

k-takata opened this issue Aug 4, 2013 · 3 comments
Labels

Comments

@k-takata
Copy link
Owner

k-takata commented Aug 4, 2013

see: https://bugs.ruby-lang.org/issues/8716

WindowsとOS Xで検証しました。

  • 再現手順 ruby 2.0.0p247 (2013-06-27) [x64-mingw32]
a="()"
(32767.times{a<<'()'}
eval "/#{a}/=~''"
  • 再現手順 ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin12.2.1]
a="()"
(1<<21).times{a<<'()'}
eval "/#{a}/=~''"

regexec.c:match_at()で呼ばれるSTACK_INITがサイズを考慮せずにxallocaしているため、スタックオーバーフローしています。

sorbits added a commit to textmate/Onigmo that referenced this issue Feb 20, 2014
@k-takata
Copy link
Owner Author

It seems that b9fba1d causes random crashes with multithread settings.

@k-takata k-takata reopened this Feb 21, 2014
k-takata added a commit that referenced this issue Apr 9, 2014
Use xalloca() to allocate a stack.
k-takata added a commit that referenced this issue Apr 13, 2014
Allocate Onigmo's stack from a heap instead of a stack when many groups
are used.

see: https://bugs.ruby-lang.org/issues/8716
(cherry picked from commit b9fba1d)

Conflicts:

	regexec.c
k-takata added a commit that referenced this issue Apr 13, 2014
Use xalloca() to allocate a stack.
(cherry picked from commit a0efc0a)
@k-takata
Copy link
Owner Author

Oops, still SEGV occurs.
There were two causes with this issue.

  1. Using alloca() may cause stack overflow when allocating very large area. (Already fixed.)
  2. Group number is managed by MemNumType which is short int. Thus, the number become negative if the number of group exceeds 32767. It may cause access errors. (Not fixed yet.)

Should Onigmo support more than 32767 groups?

@k-takata k-takata reopened this Apr 16, 2014
@k-takata
Copy link
Owner Author

Now the maximum number of capture groups is explicitly limited to 32767 (ONIG_MAX_CAPTURE_GROUP_NUM).

k-takata added a commit that referenced this issue Apr 19, 2014
The number of capture groups is handled with `MemNumType` which is
`short int`. Thus the maximum number is limited to 32767.
A check of the number is added in `scan_env_add_mem_entry()`.
Also a new error code `ONIGERR_TOO_MANY_CAPTURE_GROUPS` is added.
(cherry picked from commit c7cda4e)
k-takata added a commit to k-takata/bregonig that referenced this issue Sep 13, 2014
* Onigmo (Oniguruma-mod) 5.15.0 for bregonig.dll を使用。
  https://github.com/k-takata/Onigmo/tree/Onigmo-5.15.0_for_bregonig
  - Unicode 7.0 に対応
  - Oniguruma 5.9.5 をマージ
  - 大量のグループを使うと落ちる問題を修正
    k-takata/Onigmo#24
  - /\x{1ffc}/i =~ "\x1ff3" がマッチしない問題を修正
  - UTF-16/32 で /[a-c#]+\W/ =~ "def#" がマッチしない問題を修正
  - /(?i)\u0149\u0149/ =~ "\u0149\u0149" がマッチしない問題を修正
    k-takata/Onigmo#40
  - 文字クラスの中で /w を使い、/i オプションを指定したときの問題を修正
    k-takata/Onigmo#4
  - 文字プロパティが /i オプションを無視する問題を修正
    k-takata/Onigmo#41
  - "ab" =~ /(?!^a).*b/ がマッチしない問題を修正
    k-takata/Onigmo#44
k-takata added a commit to k-takata/bregonig that referenced this issue Sep 13, 2014
* Oniguruma 5.9.5 改変版2 for bregonig.dll V2 を使用。
  https://github.com/k-takata/Onigmo/tree/onig-5.9.5-mod2_for_bregonig-v2
  - ベースバージョンを Oniguruma 5.9.4 から 5.9.5 に変更。
  - 大量のグループを使うと落ちる問題を修正
    k-takata/Onigmo#24
  - UTF-16/32 で /[a-c#]+\W/ =~ "def#" がマッチしない問題を修正
  - /(?i)\u0149\u0149/ =~ "\u0149\u0149" がマッチしない問題を修正
    k-takata/Onigmo#40
  - "ab" =~ /(?!^a).*b/ がマッチしない問題を修正
    k-takata/Onigmo#44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant