Python logic error when deal with re and muti-threading

[bee13oy]

BPO	24555
Nosy	@ezio-melotti, @bitdancer
Superseder	bpo-23690: re functions never release GIL
Files	python_logic_error.pdf: bug detail infomation poc&bug_detail.zip: 1. normal_case.py 2. poc.py 3. bug details

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-11-16.13:51:53.787>
created_at = <Date 2015-07-03.06:20:15.467>
labels = ['expert-regex', 'type-bug']
title = 'Python logic error when deal with re and muti-threading'
updated_at = <Date 2017-11-16.13:51:53.786>
user = 'https://bugs.python.org/bee13oy'

bugs.python.org fields:

activity = <Date 2017-11-16.13:51:53.786>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-11-16.13:51:53.787>
closer = 'serhiy.storchaka'
components = ['Regular Expressions']
creation = <Date 2015-07-03.06:20:15.467>
creator = 'bee13oy'
dependencies = []
files = ['39850', '39853']
hgrepos = []
issue_num = 24555
keywords = []
message_count = 4.0
messages = ['246138', '246180', '246188', '246193']
nosy_count = 4.0
nosy_names = ['ezio.melotti', 'mrabarnett', 'r.david.murray', 'bee13oy']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '23690'
type = 'behavior'
url = 'https://bugs.python.org/issue24555'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5']

[bee13oy]

Bug 0x01 is the main problem.

t.start()
t.join(timeout)
In normal case, I run a while() in sub-thread, the main thread will get the control of the program after the sub-thread is timed out.
But, in our POC, even the sub-thread timed out, the main thread still can't execute continue. After analyzing, I found the main thread trapped into an infinite loop like I described in the PDF.

[bitdancer]

If you re-post your bug information in a plain text and/or test program format it might get faster attention.

[bee13oy]

#Python logic error when deal with re and muti-threading
##Bug Description

When use re and multi-threading it will trigger the bug.

Bug type: Logic Error

Test Enviroment:

• Windows 7 SP1 x64 + python 3.4.3
• Linux kali 3.14-kali1-amd64 + python 2.7.3

-----------------------------Normal Case------------------------

• 1. main-thread: join(timeout), wait for sub-thread finished -
• 2. sub-thread: while(1), an infinite loop -
     ----------------------------------------------------------------

Test Code:

#!/usr/bin/python
__author__ = 'bee13oy'
import re
import threading
timeout = 2
source = "(.*(.)?)*bcd\\t\\n\\r\\f\\a\\e\\071\\x3b\\$\\\\\?caxyz"
def run(source):
    while(1):
        print("test1")   
def handle():
        try:
            t = threading.Thread(target=run,args=(source,))
            t.setDaemon(True)
            t.start()
            t.join(timeout)
            print("thread finished...It's an normal case!\n")
        except:
            print("exception ...\n")		
handle()

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

-----------------------------Bug Case-----------------------------------------------------------------------------

• 1. main-thread: join(timeout), wait for sub-thread finished -
• 2. sub-thread: 1)we construct the special pattern "(.*(.)?)*bcd\\t\\n\\r\\f\\a\\e\\071\\x3b\\$\\\\\?caxyz" -
     2)regexp.search() can't deal with it, and hang up -
     3)join(timeout), and the sub-thread was over time, at this time, main-thread should have got -
     the control of the program. But it didn't. -
     ------------------------------------------------------------------------------------------------------------------

POC:

#!/usr/bin/python
__author__ = 'bee13oy'
import re
import os
import threading
timeout = 2
source = "(.*(.)?)*bcd\\t\\n\\r\\f\\a\\e\\071\\x3b\\$\\\\\?caxyz"
def run(source):
    regexp = re.compile(r''+source+'')
    sgroup = regexp.search(source)       
def handle():
        try:
            t = threading.Thread(target=run,args=(source,))
            t.setDaemon(True)
            t.start()
            t.join(timeout)
            print("finished...\n")
        except:
            print("exception ...\n")		
handle()

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
----------------------------------------------------------------

•                      Bug Analyze                         -
  

----------------------------------------------------------------
When we use Python multithreading, and use join(timeout) to wait until the thread terminates or timed out.
1. In normal case, I run a while() in sub-thread, the main thread will get the control of the program after the sub-thread is timed out.
2. In our POC, even the sub-thread timed out, the main thread still can't execute continue. After analyzing, I found the main thread trapped into an infinite loop.

At first, it will run into the sub-thread, but it can't end normally.
At this time, join(timeout) will wait for the sub-thread return or timed out, and try to call timed out function in order that main thread can get the control of the program.

The bug is that the sub-thread was into an infinite loop and the main-thread was into an infinite loop too, which causes the program to be hang up.

By analyzing the source code of Python, we found that:

• sub-thread is into an infinite loop (code block 0)
• main-thread is into an infinite loop (code block 1)

-----------------------------code block 0----------------------------------

• the following code is where sub-thread trapped into an infinite loop: -
  ---------------------------------------------------------------------------
  the following code is where the sub-thread trapped into an **infinite loop**:

LOCAL(Py_ssize_t)
SRE(match)(SRE_STATE* state, SRE_CODE* pattern, int match_all)
{
    SRE_CHAR* end = (SRE_CHAR *)state->end;
    Py_ssize_t alloc_pos, ctx_pos = -1;
    Py_ssize_t i, ret = 0;
    Py_ssize_t jump;
    unsigned int sigcount=0;
    SRE(match_context)* ctx;
    SRE(match_context)* nextctx;
    TRACE(("|%p|%p|ENTER\n", pattern, state->ptr));
    DATA_ALLOC(SRE(match_context), ctx);
    ctx->last_ctx_pos = -1;
    ctx->jump = JUMP_NONE;
    ctx->pattern = pattern;
    ctx->match_all = match_all;
    ctx_pos = alloc_pos;
	.....	
	/* Cycle code which will never return*/
	for (;;) {
	++sigcount;
	if ((0 == (sigcount & 0xfff)) && PyErr_CheckSignals())
		RETURN_ERROR(SRE_ERROR_INTERRUPTED);

	switch (*ctx->pattern++) {
	case SRE_OP_MARK:
		/* set mark */
		/* <MARK> <gid> */
		TRACE(("|%p|%p|MARK %d\n", ctx->pattern,
			   ctx->ptr, ctx->pattern[0]));
	.....
}

-----------------------------code block 1----------------------------------

• the following code is where main-thread trapped into an infinite loop: -
  ---------------------------------------------------------------------------

static void take_gil(PyThreadState *tstate)
{
    int err;
    if (tstate == NULL)
        Py_FatalError("take_gil: NULL tstate");
    
    err = errno;
    MUTEX_LOCK(gil_mutex);    
    if (!_Py_atomic_load_relaxed(&gil_locked))
        goto _ready;		
	/*Cycle code which will never return*/
    while (_Py_atomic_load_relaxed(&gil_locked)) {
        int timed_out = 0;
        unsigned long saved_switchnum;
        saved_switchnum = gil_switch_number;
        COND_TIMED_WAIT(gil_cond, gil_mutex, INTERVAL, timed_out);
        /* If we timed out and no switch occurred in the meantime, it is time
           to ask the GIL-holding thread to drop it. */
        if (timed_out &&
            _Py_atomic_load_relaxed(&gil_locked) &&
            gil_switch_number == saved_switchnum) {
            SET_GIL_DROP_REQUEST();
        }
    }
	.....
}

[mrabarnett]

Your regex is a pathological case: it suffers from catastrophic backtracking and can take a long time to finish.

The other problem is that the re module never releases the GIL, so while it's performing the search in the low-level C code, other Python threads don't get a chance to run.