CGI HTTP server not running scripts from subdirectories

[kssolnushkin]

BPO	21323
Nosy	@benjaminp, @ned-deily
Files	test.py: A trivial CGI script to test HTTP server functionality 21323.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-07-13.05:31:22.164>
created_at = <Date 2014-04-21.19:05:31.250>
labels = ['easy', 'type-bug']
title = 'CGI HTTP server not running scripts from subdirectories'
updated_at = <Date 2014-07-13.05:31:22.137>
user = 'https://bugs.python.org/kssolnushkin'

bugs.python.org fields:

activity = <Date 2014-07-13.05:31:22.137>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2014-07-13.05:31:22.164>
closer = 'ned.deily'
components = []
creation = <Date 2014-04-21.19:05:31.250>
creator = 'k.s.solnushkin'
dependencies = []
files = ['34993', '35908']
hgrepos = []
issue_num = 21323
keywords = ['patch', 'easy']
message_count = 10.0
messages = ['216960', '222402', '222421', '222422', '222594', '222603', '222729', '222793', '222910', '222912']
nosy_count = 5.0
nosy_names = ['benjamin.peterson', 'ned.deily', 'python-dev', 'k.s.solnushkin', 'zbyrne']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21323'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4', 'Python 3.5']

[kssolnushkin]

Somewhere between Python 3.3 and 3.4, a bug was introduced that forbids the "http.server" module, working in CGI server mode, to run scripts residing in subdirectories.

This will break existing software that relies on this feature.

How to reproduce the bug:

1. Create a temporary directory and enter it.
2. Create a directory "cgi-bin", and then directory "test" inside "cgi-bin".
3. Create a file "test.py" in "cgi-bin/test" with the following contents (see also attachment to this bug report):

print("""Content-type: text/plain

CGI script executed successfully!
""")

4. When run, it should print the following:

Content-type: text/plain

CGI script executed successfully!

5. Now, run Python 3.3 in CGI HTTP server mode:

c:\Python33\python.exe -m http.server --cgi 8000

A request to "http://localhost:8000/cgi-bin/test/test.py" then produces the following in the HTTP server log:

Serving HTTP on 0.0.0.0 port 8000 ...
127.0.0.1 - - [21/Apr/2014 22:59:11] "GET /cgi-bin/test/test.py HTTP/1.0" 200 -
127.0.0.1 - - [21/Apr/2014 22:59:11] command: c:\Python33\python.exe -u C:\TMP\cgi-bin\test\test.py ""
127.0.0.1 - - [21/Apr/2014 22:59:11] CGI script exited OK

6. Now, try this with Python 3.4, and the request will fail with the following in the log:

C:\TMP>c:\Python34\python.exe -m http.server --cgi 8000
Serving HTTP on 0.0.0.0 port 8000 ...
127.0.0.1 - - [21/Apr/2014 23:02:38] code 403, message CGI script is not a plain file ('/cgi-bin/test')
127.0.0.1 - - [21/Apr/2014 23:02:38] "GET /cgi-bin/test/test.py HTTP/1.0" 403 -

This _could_ be related to the change introduced by bpo-19435, although I am not sure.

Tested with Windows XP SP3.

[kssolnushkin]

Hi, I am curious about the fate of this issue -- whether it will be recognised as a bug (possibly a regression bug). Remember, it worked in Python 3.3 but stopped working in 3.4.

[ned-deily]

The change in behavior is definitely caused by the changes for bpo-19435. The code added there does not work correctly if there are nested directories in the "cgi-bin" directory; for one thing, the initial os.path.isdir(scriptdir) test in run_cgi is failing because the path in nextdir is missing the dir component: it's testing /path/to/test instead of /path/to/cgi-bin/test. I didn't look further. Along with a fix, there should be a test for nested directories. And decide whether to fix the regression it presumably caused in the security branches as well.

http://hg.python.org/cpython/file/d25ae22cc992/Lib/http/server.py#l1009

[ned-deily]

BTW, if someone is looking for an issue to help out with, this might be a good choice.

[zbyrne]

Hi, I'm new. I wrote a test for nested directories under cgi-bin and got that to pass without failing the test added for 19435 by undoing most of the changes to run_cgi() but building path from the values in self.cgi_info. Thoughts?

[ned-deily]

Zach, thanks for the patch and the test. Someone will review it in the coming days. In the meantime, if you haven't already, please review and submit the Python Contributor's Agreement: https://www.python.org/psf/contrib/contrib-form/ otherwise we won't be able to use your contribution.

[ned-deily]

The fix and test look good to me. I'll apply it after Zach submits the contributor agreement.

[zbyrne]

Done and done.

[python-dev]

New changeset d367ea865ea4 by Ned Deily in branch '2.7':
Issue bpo-21323: Fix CGIHTTPServer to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/d367ea865ea4

New changeset 4de94641ba3e by Ned Deily in branch '3.2':
Issue bpo-21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/4de94641ba3e

New changeset b957f475e41e by Ned Deily in branch '3.3':
Issue bpo-21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/b957f475e41e

New changeset 385f4406dc26 by Ned Deily in branch '3.4':
Issue bpo-21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/385f4406dc26

New changeset 22e5a85ba840 by Ned Deily in branch 'default':
Issue bpo-21323: Fix http.server to again handle scripts in CGI subdirectories,
http://hg.python.org/cpython/rev/22e5a85ba840

[ned-deily]

Since the original changes for bpo-19435 were also applied to branches in security fix mode, the fix here is applied to them as well, with the exception of 3.1 which has subsequently moved to end-of-life status. Applied for release in 2.7.9, 3.2.6, 3.3.6, 3.4.2, and 3.5.0. Thanks again, Konstantin and Zach.