-m cProfile -o f.pstats with a script that does chdir() writes to the changed directory and not .

[asottile]

BPO	40492
Nosy	@taleinat, @serhiy-storchaka, @asottile
PRs	bpo-40492: Fix --outfile when the program being profiled changes the working directory #19910 [3.9] bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) #22752 [3.9] bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) #22753 [3.8] bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) #22754 [3.8] bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910) #22755

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-10-18.21:00:06.199>
created_at = <Date 2020-05-04.03:36:27.218>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = '-m cProfile -o f.pstats with a script that does chdir() writes to the changed directory and not `.`'
updated_at = <Date 2020-10-18.21:16:27.048>
user = 'https://github.com/asottile'

bugs.python.org fields:

activity = <Date 2020-10-18.21:16:27.048>
actor = 'taleinat'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-18.21:00:06.199>
closer = 'taleinat'
components = ['Library (Lib)']
creation = <Date 2020-05-04.03:36:27.218>
creator = 'Anthony Sottile'
dependencies = []
files = []
hgrepos = []
issue_num = 40492
keywords = ['patch']
message_count = 9.0
messages = ['368008', '368018', '368918', '368930', '378851', '378895', '378897', '378900', '378901']
nosy_count = 3.0
nosy_names = ['taleinat', 'serhiy.storchaka', 'Anthony Sottile']
pr_nums = ['19910', '22752', '22753', '22754', '22755']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40492'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[asottile]

just spent quite a bit of time debugging this, here's a minimal example:

# rm -rf a (if you want to reset)
cd /tmp
mkdir -p a/b
echo 'import os; os.chdir("..")' > a/b/c.py
cd a/b
python3 -m cProfile -o log.pstats c.py
ls -al
ls -al ..

example output:

$ cd /tmp
$ mkdir -p a/b
$ echo 'import os; os.chdir("..")' > a/b/c.py
$ cd a/b
$ python3 -m cProfile -o log.pstats c.py
$ ls -al
total 12
drwxr-xr-x 2 asottile asottile 4096 May  3 20:35 .
drwxr-xr-x 3 asottile asottile 4096 May  3 20:35 ..
-rw-r--r-- 1 asottile asottile   26 May  3 20:35 c.py
$ ls -al ..
total 16
drwxr-xr-x  3 asottile asottile 4096 May  3 20:35 .
drwxrwxrwt 28 root     root     4096 May  3 20:35 ..
drwxr-xr-x  2 asottile asottile 4096 May  3 20:35 b
-rw-r--r--  1 asottile asottile  395 May  3 20:35 log.pstats

happy to work on a patch if this seems like a good idea to fix

[serhiy-storchaka]

I think it is worth to fix and it should be not difficult to fix. Either make the output file name absolute before executing the script, or open the output file before executing the script (what is easier). Unless I miss something.

[asottile]

@serhiy: this was pretty straightfowrard to fix as you suspected -- would you be able to review it?

[serhiy-storchaka]

I am not sure what is better: to fix it at high level, in main() for cProfile and cProfile, or at low level, in methods runctx() and run() of profile._Utils.

[taleinat]

I am not sure what is better: to fix it at high level, in main() for cProfile and cProfile, or at low level, in methods runctx() and run() of profile._Utils.

Maybe both, to keep them decoupled and be sure of correct behavior in all cases?

[taleinat]

New changeset 3c0ac18 by Anthony Sottile in branch 'master':
bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910)
3c0ac18

[taleinat]

Many thanks for the report and the PR, Anthony!

Serhiy, do you think it is worth also calling abspath in the profile._Utils as you mentioned, for cases where those may be called directly?

[taleinat]

New changeset 1c5a657 by Anthony Sottile in branch '3.8':
bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910)
1c5a657

[taleinat]

New changeset 7c94902 by Anthony Sottile in branch '3.9':
bpo-40492: Fix --outfile with relative path when the program changes it working dir (GH-19910)
7c94902