Skip to content

Commit c3cd9df

Browse files
nnorwitznnorwitz
committed
Add some doc about using valgrind
1 parent b5d7702 commit c3cd9df

File tree

3 files changed

+289
-0
lines changed

3 files changed

+289
-0
lines changed

‎Misc/README

+2
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,10 @@ pymemcompat.h Memory interface compatibility file.
2424
python.man UNIX man page for the python interpreter
2525
python-mode.el Emacs mode for editing Python programs
2626
README The file you're reading now
27+
README.valgrind Information for Valgrind users, see valgrind-python.supp
2728
RFD Request For Discussion about a Python newsgroup
2829
RPM (Old) tools to build RPMs
2930
SpecialBuilds.txt Describes extra symbols you can set for debug builds
3031
setuid-prog.c C helper program for set-uid Python scripts
3132
vgrindefs Python configuration for vgrind (a generic pretty printer)
33+
valgrind-python.supp Valgrind suppression file, see README.valgrind

‎Misc/README.valgrind

+71
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
This document describes some caveats about the use of Valgrind with
2+
Python. Valgrind is used periodically by Python developers to try
3+
to ensure there are no memory leaks or invalid memory reads/writes.
4+
5+
If you don't want to read about the details of using Valgrind, there
6+
are still two things you must do to suppress the warnings. First,
7+
you must use a suppressions file. One is supplied in
8+
Misc/valgrind-python.supp. Second, you must do one of the following:
9+
10+
* Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c,
11+
then rebuild Python
12+
* Uncomment the lines in Misc/valgrind-python.supp that
13+
suppress the warnings for PyObject_Free and PyObject_Realloc
14+
15+
Details:
16+
--------
17+
Python uses its own allocation scheme on top of malloc called PyMalloc.
18+
Valgrind my show some unexpected results when PyMalloc is used.
19+
Starting with Python 2.3, PyMalloc is used by default. You can disable
20+
PyMalloc when configuring python by adding the --without-pymalloc option.
21+
If you disable PyMalloc, most of the information in this document and
22+
the supplied suppressions file will not be useful.
23+
24+
If you use valgrind on a default build of Python, you will see
25+
many errors like:
26+
27+
==6399== Use of uninitialised value of size 4
28+
==6399== at 0x4A9BDE7E: PyObject_Free (obmalloc.c:711)
29+
==6399== by 0x4A9B8198: dictresize (dictobject.c:477)
30+
31+
These are expected and not a problem. Tim Peters explains
32+
the situation:
33+
34+
PyMalloc needs to know whether an arbitrary address is one
35+
that's managed by it, or is managed by the system malloc.
36+
The current scheme allows this to be determined in constant
37+
time, regardless of how many memory areas are under pymalloc's
38+
control.
39+
40+
The memory pymalloc manages itself is in one or more "arenas",
41+
each a large contiguous memory area obtained from malloc.
42+
The base address of each arena is saved by pymalloc
43+
in a vector, and a field at the start of each arena contains
44+
the index of that arena's base address in that vector.
45+
46+
Given an arbitrary address, pymalloc computes the arena base
47+
address corresponding to it, then looks at "the index" stored
48+
near there. If the index read up is out of bounds for the
49+
vector of arena base addresses pymalloc maintains, then
50+
pymalloc knows for certain that this address is not under
51+
pymalloc's control. Otherwise the index is in bounds, and
52+
pymalloc compares
53+
54+
the arena base address stored at that index in the vector
55+
56+
to
57+
58+
the computed arena address
59+
60+
pymalloc controls this arena if and only if they're equal.
61+
62+
It doesn't matter whether the memory pymalloc reads up ("the
63+
index") is initialized. If it's not initialized, then
64+
whatever trash gets read up will lead pymalloc to conclude
65+
(correctly) that the address isn't controlled by it.
66+
67+
This determination has to be made on every call to one of
68+
pymalloc's free/realloc entry points, so its speed is critical
69+
(Python allocates and frees dynamic memory at a ferocious rate
70+
-- everything in Python, from integers to "stack frames",
71+
lives in the heap).

‎Misc/valgrind-python.supp

+216
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
#
2+
# This is a valgrind suppression file that should be used when using valgrind.
3+
#
4+
# Here's an example of running valgrind:
5+
#
6+
# cd python/dist/src
7+
# valgrind --tool=memcheck --suppressions=Misc/valgrind-python.supp \
8+
# ./python -E -tt ./Lib/test/regrtest.py -u bsddb,network
9+
#
10+
# You must edit Objects/obmalloc.c and uncomment Py_USING_MEMORY_DEBUGGER
11+
# to use the preferred suppressions with Py_ADDRESS_IN_RANGE.
12+
#
13+
# If you do not want to recompile Python, you can uncomment
14+
# suppressions for PyObject_Free and PyObject_Realloc.
15+
#
16+
# See Misc/README.valgrind for more information.
17+
18+
# all tool names: Addrcheck,Memcheck,cachegrind,helgrind,massif
19+
{
20+
ADDRESS_IN_RANGE/Invalid read of size 4
21+
Memcheck:Addr4
22+
fun:Py_ADDRESS_IN_RANGE
23+
}
24+
25+
{
26+
ADDRESS_IN_RANGE/Invalid read of size 4
27+
Memcheck:Value4
28+
fun:Py_ADDRESS_IN_RANGE
29+
}
30+
31+
{
32+
ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
33+
Memcheck:Cond
34+
fun:Py_ADDRESS_IN_RANGE
35+
}
36+
37+
###{
38+
### ADDRESS_IN_RANGE/Invalid read of size 4
39+
### Memcheck:Addr4
40+
### fun:PyObject_Free
41+
###}
42+
###
43+
###{
44+
### ADDRESS_IN_RANGE/Invalid read of size 4
45+
### Memcheck:Value4
46+
### fun:PyObject_Free
47+
###}
48+
###
49+
###{
50+
### ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
51+
### Memcheck:Cond
52+
### fun:PyObject_Free
53+
###}
54+
55+
###{
56+
### ADDRESS_IN_RANGE/Invalid read of size 4
57+
### Memcheck:Addr4
58+
### fun:PyObject_Realloc
59+
###}
60+
###
61+
###{
62+
### ADDRESS_IN_RANGE/Invalid read of size 4
63+
### Memcheck:Value4
64+
### fun:PyObject_Realloc
65+
###}
66+
###
67+
###{
68+
### ADDRESS_IN_RANGE/Conditional jump or move depends on uninitialised value
69+
### Memcheck:Cond
70+
### fun:PyObject_Realloc
71+
###}
72+
73+
###
74+
### All the suppressions below are for errors that occur within libraries
75+
### that Python uses. The problems to not appear to be related to Python's
76+
### use of the libraries.
77+
###
78+
{
79+
GDBM problems, see test_gdbm
80+
Memcheck:Param
81+
write(buf)
82+
fun:write
83+
fun:gdbm_open
84+
85+
}
86+
87+
###
88+
### These occur from somewhere within the SSL, when running
89+
### test_socket_sll. They are too general to leave on by default.
90+
###
91+
###{
92+
### somewhere in SSL stuff
93+
### Memcheck:Cond
94+
### fun:memset
95+
###}
96+
###{
97+
### somewhere in SSL stuff
98+
### Memcheck:Value4
99+
### fun:memset
100+
###}
101+
###
102+
###{
103+
### somewhere in SSL stuff
104+
### Memcheck:Cond
105+
### fun:MD5_Update
106+
###}
107+
###
108+
###{
109+
### somewhere in SSL stuff
110+
### Memcheck:Value4
111+
### fun:MD5_Update
112+
###}
113+
114+
#
115+
# All of these problems come from using test_socket_ssl
116+
#
117+
{
118+
from test_socket_ssl
119+
Memcheck:Cond
120+
fun:BN_bin2bn
121+
}
122+
123+
{
124+
from test_socket_ssl
125+
Memcheck:Cond
126+
fun:BN_num_bits_word
127+
}
128+
129+
{
130+
from test_socket_ssl
131+
Memcheck:Value4
132+
fun:BN_num_bits_word
133+
}
134+
135+
{
136+
from test_socket_ssl
137+
Memcheck:Cond
138+
fun:BN_mod_exp_mont_word
139+
}
140+
141+
{
142+
from test_socket_ssl
143+
Memcheck:Cond
144+
fun:BN_mod_exp_mont
145+
}
146+
147+
{
148+
from test_socket_ssl
149+
Memcheck:Param
150+
write(buf)
151+
fun:write
152+
obj:/usr/lib/libcrypto.so.0.9.7
153+
}
154+
155+
{
156+
from test_socket_ssl
157+
Memcheck:Cond
158+
fun:RSA_verify
159+
}
160+
161+
{
162+
from test_socket_ssl
163+
Memcheck:Value4
164+
fun:RSA_verify
165+
}
166+
167+
{
168+
from test_socket_ssl
169+
Memcheck:Value4
170+
fun:DES_set_key_unchecked
171+
}
172+
173+
{
174+
from test_socket_ssl
175+
Memcheck:Value4
176+
fun:DES_encrypt2
177+
}
178+
179+
{
180+
from test_socket_ssl
181+
Memcheck:Cond
182+
obj:/usr/lib/libssl.so.0.9.7
183+
}
184+
185+
{
186+
from test_socket_ssl
187+
Memcheck:Value4
188+
obj:/usr/lib/libssl.so.0.9.7
189+
}
190+
191+
{
192+
from test_socket_ssl
193+
Memcheck:Cond
194+
fun:BUF_MEM_grow_clean
195+
}
196+
197+
{
198+
from test_socket_ssl
199+
Memcheck:Cond
200+
fun:memcpy
201+
fun:ssl3_read_bytes
202+
}
203+
204+
{
205+
from test_socket_ssl
206+
Memcheck:Cond
207+
fun:SHA1_Update
208+
}
209+
210+
{
211+
from test_socket_ssl
212+
Memcheck:Value4
213+
fun:SHA1_Update
214+
}
215+
216+

0 commit comments

Comments
 (0)