/
dist.tex
3795 lines (3095 loc) · 156 KB
/
dist.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\documentclass{manual}
\usepackage{distutils}
% $Id$
% TODO
% Document extension.read_setup_file
% Document build_clib command
%
\title{Distributing Python Modules}
\input{boilerplate}
\author{Greg Ward\\
Anthony Baxter}
\authoraddress{
\strong{Python Software Foundation}\\
Email: \email{distutils-sig@python.org}
}
\makeindex
\makemodindex
\begin{document}
\maketitle
\input{copyright}
\begin{abstract}
\noindent
This document describes the Python Distribution Utilities
(``Distutils'') from the module developer's point of view, describing
how to use the Distutils to make Python modules and extensions easily
available to a wider audience with very little overhead for
build/release/install mechanics.
\end{abstract}
% The ugly "%begin{latexonly}" pseudo-environment suppresses the table
% of contents for HTML generation.
%
%begin{latexonly}
\tableofcontents
%end{latexonly}
\chapter{An Introduction to Distutils}
\label{intro}
This document covers using the Distutils to distribute your Python
modules, concentrating on the role of developer/distributor: if
you're looking for information on installing Python modules, you
should refer to the \citetitle[../inst/inst.html]{Installing Python
Modules} manual.
\section{Concepts \& Terminology}
\label{concepts}
Using the Distutils is quite simple, both for module developers and for
users/administrators installing third-party modules. As a developer,
your responsibilities (apart from writing solid, well-documented and
well-tested code, of course!) are:
\begin{itemize}
\item write a setup script (\file{setup.py} by convention)
\item (optional) write a setup configuration file
\item create a source distribution
\item (optional) create one or more built (binary) distributions
\end{itemize}
Each of these tasks is covered in this document.
Not all module developers have access to a multitude of platforms, so
it's not always feasible to expect them to create a multitude of built
distributions. It is hoped that a class of intermediaries, called
\emph{packagers}, will arise to address this need. Packagers will take
source distributions released by module developers, build them on one or
more platforms, and release the resulting built distributions. Thus,
users on the most popular platforms will be able to install most popular
Python module distributions in the most natural way for their platform,
without having to run a single setup script or compile a line of code.
\section{A Simple Example}
\label{simple-example}
The setup script is usually quite simple, although since it's written
in Python, there are no arbitrary limits to what you can do with it,
though you should be careful about putting arbitrarily expensive
operations in your setup script. Unlike, say, Autoconf-style configure
scripts, the setup script may be run multiple times in the course of
building and installing your module distribution.
If all you want to do is distribute a module called \module{foo},
contained in a file \file{foo.py}, then your setup script can be as
simple as this:
\begin{verbatim}
from distutils.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'],
)
\end{verbatim}
Some observations:
\begin{itemize}
\item most information that you supply to the Distutils is supplied as
keyword arguments to the \function{setup()} function
\item those keyword arguments fall into two categories: package
metadata (name, version number) and information about what's in the
package (a list of pure Python modules, in this case)
\item modules are specified by module name, not filename (the same will
hold true for packages and extensions)
\item it's recommended that you supply a little more metadata, in
particular your name, email address and a URL for the project
(see section~\ref{setup-script} for an example)
\end{itemize}
To create a source distribution for this module, you would create a
setup script, \file{setup.py}, containing the above code, and run:
\begin{verbatim}
python setup.py sdist
\end{verbatim}
which will create an archive file (e.g., tarball on \UNIX, ZIP file on
Windows) containing your setup script \file{setup.py}, and your module
\file{foo.py}. The archive file will be named \file{foo-1.0.tar.gz} (or
\file{.zip}), and will unpack into a directory \file{foo-1.0}.
If an end-user wishes to install your \module{foo} module, all she has
to do is download \file{foo-1.0.tar.gz} (or \file{.zip}), unpack it,
and---from the \file{foo-1.0} directory---run
\begin{verbatim}
python setup.py install
\end{verbatim}
which will ultimately copy \file{foo.py} to the appropriate directory
for third-party modules in their Python installation.
This simple example demonstrates some fundamental concepts of the
Distutils. First, both developers and installers have the same basic
user interface, i.e. the setup script. The difference is which
Distutils \emph{commands} they use: the \command{sdist} command is
almost exclusively for module developers, while \command{install} is
more often for installers (although most developers will want to install
their own code occasionally).
If you want to make things really easy for your users, you can create
one or more built distributions for them. For instance, if you are
running on a Windows machine, and want to make things easy for other
Windows users, you can create an executable installer (the most
appropriate type of built distribution for this platform) with the
\command{bdist\_wininst} command. For example:
\begin{verbatim}
python setup.py bdist_wininst
\end{verbatim}
will create an executable installer, \file{foo-1.0.win32.exe}, in the
current directory.
Other useful built distribution formats are RPM, implemented by the
\command{bdist\_rpm} command, Solaris \program{pkgtool}
(\command{bdist\_pkgtool}), and HP-UX \program{swinstall}
(\command{bdist_sdux}). For example, the following command will
create an RPM file called \file{foo-1.0.noarch.rpm}:
\begin{verbatim}
python setup.py bdist_rpm
\end{verbatim}
(The \command{bdist\_rpm} command uses the \command{rpm} executable,
therefore this has to be run on an RPM-based system such as Red Hat
Linux, SuSE Linux, or Mandrake Linux.)
You can find out what distribution formats are available at any time by
running
\begin{verbatim}
python setup.py bdist --help-formats
\end{verbatim}
\section{General Python terminology}
\label{python-terms}
If you're reading this document, you probably have a good idea of what
modules, extensions, and so forth are. Nevertheless, just to be sure
that everyone is operating from a common starting point, we offer the
following glossary of common Python terms:
\begin{description}
\item[module] the basic unit of code reusability in Python: a block of
code imported by some other code. Three types of modules concern us
here: pure Python modules, extension modules, and packages.
\item[pure Python module] a module written in Python and contained in a
single \file{.py} file (and possibly associated \file{.pyc} and/or
\file{.pyo} files). Sometimes referred to as a ``pure module.''
\item[extension module] a module written in the low-level language of
the Python implementation: C/\Cpp{} for Python, Java for Jython.
Typically contained in a single dynamically loadable pre-compiled
file, e.g. a shared object (\file{.so}) file for Python extensions on
\UNIX, a DLL (given the \file{.pyd} extension) for Python extensions
on Windows, or a Java class file for Jython extensions. (Note that
currently, the Distutils only handles C/\Cpp{} extensions for Python.)
\item[package] a module that contains other modules; typically contained
in a directory in the filesystem and distinguished from other
directories by the presence of a file \file{\_\_init\_\_.py}.
\item[root package] the root of the hierarchy of packages. (This isn't
really a package, since it doesn't have an \file{\_\_init\_\_.py}
file. But we have to call it something.) The vast majority of the
standard library is in the root package, as are many small, standalone
third-party modules that don't belong to a larger module collection.
Unlike regular packages, modules in the root package can be found in
many directories: in fact, every directory listed in \code{sys.path}
contributes modules to the root package.
\end{description}
\section{Distutils-specific terminology}
\label{distutils-term}
The following terms apply more specifically to the domain of
distributing Python modules using the Distutils:
\begin{description}
\item[module distribution] a collection of Python modules distributed
together as a single downloadable resource and meant to be installed
\emph{en masse}. Examples of some well-known module distributions are
Numeric Python, PyXML, PIL (the Python Imaging Library), or
mxBase. (This would be called a \emph{package}, except that term
is already taken in the Python context: a single module distribution
may contain zero, one, or many Python packages.)
\item[pure module distribution] a module distribution that contains only
pure Python modules and packages. Sometimes referred to as a ``pure
distribution.''
\item[non-pure module distribution] a module distribution that contains
at least one extension module. Sometimes referred to as a ``non-pure
distribution.''
\item[distribution root] the top-level directory of your source tree (or
source distribution); the directory where \file{setup.py} exists. Generally
\file{setup.py} will be run from this directory.
\end{description}
\chapter{Writing the Setup Script}
\label{setup-script}
The setup script is the centre of all activity in building,
distributing, and installing modules using the Distutils. The main
purpose of the setup script is to describe your module distribution to
the Distutils, so that the various commands that operate on your modules
do the right thing. As we saw in section~\ref{simple-example} above,
the setup script consists mainly of a call to \function{setup()}, and
most information supplied to the Distutils by the module developer is
supplied as keyword arguments to \function{setup()}.
Here's a slightly more involved example, which we'll follow for the next
couple of sections: the Distutils' own setup script. (Keep in mind that
although the Distutils are included with Python 1.6 and later, they also
have an independent existence so that Python 1.5.2 users can use them to
install other module distributions. The Distutils' own setup script,
shown here, is used to install the package into Python 1.5.2.)
\begin{verbatim}
#!/usr/bin/env python
from distutils.core import setup
setup(name='Distutils',
version='1.0',
description='Python Distribution Utilities',
author='Greg Ward',
author_email='gward@python.net',
url='http://www.python.org/sigs/distutils-sig/',
packages=['distutils', 'distutils.command'],
)
\end{verbatim}
There are only two differences between this and the trivial one-file
distribution presented in section~\ref{simple-example}: more
metadata, and the specification of pure Python modules by package,
rather than by module. This is important since the Distutils consist of
a couple of dozen modules split into (so far) two packages; an explicit
list of every module would be tedious to generate and difficult to
maintain. For more information on the additional meta-data, see
section~\ref{meta-data}.
Note that any pathnames (files or directories) supplied in the setup
script should be written using the \UNIX{} convention, i.e.
slash-separated. The Distutils will take care of converting this
platform-neutral representation into whatever is appropriate on your
current platform before actually using the pathname. This makes your
setup script portable across operating systems, which of course is one
of the major goals of the Distutils. In this spirit, all pathnames in
this document are slash-separated. (Mac OS 9 programmers should keep in
mind that the \emph{absence} of a leading slash indicates a relative
path, the opposite of the Mac OS convention with colons.)
This, of course, only applies to pathnames given to Distutils
functions. If you, for example, use standard Python functions such as
\function{glob.glob()} or \function{os.listdir()} to specify files, you
should be careful to write portable code instead of hardcoding path
separators:
\begin{verbatim}
glob.glob(os.path.join('mydir', 'subdir', '*.html'))
os.listdir(os.path.join('mydir', 'subdir'))
\end{verbatim}
\section{Listing whole packages}
\label{listing-packages}
The \option{packages} option tells the Distutils to process (build,
distribute, install, etc.) all pure Python modules found in each package
mentioned in the \option{packages} list. In order to do this, of
course, there has to be a correspondence between package names and
directories in the filesystem. The default correspondence is the most
obvious one, i.e. package \module{distutils} is found in the directory
\file{distutils} relative to the distribution root. Thus, when you say
\code{packages = ['foo']} in your setup script, you are promising that
the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might
be spelled differently on your system, but you get the idea) relative to
the directory where your setup script lives. If you break this
promise, the Distutils will issue a warning but still process the broken
package anyways.
If you use a different convention to lay out your source directory,
that's no problem: you just have to supply the \option{package\_dir}
option to tell the Distutils about your convention. For example, say
you keep all Python source under \file{lib}, so that modules in the
``root package'' (i.e., not in any package at all) are in
\file{lib}, modules in the \module{foo} package are in \file{lib/foo},
and so forth. Then you would put
\begin{verbatim}
package_dir = {'': 'lib'}
\end{verbatim}
in your setup script. The keys to this dictionary are package names,
and an empty package name stands for the root package. The values are
directory names relative to your distribution root. In this case, when
you say \code{packages = ['foo']}, you are promising that the file
\file{lib/foo/\_\_init\_\_.py} exists.
Another possible convention is to put the \module{foo} package right in
\file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc. This
would be written in the setup script as
\begin{verbatim}
package_dir = {'foo': 'lib'}
\end{verbatim}
A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}
dictionary implicitly applies to all packages below \var{package}, so
the \module{foo.bar} case is automatically handled here. In this
example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils
to look for \file{lib/\_\_init\_\_.py} and
\file{lib/bar/\_\_init\_\_.py}. (Keep in mind that although
\option{package\_dir} applies recursively, you must explicitly list all
packages in \option{packages}: the Distutils will \emph{not} recursively
scan your source tree looking for any directory with an
\file{\_\_init\_\_.py} file.)
\section{Listing individual modules}
\label{listing-modules}
For a small module distribution, you might prefer to list all modules
rather than listing packages---especially the case of a single module
that goes in the ``root package'' (i.e., no package at all). This
simplest case was shown in section~\ref{simple-example}; here is a
slightly more involved example:
\begin{verbatim}
py_modules = ['mod1', 'pkg.mod2']
\end{verbatim}
This describes two modules, one of them in the ``root'' package, the
other in the \module{pkg} package. Again, the default package/directory
layout implies that these two modules can be found in \file{mod1.py} and
\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.
And again, you can override the package/directory correspondence using
the \option{package\_dir} option.
\section{Describing extension modules}
\label{describing-extensions}
% XXX read over this section
Just as writing Python extension modules is a bit more complicated than
writing pure Python modules, describing them to the Distutils is a bit
more complicated. Unlike pure modules, it's not enough just to list
modules or packages and expect the Distutils to go out and find the
right files; you have to specify the extension name, source file(s), and
any compile/link requirements (include directories, libraries to link
with, etc.).
All of this is done through another keyword argument to
\function{setup()}, the \option{ext_modules} option. \option{ext_modules}
is just a list of \class{Extension} instances, each of which describes a
single extension module. Suppose your distribution includes a single
extension, called \module{foo} and implemented by \file{foo.c}. If no
additional instructions to the compiler/linker are needed, describing
this extension is quite simple:
\begin{verbatim}
Extension('foo', ['foo.c'])
\end{verbatim}
The \class{Extension} class can be imported from
\module{distutils.core} along with \function{setup()}. Thus, the setup
script for a module distribution that contains only this one extension
and nothing else might be:
\begin{verbatim}
from distutils.core import setup, Extension
setup(name='foo',
version='1.0',
ext_modules=[Extension('foo', ['foo.c'])],
)
\end{verbatim}
The \class{Extension} class (actually, the underlying extension-building
machinery implemented by the \command{build\_ext} command) supports a
great deal of flexibility in describing Python extensions, which is
explained in the following sections.
\subsection{Extension names and packages}
The first argument to the \class{Extension} constructor is always the
name of the extension, including any package names. For example,
\begin{verbatim}
Extension('foo', ['src/foo1.c', 'src/foo2.c'])
\end{verbatim}
describes an extension that lives in the root package, while
\begin{verbatim}
Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])
\end{verbatim}
describes the same extension in the \module{pkg} package. The source
files and resulting object code are identical in both cases; the only
difference is where in the filesystem (and therefore where in Python's
namespace hierarchy) the resulting extension lives.
If you have a number of extensions all in the same package (or all under
the same base package), use the \option{ext\_package} keyword argument
to \function{setup()}. For example,
\begin{verbatim}
setup(...
ext_package='pkg',
ext_modules=[Extension('foo', ['foo.c']),
Extension('subpkg.bar', ['bar.c'])],
)
\end{verbatim}
will compile \file{foo.c} to the extension \module{pkg.foo}, and
\file{bar.c} to \module{pkg.subpkg.bar}.
\subsection{Extension source files}
The second argument to the \class{Extension} constructor is a list of
source files. Since the Distutils currently only support C, \Cpp, and
Objective-C extensions, these are normally C/\Cpp/Objective-C source
files. (Be sure to use appropriate extensions to distinguish \Cpp\
source files: \file{.cc} and \file{.cpp} seem to be recognized by both
\UNIX{} and Windows compilers.)
However, you can also include SWIG interface (\file{.i}) files in the
list; the \command{build\_ext} command knows how to deal with SWIG
extensions: it will run SWIG on the interface file and compile the
resulting C/\Cpp{} file into your extension.
\XXX{SWIG support is rough around the edges and largely untested;
especially SWIG support for \Cpp{} extensions! Explain in more detail
here when the interface firms up.}
On some platforms, you can include non-source files that are processed
by the compiler and included in your extension. Currently, this just
means Windows message text (\file{.mc}) files and resource definition
(\file{.rc}) files for Visual \Cpp. These will be compiled to binary resource
(\file{.res}) files and linked into the executable.
\subsection{Preprocessor options}
Three optional arguments to \class{Extension} will help if you need to
specify include directories to search or preprocessor macros to
define/undefine: \code{include\_dirs}, \code{define\_macros}, and
\code{undef\_macros}.
For example, if your extension requires header files in the
\file{include} directory under your distribution root, use the
\code{include\_dirs} option:
\begin{verbatim}
Extension('foo', ['foo.c'], include_dirs=['include'])
\end{verbatim}
You can specify absolute directories there; if you know that your
extension will only be built on \UNIX{} systems with X11R6 installed to
\file{/usr}, you can get away with
\begin{verbatim}
Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11'])
\end{verbatim}
You should avoid this sort of non-portable usage if you plan to
distribute your code: it's probably better to write C code like
\begin{verbatim}
#include <X11/Xlib.h>
\end{verbatim}
If you need to include header files from some other Python extension,
you can take advantage of the fact that header files are installed in a
consistent way by the Distutils \command{install\_header} command. For
example, the Numerical Python header files are installed (on a standard
\UNIX{} installation) to \file{/usr/local/include/python1.5/Numerical}.
(The exact location will differ according to your platform and Python
installation.) Since the Python include
directory---\file{/usr/local/include/python1.5} in this case---is always
included in the search path when building Python extensions, the best
approach is to write C code like
\begin{verbatim}
#include <Numerical/arrayobject.h>
\end{verbatim}
If you must put the \file{Numerical} include directory right into your
header search path, though, you can find that directory using the
Distutils \refmodule{distutils.sysconfig} module:
\begin{verbatim}
from distutils.sysconfig import get_python_inc
incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical')
setup(...,
Extension(..., include_dirs=[incdir]),
)
\end{verbatim}
Even though this is quite portable---it will work on any Python
installation, regardless of platform---it's probably easier to just
write your C code in the sensible way.
You can define and undefine pre-processor macros with the
\code{define\_macros} and \code{undef\_macros} options.
\code{define\_macros} takes a list of \code{(name, value)} tuples, where
\code{name} is the name of the macro to define (a string) and
\code{value} is its value: either a string or \code{None}. (Defining a
macro \code{FOO} to \code{None} is the equivalent of a bare
\code{\#define FOO} in your C source: with most compilers, this sets
\code{FOO} to the string \code{1}.) \code{undef\_macros} is just
a list of macros to undefine.
For example:
\begin{verbatim}
Extension(...,
define_macros=[('NDEBUG', '1'),
('HAVE_STRFTIME', None)],
undef_macros=['HAVE_FOO', 'HAVE_BAR'])
\end{verbatim}
is the equivalent of having this at the top of every C source file:
\begin{verbatim}
#define NDEBUG 1
#define HAVE_STRFTIME
#undef HAVE_FOO
#undef HAVE_BAR
\end{verbatim}
\subsection{Library options}
You can also specify the libraries to link against when building your
extension, and the directories to search for those libraries. The
\code{libraries} option is a list of libraries to link against,
\code{library\_dirs} is a list of directories to search for libraries at
link-time, and \code{runtime\_library\_dirs} is a list of directories to
search for shared (dynamically loaded) libraries at run-time.
For example, if you need to link against libraries known to be in the
standard library search path on target systems
\begin{verbatim}
Extension(...,
libraries=['gdbm', 'readline'])
\end{verbatim}
If you need to link with libraries in a non-standard location, you'll
have to include the location in \code{library\_dirs}:
\begin{verbatim}
Extension(...,
library_dirs=['/usr/X11R6/lib'],
libraries=['X11', 'Xt'])
\end{verbatim}
(Again, this sort of non-portable construct should be avoided if you
intend to distribute your code.)
\XXX{Should mention clib libraries here or somewhere else!}
\subsection{Other options}
There are still some other options which can be used to handle special
cases.
The \option{extra\_objects} option is a list of object files to be passed
to the linker. These files must not have extensions, as the default
extension for the compiler is used.
\option{extra\_compile\_args} and \option{extra\_link\_args} can be used
to specify additional command line options for the respective compiler and
linker command lines.
\option{export\_symbols} is only useful on Windows. It can contain a list
of symbols (functions or variables) to be exported. This option
is not needed when building compiled extensions: Distutils
will automatically add \code{initmodule}
to the list of exported symbols.
\section{Relationships between Distributions and Packages}
A distribution may relate to packages in three specific ways:
\begin{enumerate}
\item It can require packages or modules.
\item It can provide packages or modules.
\item It can obsolete packages or modules.
\end{enumerate}
These relationships can be specified using keyword arguments to the
\function{distutils.core.setup()} function.
Dependencies on other Python modules and packages can be specified by
supplying the \var{requires} keyword argument to \function{setup()}.
The value must be a list of strings. Each string specifies a package
that is required, and optionally what versions are sufficient.
To specify that any version of a module or package is required, the
string should consist entirely of the module or package name.
Examples include \code{'mymodule'} and \code{'xml.parsers.expat'}.
If specific versions are required, a sequence of qualifiers can be
supplied in parentheses. Each qualifier may consist of a comparison
operator and a version number. The accepted comparison operators are:
\begin{verbatim}
< > ==
<= >= !=
\end{verbatim}
These can be combined by using multiple qualifiers separated by commas
(and optional whitespace). In this case, all of the qualifiers must
be matched; a logical AND is used to combine the evaluations.
Let's look at a bunch of examples:
\begin{tableii}{l|l}{code}{Requires Expression}{Explanation}
\lineii{==1.0} {Only version \code{1.0} is compatible}
\lineii{>1.0, !=1.5.1, <2.0} {Any version after \code{1.0} and before
\code{2.0} is compatible, except
\code{1.5.1}}
\end{tableii}
Now that we can specify dependencies, we also need to be able to
specify what we provide that other distributions can require. This is
done using the \var{provides} keyword argument to \function{setup()}.
The value for this keyword is a list of strings, each of which names a
Python module or package, and optionally identifies the version. If
the version is not specified, it is assumed to match that of the
distribution.
Some examples:
\begin{tableii}{l|l}{code}{Provides Expression}{Explanation}
\lineii{mypkg} {Provide \code{mypkg}, using the distribution version}
\lineii{mypkg (1.1} {Provide \code{mypkg} version 1.1, regardless of the
distribution version}
\end{tableii}
A package can declare that it obsoletes other packages using the
\var{obsoletes} keyword argument. The value for this is similar to
that of the \var{requires} keyword: a list of strings giving module or
package specifiers. Each specifier consists of a module or package
name optionally followed by one or more version qualifiers. Version
qualifiers are given in parentheses after the module or package name.
The versions identified by the qualifiers are those that are obsoleted
by the distribution being described. If no qualifiers are given, all
versions of the named module or package are understood to be
obsoleted.
\section{Installing Scripts}
So far we have been dealing with pure and non-pure Python modules,
which are usually not run by themselves but imported by scripts.
Scripts are files containing Python source code, intended to be
started from the command line. Scripts don't require Distutils to do
anything very complicated. The only clever feature is that if the
first line of the script starts with \code{\#!} and contains the word
``python'', the Distutils will adjust the first line to refer to the
current interpreter location. By default, it is replaced with the
current interpreter location. The \longprogramopt{executable} (or
\programopt{-e}) option will allow the interpreter path to be
explicitly overridden.
The \option{scripts} option simply is a list of files to be handled
in this way. From the PyXML setup script:
\begin{verbatim}
setup(...
scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val']
)
\end{verbatim}
\section{Installing Package Data}
Often, additional files need to be installed into a package. These
files are often data that's closely related to the package's
implementation, or text files containing documentation that might be
of interest to programmers using the package. These files are called
\dfn{package data}.
Package data can be added to packages using the \code{package_data}
keyword argument to the \function{setup()} function. The value must
be a mapping from package name to a list of relative path names that
should be copied into the package. The paths are interpreted as
relative to the directory containing the package (information from the
\code{package_dir} mapping is used if appropriate); that is, the files
are expected to be part of the package in the source directories.
They may contain glob patterns as well.
The path names may contain directory portions; any necessary
directories will be created in the installation.
For example, if a package should contain a subdirectory with several
data files, the files can be arranged like this in the source tree:
\begin{verbatim}
setup.py
src/
mypkg/
__init__.py
module.py
data/
tables.dat
spoons.dat
forks.dat
\end{verbatim}
The corresponding call to \function{setup()} might be:
\begin{verbatim}
setup(...,
packages=['mypkg'],
package_dir={'mypkg': 'src/mypkg'},
package_data={'mypkg': ['data/*.dat']},
)
\end{verbatim}
\versionadded{2.4}
\section{Installing Additional Files}
The \option{data\_files} option can be used to specify additional
files needed by the module distribution: configuration files, message
catalogs, data files, anything which doesn't fit in the previous
categories.
\option{data\_files} specifies a sequence of (\var{directory},
\var{files}) pairs in the following way:
\begin{verbatim}
setup(...
data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
('config', ['cfg/data.cfg']),
('/etc/init.d', ['init-script'])]
)
\end{verbatim}
Note that you can specify the directory names where the data files
will be installed, but you cannot rename the data files themselves.
Each (\var{directory}, \var{files}) pair in the sequence specifies the
installation directory and the files to install there. If
\var{directory} is a relative path, it is interpreted relative to the
installation prefix (Python's \code{sys.prefix} for pure-Python
packages, \code{sys.exec_prefix} for packages that contain extension
modules). Each file name in \var{files} is interpreted relative to
the \file{setup.py} script at the top of the package source
distribution. No directory information from \var{files} is used to
determine the final location of the installed file; only the name of
the file is used.
You can specify the \option{data\_files} options as a simple sequence
of files without specifying a target directory, but this is not recommended,
and the \command{install} command will print a warning in this case.
To install data files directly in the target directory, an empty
string should be given as the directory.
\section{Additional meta-data}
\label{meta-data}
The setup script may include additional meta-data beyond the name and
version. This information includes:
\begin{tableiv}{l|l|l|c}{code}%
{Meta-Data}{Description}{Value}{Notes}
\lineiv{name}{name of the package}
{short string}{(1)}
\lineiv{version}{version of this release}
{short string}{(1)(2)}
\lineiv{author}{package author's name}
{short string}{(3)}
\lineiv{author_email}{email address of the package author}
{email address}{(3)}
\lineiv{maintainer}{package maintainer's name}
{short string}{(3)}
\lineiv{maintainer_email}{email address of the package maintainer}
{email address}{(3)}
\lineiv{url}{home page for the package}
{URL}{(1)}
\lineiv{description}{short, summary description of the package}
{short string}{}
\lineiv{long_description}{longer description of the package}
{long string}{}
\lineiv{download_url}{location where the package may be downloaded}
{URL}{(4)}
\lineiv{classifiers}{a list of classifiers}
{list of strings}{(4)}
\end{tableiv}
\noindent Notes:
\begin{description}
\item[(1)] These fields are required.
\item[(2)] It is recommended that versions take the form
\emph{major.minor\optional{.patch\optional{.sub}}}.
\item[(3)] Either the author or the maintainer must be identified.
\item[(4)] These fields should not be used if your package is to be
compatible with Python versions prior to 2.2.3 or 2.3. The list is
available from the \ulink{PyPI website}{http://www.python.org/pypi}.
\item['short string'] A single line of text, not more than 200 characters.
\item['long string'] Multiple lines of plain text in reStructuredText
format (see \url{http://docutils.sf.net/}).
\item['list of strings'] See below.
\end{description}
None of the string values may be Unicode.
Encoding the version information is an art in itself. Python packages
generally adhere to the version format
\emph{major.minor\optional{.patch}\optional{sub}}. The major number is
0 for
initial, experimental releases of software. It is incremented for
releases that represent major milestones in a package. The minor
number is incremented when important new features are added to the
package. The patch number increments when bug-fix releases are
made. Additional trailing version information is sometimes used to
indicate sub-releases. These are "a1,a2,...,aN" (for alpha releases,
where functionality and API may change), "b1,b2,...,bN" (for beta
releases, which only fix bugs) and "pr1,pr2,...,prN" (for final
pre-release release testing). Some examples:
\begin{description}
\item[0.1.0] the first, experimental release of a package
\item[1.0.1a2] the second alpha release of the first patch version of 1.0
\end{description}
\option{classifiers} are specified in a python list:
\begin{verbatim}
setup(...
classifiers=[
'Development Status :: 4 - Beta',
'Environment :: Console',
'Environment :: Web Environment',
'Intended Audience :: End Users/Desktop',
'Intended Audience :: Developers',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: Python Software Foundation License',
'Operating System :: MacOS :: MacOS X',
'Operating System :: Microsoft :: Windows',
'Operating System :: POSIX',
'Programming Language :: Python',
'Topic :: Communications :: Email',
'Topic :: Office/Business',
'Topic :: Software Development :: Bug Tracking',
],
)
\end{verbatim}
If you wish to include classifiers in your \file{setup.py} file and also
wish to remain backwards-compatible with Python releases prior to 2.2.3,
then you can include the following code fragment in your \file{setup.py}
before the \function{setup()} call.
\begin{verbatim}
# patch distutils if it can't cope with the "classifiers" or
# "download_url" keywords
from sys import version
if version < '2.2.3':
from distutils.dist import DistributionMetadata
DistributionMetadata.classifiers = None
DistributionMetadata.download_url = None
\end{verbatim}
\section{Debugging the setup script}
Sometimes things go wrong, and the setup script doesn't do what the
developer wants.
Distutils catches any exceptions when running the setup script, and
print a simple error message before the script is terminated. The
motivation for this behaviour is to not confuse administrators who
don't know much about Python and are trying to install a package. If
they get a big long traceback from deep inside the guts of Distutils,
they may think the package or the Python installation is broken
because they don't read all the way down to the bottom and see that
it's a permission problem.
On the other hand, this doesn't help the developer to find the cause
of the failure. For this purpose, the DISTUTILS_DEBUG environment
variable can be set to anything except an empty string, and distutils
will now print detailed information what it is doing, and prints the
full traceback in case an exception occurs.
\chapter{Writing the Setup Configuration File}
\label{setup-config}
Often, it's not possible to write down everything needed to build a
distribution \emph{a priori}: you may need to get some information from
the user, or from the user's system, in order to proceed. As long as
that information is fairly simple---a list of directories to search for
C header files or libraries, for example---then providing a
configuration file, \file{setup.cfg}, for users to edit is a cheap and
easy way to solicit it. Configuration files also let you provide
default values for any command option, which the installer can then
override either on the command-line or by editing the config file.
% (If you have more advanced needs, such as determining which extensions
% to build based on what capabilities are present on the target system,
% then you need the Distutils ``auto-configuration'' facility. This
% started to appear in Distutils 0.9 but, as of this writing, isn't mature
% or stable enough yet for real-world use.)
The setup configuration file is a useful middle-ground between the setup
script---which, ideally, would be opaque to installers\footnote{This
ideal probably won't be achieved until auto-configuration is fully
supported by the Distutils.}---and the command-line to the setup
script, which is outside of your control and entirely up to the
installer. In fact, \file{setup.cfg} (and any other Distutils
configuration files present on the target system) are processed after
the contents of the setup script, but before the command-line. This has
several useful consequences:
\begin{itemize}
\item installers can override some of what you put in \file{setup.py} by
editing \file{setup.cfg}
\item you can provide non-standard defaults for options that are not
easily set in \file{setup.py}
\item installers can override anything in \file{setup.cfg} using the
command-line options to \file{setup.py}
\end{itemize}
The basic syntax of the configuration file is simple:
\begin{verbatim}
[command]
option=value
...
\end{verbatim}
where \var{command} is one of the Distutils commands (e.g.
\command{build\_py}, \command{install}), and \var{option} is one of
the options that command supports. Any number of options can be
supplied for each command, and any number of command sections can be