forked from ocaml/opam
/
roadmap.tex
1457 lines (1126 loc) · 50.1 KB
/
roadmap.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\documentclass[a4paper,11pt]{article}
\usepackage{url}
\usepackage{eurosym}
%\usepackage[french]{babel}
\usepackage[T1]{fontenc}
\usepackage{pdfswitch}
\usepackage{verbatim}
\usepackage{fullpage}
\usepackage{fancyvrb}
\newenvironment{remark}[1][Remark]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\title{
OPAM: a Package Management Systems for OCaml\\
Version 1.0.0 Roadmap\\ ~\ \\
THIS DOCUMENT IS A DRAFT\\
~\ \\}
\author{Thomas GAZAGNAIRE\\
\url{thomas.gazagnaire@ocamlpro.com}\\
}
\begin{document}
\maketitle
\vfill
\tableofcontents
\section*{Overview}
This document specifies the design of a package management system for
OCaml (OPAM). For the first version of OPAM, we have tried to consider
the simplest design choices, even if these choices restrict user
possibilities (but we hope not too much). Our goal is to propose a
system that we can build in a few months. Some of the design choices
might evolve to more complex tasks later, if needed. \\
A package management system has typically two kinds of users: {\em
end-users} who install and use packages for their own projects; and
{\em packagers}, who create and upload packages. End-users want to
install on their machine a consistent collection of {\em packages} --
a package being a collection of OCaml libraries and/or programs.
Packagers want to take a collection of their own libraries and
programs and make them available to other developpers.
This document describes the fonctional requirements for both kinds of
users.
\subsection*{Conventions}
In this document, \verb+$home+, \verb+$opam+, \verb+$package+ and
\verb+$path+ are assumed to be defined as follows:
\begin{itemize}
\item {\tt \$home} refers to the end-user home path, typically {\tt
/home/thomas/} on linux, {\tt /Users/thomas/} on OSX {\tt
C:\textbackslash Documents and Settings\textbackslash
thomas\textbackslash} on Windows.
\item {\tt \$opam} refers to the filesystem subtree containing the
client state. Default directory is {\tt \$home/.opam}.
\item {\tt \$package} refers to a path in the packager filesystem, where
lives the collection of libraries and programs he wants to package.
\item {\tt \$path} refers to a list of paths in the packager filesystem, where
lives the collection of programs ({\tt ocamlc}, {\tt ocamldep}, {\tt ocamlopt},
{\tt ocamlbuild}, ...).
\end{itemize}
User variables are written in capital letters, prefixed by \verb+$+. For
instance package names will be written \verb+$NAME+, package versions
\verb+$VERSION+, and the version of the ocaml compiler currently
installed \verb+$OVERSION+.
\section{Milestone 1: Foundations}
The first milestone of OPAM focuses on providing a limited set of
features, dedicated to package management of OCaml packages. OPAM rely
on external tools to compile and provide full configuration options to
the build tools. The goal for this first milestone is to be as much as
possible compatible with any existing build system (including {\tt
ocamlfind} and {\tt oasis}) modulo few modifications.
\subsection{Client state}
\label{client}
The client state is stored on the filesystem, under {\tt \$opam}:
\subsubsection{Global state}
\label{state-global}
\begin{itemize}
\item {\tt \$opam/config} is the main configuration file. It defines
the OPAM version, the repository addresses and the current compiler
version. The file format is described in \S\ref{config}.
\item \verb+$opam/opam/$NAME.$VERSION.opam+ is the OPAM specification
for the package \verb+$NAME+ with version \verb+$VERSION+ (which
might not be installed). The format of OPAM files is described in
\S\ref{dotopam}.
\item \verb+$opam/descr/$NAME.$VERSION+ contains the description for the
version \verb+$VERSION+ of package \verb+$NAME+ (which might not be
installed). The first line of this file is the package synopsis.
\item \verb+$opam/archives/$NAME.$VERSION.tar.gz+ contains the source
archives for the version \verb+$VERSION+ of package \verb+$NAME+.
\item \verb+$opam/repo/index+ contains the location of packages
(ie. in which repositories they can be found and the priority
between repositories). The file format is described in
\ref{index}.
\end{itemize}
\subsubsection{Compiler-specific state}
All the configurations files, libraries and binaries related to the
specific \verb+$OVERSION+ of the OCaml compiler are stored in
\verb+$opam/$OVERSION+.
\begin{itemize}
\item \verb+$opam/$OVERSION/installed+ is the list of installed
packages for the compiler version \verb+$OVERSION+. The file format
is described in \S\ref{installed}.
\item \verb+$opam/$OVERSION/config/$NAME.config+ is a
platform-specific configuration file of for the installed package
\verb+$NAME+ with the compiler version \verb+$OVERSION+. The file
format is described in \S\ref{dotconfig}.
\verb+$opam/$OVERSION/config/+ can be shortened to \verb+$config/+
for more readability.
\item \verb+$opam/$OVERSION/install/$NAME.install+ is a
platform-specific package installation file for the installed
package \verb+$NAME+ with the compiler version \verb+$OVERSION+. The
file format is described in \S\ref{dotinstall}.
\verb+$opam/$OVERSION/install+ can be shortened to \verb+$install/+
for more readability.
\item \verb+$opam/$OVERSION/lib/$NAME/+ contains the libraries
associated to the installed package \verb+$NAME+ with the compiler
version \verb+$OVERSION+. \verb+$opam/$OVERSION/lib/+ can be
shortened to \verb+$lib/+ for more readability.
\item \verb+$opam/$OVERSION/doc/$NAME/+ contains the documentation
associated to the installed package {\tt NAME} with the compiler
version \verb+$OVERSION+. \verb+$opam/OVERSION/doc/+ can be
shortened to \verb+$doc/+ for more readability.
\item \verb+$opam/$OVERSION/bin/+ contains the program files for all
installed packages with the compiler version
\verb+$OVERSION+. \verb+$opam/$OVERSION/bin/+ can be shortened to
\verb+$bin/+ for more readability.
\item \verb+$opam/$OVERSION/build/$NAME.$VERSION/+ is a tempory folder
used to build package \verb+$NAME+ with version \verb+$VERSION+,
with compiler version \verb+$OVERSION+. \verb+$opam/$OVERSION/build/+
can be shortened to \verb+$build/+ for more readability.
\item \verb+$opam/$OVERSION/reinstall+ contains the list of packages
which has been changed upstream since the last upgrade. This can
happen for instance when a packager uploads a new archive or fix the
OPAM file for a specific package version. Every package appearing in
this file will be reinstalled (or upgraded if a new version is
available) during the next upgrade when the current version of the
compiler is \verb+$OVERSION+. The file format is similar to the one
described in \S\ref{installed}.
\end{itemize}
\subsubsection{Repository-specific state}
\label{state-repo}
Configuration files for OPAM repositories \verb+REPO+ are stored in
\verb+$opam/repo/$REPO+. Repositories can be of different kinds
(stored on the local filesystem, available via HTTP, available using a
custom binary protocol, stored under git, $\ldots$); they all share
the same base filesystem which is initialized using the {\tt
opam-<kind>-init} script (see \S\ref{script-init}) and use only the
{\tt opam-<kind>-update} (see \S\ref{script-update}) and {\tt
opam-<kind>-upload} (see \S\ref{script-upload}) scripts to exchange
data between the client and the corresponding OPAM repository.
\begin{itemize}
\item \verb+$opam/repo/$REPO/kind+ contains the kind associated to the
OPAM repository \verb+$REPO+ have. The kind is stored as a single
word (containing only letters and digits) and specifies which {\tt
opam-<kind>-*} scripts to call when updating and uploading this
repository.
\item \verb+$opam/repo/$REPO/address+ contains the address of the
OPAM repository \verb+$REPO+. This address is passed as argument to
the {\tt opam-<kind>-*} scripts.
\item \verb+$opam/repo/$REPO/opam/$NAME.$VERSION.opam+ is the OPAM
specification for the package \verb+$NAME+ with version
\verb+$VERSION+ (which might not be installed). The format of OPAM
files is described in \S\ref{dotopam}.
\item \verb+$opam/repo/$REPO/descr/$NAME.$VERSION+ contains the textual
description for the version \verb+$VERSION+ of package \verb+$NAME+
(which might not be installed). The first line of this file is the
package synopsis.
\item \verb+$opam/repo/$REPO/archives/$NAME.$VERSION.tar.gz+ contains
the source archives for the version \verb+$VERSION+ of package
\verb+$NAME+. This folder is populated by calling the corresponding
{\tt opam-<kind>-download} script (see \S\ref{script-download}).
\item \verb+$opam/repo/$REPO/updated+ contains the new available
packages which have not yet been synchronized with the client
state. This file is created by the {\tt opam-<kind>-update} script
(see \S\ref{script-update}). If the file empty, this means that the
client state is up-to-date. The file format is the same as the one
described in \S\ref{installed}.
\item \verb+$opam/repo/$REPO/upload/$NAME.$VERSION/+ contains the
OPAM, description and archive files to upload to the OPAM repository
for the version \verb+$VERSION+ of package \verb+$NAME+. The script
{\tt opam-<kind>-update} script (see \S\ref{script-upload}) read the
contents of {\tt upload/} and send it to the repository (if the
repository support upload).
\end{itemize}
\subsection{File syntax}
\subsubsection{List of packages}
\label{installed}
The following configuration files: \verb+$opam/$OVERSION/installed+,
\verb+$opam/$OVERSION/reinstall+, and \verb+$opam/repo/$REPO/updated+
follow a very simple syntax. The file is a list of lines which
contains a space-separated name and a version. Each line
\verb+$NAME $VERSION+ means that the version \verb+$VERSION+ of
package \verb+$NAME+ has been compiled with OCaml version
\verb+$OVERSION+ and has been installed on the system in
\verb+$lib/$NAME+ and \verb+$bin/+. \\
For instance, if {\tt batteries} version {\tt 1.0+beta} and {\tt
ocamlfind} version {\tt 1.2} are installed, then
\verb+$opam/$OVERSION/installed+ will contain:
{\small
\begin{Verbatim}[frame=single]
batteries 1.0+beta
ocamlfind 1.2
\end{Verbatim}
}
\subsubsection{Index of packages}
\label{index}
\verb+$opam/repo/index+ follows a very simple syntax: each line of the
file contains a space separated list of words \verb+$NAME $REPO+
specifying that all the versions of package \verb+$NAME+ are available
in the OPAM repository \verb+$REPO+. The file contains information on
all available packages (e.g. not only on the installed one). \\
For instance, if {\tt batteries} version {\tt 1.0+beta} is available
in the {\tt testing} repository and {\tt ocamlfind} version {\tt 1.2}
is available in the {\tt default} and testing repositories (where {\tt
default} is one being used), then \verb+$opam/repo/index+ will
contain:
{\small
\begin{Verbatim}[frame=single]
batteries testing
ocamlfind default
\end{Verbatim}
}
\subsubsection{General syntax}
\label{syntax}
Most of the files in the client and server states share the same
syntax defined in this section.
\begin{description}
\item[Base types] The base types for values are:
\begin{itemize}
\item {\tt BOOL} is either {\tt true} or {\tt false}
\item {\tt STRING} is a doubly-quoted OCaml string, for instance: {\tt
"foo"}, {\tt "foo-bar"}, $\ldots$
\item {\tt SYMBOL} contains only non-letter and non-digit characters,
for instance: {\tt =}, {\tt <=}, $\ldots$ Some symbols have a special
meaning and thus are not valid {\tt SYMBOL}s: ``\verb+(+ \verb+)+
\verb+[+ \verb+]+ \verb+{+ \verb+}+ \verb+:+''.
\item {\tt IDENT} starts by a letter and is followed by any number of
letters, digit and symbols, for instance: {\tt foo}, {\tt foo-bar},
$\ldots$.
\end{itemize}
\item[Compound types] Types can be composed together to build more
complex values:
\begin{itemize}
\item {\tt X Y } is a space-separated pair of value.
\item {\tt X | Y } is a value of type either {\tt X} or {\tt Y}.
\item {\tt ?X} is zero or one occurrence of a value of type {\tt X}.
\item {\tt X+} is a space-separated list of values of at least one value
of type {\tt X}.
\item {\tt X*} is a space-separated list of values of values of type
{\tt X} (it might contain no value).
\end{itemize}
\end{description}
All structured OPAM files share the same syntax:
{\small
\begin{Verbatim}[frame=single]
<file> := <item>*
<item> := IDENT : <value>
| ?IDENT: <value>
| IDENT STRING { <item>+ }
<value> := BOOL
| STRING
| SYMBOL
| IDENT
| [ <value>+ ]
| value { <value>+ }
\end{Verbatim}
}
\subsubsection{Global configuration file}
\label{config}
\verb+$opam/config+ follows the syntax defined in \S\ref{syntax} with
the following restrictions:
{\small
\begin{Verbatim}[frame=single]
<file> :=
opam-version: "1"
?repositories: [ <repo>+ ]
ocaml-version: STRING
<repo> := STRING { STRING }
\end{Verbatim}
}
The field {\tt repositories} contains the list of OPAM
repositories with their kind.
The field {\tt ocaml-version} corresponds to the current OCaml
compiler (available in the path).
\subsubsection{Package specification files: {\tt .opam}}
\label{dotopam}
\verb+$opam/opam/$NAME.$VERSION.opam+ follows the syntax defined in
\S\ref{syntax} with the following restrictions:
{\small
\begin{Verbatim}[frame=single]
<file> :=
opam-version: "1"
package STRING {
version: STRING
maintainer: STRING
?subst: [ STRING+ ]
?build: [ command+ ]
?remove: command
?depends: <cnf-formula>
?optdeps: <cnf-formula>
?conflicts: <and-formula>
?libraries: [ STRING+ ]
?syntax: [ STRING+ ]
}
<cnf-formula> := <cnf-formula> <cnf-formula>
| ( <cnf-formula> )
| <or-formula>
<or-formula> := STRING
| STRING { <constraints> }
| <or-formula> '|' <or-formula>
<and-formula> := STRING
| STRING { <constraints> }
| <and-formula> <and-formula>
<constraints> := <comp> STRING
| <constraints> <constraints>
<comp> := '=' | '<' | '>' | '>=' | '<='
<command> := [ STRING+ ]
\end{Verbatim}
}
\begin{itemize}
\item The first line specifies the OPAM version.
\item The string after {\tt package} should not contain any dot ({\tt
'.'}) nor space ({\tt ' '}).
\item The contents of {\tt version} is \verb+$VERSION+. The content of
{\tt maintainer} is the contact address of the package maintainer.
\item The content of {\tt subst} is the list of files to substitute
variable (see \S\ref{subst} for the file format and
\S\ref{opam-config} for the semantic of file substitution).
\item The content of {\tt build} is the list of commands to run in
order to build the package libraries. The build script should build
all the libraries and syntax extensions exported by the package and
it should produce the platform-specific configuration and install
files (e.g. \verb+$NAME.config+ and \verb+$NAME.install+, see
\S\ref{dotconfig} and \S\ref{dotinstall}).
\item The content of {\tt remove} is the command to run before deleting
the installed file.
\item The {\tt depends} and {\tt conflicts} fields contain
respectively {\em cnf-} and {\em and-}formulas over package names,
optionally parametrized by version constraints. An expression is
either:
\begin{itemize}
\item A package name: {\tt "foo"};
\item A package name with version constraints:
\verb+"foo" (>= "1.2" & <= "3.4")+
\item A disjunction of formulas: \verb+E | F+
\item A conjunction of formulas: \verb+E F+
\item A formula between parenthesis: \verb+( E )+
\end{itemize}
For instance \verb+ "foo" {<= "1.2"} ("bar" | "gna" {= "3.14"})+ is a
valid formula whose semantic is: {\em both any version of package
{\tt "foo"} lesser or equal to $1.2$ and either any version of package
{\tt "bar"} or the version $3.14$ of package {\tt "gna"}.} \\
\item The {\tt optdeps} field contains a {\em cnf-}formula over
package names, optionally parametrized by version constraints. When
installing a package, OPAM will not try to install the optional
dependencies. However, it will check if the package is an optional
dependency of already installed packages. If it is the case, it will
re-install the packages (and their transitive forward-dependency
closure).
\item The {\tt libraries} and {\tt syntax} fields contain the
libraries and syntax extensions defined by the package.
\end{itemize}
\subsubsection{Package configuration files: {\tt .config}}
\label{dotconfig}
\verb+$opam/OVERSION/config/NAME.config+ follows the syntax defined in
\S\ref{syntax}, with the following restrictions:
{\small
\begin{Verbatim}[frame=single]
<file> :=
opam-version: "1"
<item>*
<item> := <def> | <section>
<section> :=
<kind> STRING {
?asmcomp: [ STRING+ ]
?bytecomp: [ STRING+ ]
?asmlink : [ STRING+ ]
?bytelink: [ STRING+ ]
?requires: [ STRING+ ]
<def>*
}
<kind> := library | syntax
<def> := IDENT: BOOL
| IDENT: STRING
| IDENT: [ STRING+ ]
\end{Verbatim}
}
\verb+$NAME.config+ contains platform-dependent information which can
be useful for other libraries or syntax extensions that want to use
libraries defined in the package \verb+$NAME+.
\paragraph{Local and global variables}
The definitions ``{\tt IDENT: BOOL}'', ``{\tt IDENT: STRING}'' and ``{\tt IDENT:
[ STRING+ ]}'', are used to defined variables associated to this
package, and are used to substitute variables in template files (see
\S\ref{subst}):
\begin{itemize}
\item \verb+%{$NAME:$VAR}%+ will refer to the variable \verb+$VAR+
defined at the root of the configuration file \verb+$config/NAME.config+.
\item \verb+%{$NAME.$LIB:$VAR}%+ will refer to the variable \verb+$VAR+
defined in the {\tt library} or {\tt syntax} section named
\verb+$LIB+ in the configuration file \verb+$config/$NAME.config+.
\end{itemize}
\paragraph{Library and syntax sections}
Each {\tt library} and {\tt syntax} section defines an OCaml library
and the specific compilation flags to enable when using and linking
with this library.
The distinction between libraries and syntax extensions is only useful
at compile time to know whether the options should be used as
compilation or pre-processing arguments (ie. should they go on the
compiler command line or should they be passed to the {\tt -pp}
option). This is the responsibility of the build tool to do the right
thing and the {\tt <kind>} of sections is only used for documentation
purposes in OPAM. \\
The available options are:
\begin{itemize}
\item {\tt asmcomp} are compilation options to give to the native
compiler (when using the {\tt -c} option)
\item {\tt bytecomp} are compilation options to give to the bytecode
compiler (when using the {\tt -c} option)
\item {\tt asmlink} are linking options to give to the native compiler
\item {\tt bytlink} are linking options to give to the bytecode
compiler
\item {\tt requires} is the list of libraries and syntax extensions
the current block is depending on. The full list of compilation
and linking options is built by looking at the transitive closure of
dependencies. The contents of {\tt dep}s is the list of libraries or
syntax extension the current section depends on. Note that we do not
refer here to any package name, as multiple packages can expose libraries
with the same name and interface and thus we want the user to be able
to switch between them easily.
\end{itemize}
\subsubsection{Package installation files: {\tt .install}}
\label{dotinstall}
\verb+$opam/OVERSION/install/NAME.install+ follows the syntax defined
in \S\ref{syntax} with the following restrictions:
{\small
\begin{Verbatim}[frame=single]
<file> :=
opam-version: "1"
?lib: [ STRING+ ]
?bin: [ <mv>+ ]
?doc: [ STRING+ ]
?misc: [ <mv>+ ]
<mv> := STRING
| STRING { STRING }
\end{Verbatim}
}
\begin{itemize}
\item Files listed under {\tt lib} are copied to \verb+$lib/$NAME/+.
\item Files listed under {\tt bin} are copied to \verb+$bin/+ (they
can be renamed using \verb+$SRC { $DST }+; in this case \verb+$SRC+
should be a simple filename, ie. it should not start with a
directory name).
\item Files listed under {\tt doc} are copied to \verb+$doc/$NAME/+.
\item Files listed under {\tt misc} should be processed as follows:
for each pair \verb+$SRC { $DST }+, the tool should ask the user if
he wants to install \verb+$SRC+ to the absolute path \verb+$DST+.
\end{itemize}
\subsubsection{Substitution files}
\label{subst}
All of the previous files can be generated using a special mode of
{\tt opam} which can perform tests and substitutes variables (see
\S\ref{opam-config} for the exact command to run). Substitution files
contains some templates which will be replaced by some contents. The
syntax of templates is the following:
\begin{itemize}
\item templates such as \verb+%{$NAME:$VAR}%+ are replaced by the value
of the variable \verb+$VAR+ defined at the root of the file
\verb+$config/NAME.config+.
\item templates such as \verb+%{$NAME.$LIB:$VAR}%+ are replaced by the
value of the variable \verb+$VAR+ defined in the \verb+$LIB+ section
in the file \verb+$config/PACKAGE.config+
%% \item templates such as \verb+%{IF $test %{ $then %} THEN %{ $else %}}+ are
%% replaced by \verb+$then+ if \verb+$test+ is either:
%% \begin{itemize}
%% \item the ident {\tt true}
%% \item or a (global or local) variable whose value is the ident {\tt
%% true}
%% \item or an expression \verb+$var1 = $var2+ or \verb+$var = STRING+
%% where the contents of both sides of {\tt =} evaluates to the same
%% value.
%% \end{itemize}
%% Otherwise, the template is replaced by \verb+$else+.
\end{itemize}
\subsection{Client commands}
\subsubsection{Creating a fresh client state}
\label{opam-init}
When an end-user starts OPAM for the first time, he needs to
initialize \verb+$opam/+ in a consistent state. In order to do so, he
should run:
\begin{verbatim}
$ opam init [-kind $KIND] $REPO $ADDRESS
\end{verbatim}
Where:
\begin{itemize}
\item \verb+$KIND+ is the kind of OPAM repository (default is {\tt
http});
\item \verb+$REPO+ is the name of the repository (default is {\tt
default}); and
\item \verb+ADDRESS+ is the repository address (default is
\verb+http://opam.ocamlpro.com/pub+).
\end{itemize}
This command will:
\begin{enumerate}
\item Create the file \verb+$opam/config+ (as specified in
\S\ref{config})
\item Create an empty \verb+$opam/$OVERSION/installed+ file,
\verb+$OVERSION+ %being the result of ``{\tt ocamlc -version}''.
is the version from the OCaml used to compile \verb+$opam+.
In particular, we will not fail now
if there is no \verb+ocamlc+ in \verb+$path+.
\item Initialize \verb+$opam/repo/$REPO+ by running
\verb+'opam-$KIND-init $ADDRESS'+ (see \S\ref{script-update}). If
the script cannot be found in the path, the command should be
canceled and should return a well-defined error.
\item Symlink all OPAM and description files (ie. create a symbolic
link from every file in \verb+$opam/repo/$REPO/opam/+ to
\verb+$opam/opam/+ and from every file in
\verb+$opam/repo/$REPO/descr/+ to \verb+$opam/descr/+).
\item Create \verb+$opam/repo/index+ and for each version
\verb+$VERSION+ of package \verb+$NAME+ appearing in the repository,
append the line \verb+'$REPO $NAME $VERSION'+ to the file.
\item Create the empty directories \verb+$opam/archives+,
\verb+$lib/+, \verb+$bin/+ and \verb+$doc/+.
\end{enumerate}
\subsubsection{Listing packages}
\label{opam-list}
When an end-user wants to have information on all available packages,
he should run:
\begin{verbatim}
$ opam list
\end{verbatim}
This command will parse \verb+$opam/$OVERSION/installed+ to know the
installed packages, and \verb+$opam/opam/*.opam+ to get all the
available packages. It will then build a summary of each packages. The
description of each package will be read in \verb+$opam/descr/+ if it
exists.
For instance, if {\tt batteries} version {\tt 1.1.3} is installed,
{\tt ounit} version {\tt 2.3+dev} is installed and {\tt camomille} is
not installed, then running the previous command should display:
\begin{verbatim}
batteries 1.1.3 Batteries is a standard library replacement
ounit 2.3+dev Test framework
camomille -- Unicode support
\end{verbatim}
\subsubsection{Getting package info}
In case the end-user wants a more details view of a specific package,
he should run:
\begin{verbatim}
$ opam info $NAME
\end{verbatim}
This command will parse \verb+$opam/$OVERSION/installed+ to get the
installed version of \verb+$NAME+, will process
\verb+$opam/repo/index+ to get the repository where the package comes
from and will look for \verb+$opam/opam/$NAME.*.opam+ to get available
versions of \verb+$NAME+. It can then display:
\begin{verbatim}
package: $NAME
version: $VERSION # '--' if not installed
versions: $VERSION1, $VERSION2, ...
libraries: $LIB1, $LIB2, ...
syntax: $SYNTAX1, $SYNTAX2, ...
repository: $REPO
description:
$SYNOPSIS
$LINE1
$LINE2
$LINE3
...
\end{verbatim}
\subsubsection{Installing a package}
\label{opam-install}
When an end-user wants to install a new package, he should run:
\begin{verbatim}
$ opam install $NAME
\end{verbatim}
This command will:
\begin{enumerate}
\item Compute the transitive closure of dependencies and conflicts of
packages using the dependency solver (see \S\ref{deps}). If the
dependency solver returns more than one answer, the tool will ask
the user to pick one, otherwise it will proceed directly. The
dependency solver should also mark the packages to recompile.
\item The dependency solver sorts the collections of packages in
topological order. Then, for each of them do:
\begin{enumerate}
\item Check whether the package is already installed by looking for
the line \verb+$NAME $VERSION+ in \verb+$opam/$OVERSION/installed+.
If not, then:
\item Look into the archive cache to see whether it has already been
downloaded. The cache location is:
\verb+$opam/archives/$NAME.VERSION.tar.gz+
\item If not, process \verb+$opam/repo/index/+ to get the repository
\verb+$REPO+ where the archive is available, get the repository kind
by looking at \verb+$opam/repo/$REPO/kind+ and then ask the
repository to download the archive by calling
\verb+opam-$KIND-download+ (see \S\ref{script-download}).
Once this is done, symlink the archive in \verb+$opam/archives+.
\item Decompress the archive into \verb+$build/$NAME.$VERSION/+.
\item Substitute the required files.
\item Run the list of commands to build the package with \verb+$bin+
in the path.
\item Process \verb+$build/$NAME.$VERSION/$NAME.install+ to install
the created files. The file format is described in \S\ref{dotinstall}.
\item Install the installation file
\verb+$build/$NAME.$VERSION/$NAME.install+ in \verb+$install/+ and
the configuration file \verb+$build/$NAME.$VERSION/$NAME.config+ in
\verb+$config/+.
\end{enumerate}
\end{enumerate}
\subsubsection{Updating index files}
\label{opam-update}
When an end-user wants to know what are the latest packages available,
he will write:
\begin{verbatim}
$ opam update
\end{verbatim}
This command will follow the following steps:
\begin{itemize}
\item For each repositories in \verb+$opam/config+, run the
appropriate \verb+opam-$KIND-update+ script (see
\S\ref{script-update}).
\item For each repositories in \verb+$opam/config+, process
\verb+$opam/repo/$REPO/updated+ and update \verb+$opam/repo/index+,
\verb+$opam/opam/+ and \verb+$opam/desc+ accordingly (ie. add the
right lines in \verb+$opam/repo/index+ and create the missing
symlinks). Here, the order in which the repositories are specified
is important: the first repository containing a given version for a
package will be the one providing it (this can be changed manually
by editing \verb+$opam/repo/index+ later).
\item For each line \verb+$REPO $NAME $VERSION+ in
\verb+$opam/repo/index+, if the version \verb+$VERSION+ of package
\verb+$NAME+ has been modified upstream (ie. if the line
\verb+$NAME $VERSION+ appears in \verb+$opam/repo/$REPO/$updated+)
and if the package is already installed (ie. it appears in
\verb+opam/$OVERSION/installed+), then update
\verb+$opam/$OVERSION/reinstall+ accordingly (for each compiler
version \verb+$OVERSION+).
Packages in \verb+$opam/$OVERSION/reinstall+ will be reinstalled (or
upgraded if a new version is available) on the next {\tt opam
upgrade} (see \S\ref{opam-upgrade}), with \verb+$OVERSION+ being
the current compiler version when the upgrade command is run.
\item Delete each \verb+$opam/repo/$REPO/$updated+
\end{itemize}
\subsubsection{Upgrading installed packages}
\label{opam-upgrade}
When an end-user wants to upgrade the packages installed on his host,
he will write:
\begin{verbatim}
$ opam upgrade
\end{verbatim}
This command will:
\begin{itemize}
\item Call the dependency solver (see \S\ref{deps}) to
find a consistent state where {\bf most} of the installed packages are
upgraded to their latest version. Moreover, packages listed in
\verb+$opam/$OVERSION/reinstall+ will be reinstalled (or upgraded if a new
version is available). It will install each non-installed packages in
topological order, similar to what it is done during the install step,
See \S\ref{opam-install}.
\item Once this is done the command will delete
\verb+$opam/$OVERSION/reinstall+.
\end{itemize}
\subsubsection{Getting package configuration}
\label{opam-config}
The first version of OPAM contains the minimal information to be able
to use installed libraries. In order to do so, the end-user (or the
packager) should run:
\begin{verbatim}
$ opam config -list-vars
$ opam config -var $NAME:$VAR
$ opam config -var $NAME.$LIB:$VAR
$ opam config -subst $FILENAME+
$ opam config [-r] -I $NAME+
$ opam config [-r] -bytecomp $NAME.$LIB+
$ opam config [-r] -asmcomp $NAME.$LIB+
$ opam config [-r] -bytelink $NAME.$LIB+
$ opam config [-r] -asmlink $NAME.$LIB+
\end{verbatim}
\begin{itemize}
\item \verb+-list-vars+ will return the list of all variables defined
in installed packages (see \S\ref{dotconfig})
\item \verb+-var $var+ will return the value associated to the
variable \verb+$var+
\item \verb+-subst $FILENAME+ replace any occurrence of
\verb+%{$NAME:$VAR}%+ and \verb+%{$NAME.$LIB:$VAR}%+ as specified in
\S\ref{subst} in \verb+$FILENAME.in+ to create \verb+$FILENAME+.
\item \verb+-I $NAME+ will return the list of paths to include when
compiling a project using the package \verb+$NAME+ (\verb+-r+ gives
a result taking into account the transitive closure of
dependencies).
\item \verb+-bytecomp+, \verb+-asmcomp+, \verb+-bytelink+ and
\verb+-asmlink+ return the associated value for the section
\verb+$LIB+ in the file \verb+$config/$NAME.config+ (\verb+-r+ gives
a result taking into account the transitive closure of all
dependencies).
\end{itemize}
\subsubsection{Uploading packages}
\label{opam-upload}
When a packager wants to create a package, he should:
\begin{enumerate}
\item create \verb+$package/$NAME.$VERSION.opam+ containing in the format
specified in \S\ref{dotopam}.
\item create a file describing the package
\item make sure the build scripts:
\begin{itemize}
\item build the libraries and packages advertised in
\verb+$package/$NAME.$VERSION.opam+
\item generates a valid \verb+$package/$NAME.install+ containing the
list of files to install (the file format is described in
\ref{dotinstall}).
\item generates a valid \verb+$package/$NAME.config+ containing the
configuration flags for libraries exported by this package (the file
format is described in \ref{dotconfig}).
\end{itemize}
\item create an archive \verb+$NAME.$VERSION.tar.gz+ with the sources he
wants to distribute.
\item run the following command:
\begin{verbatim}
$ opam upload -opam $OPAM -descr $DESCR -archive $ARCHIVE -repo $REPO
\end{verbatim}
This command will parse \verb+$OPAM+ to get the package name and
version and it will:
\begin{itemize}
\item move \verb+$OPAM+ to \verb+$opam/repo/$REPO/upload/$NAME.$VERSION.opam+
\item move \verb+$DESCR+ to \verb+$opam/repo/$REPO/descr/$NAME.$VERSION+
\item move \verb+$ARCHIVE+ to \verb+$opam/repo/$REPO/archives/$NAME.$VERSION.tar.gz+
\end{itemize}
It will then call \verb+opam-$KIND-upload+ to upload the files
upstream (see \S\ref{script-upload}).
This command will work only for writable repositories; in case of
errors (for instance if the repository is read-only) the script
returns a non-zero exit code and stores the error message in
\verb+$opam/repo/$REPO/error+ (see \S\ref{script-error}).
\end{enumerate}
\subsubsection{Removing packages}
\label{opam-remove}
When the user wants to remove a package, he should write:
\begin{verbatim}
$ opam remove $NAME
\end{verbatim}
This command will check whether the package \verb+$NAME+ is installed,
and if yes, it will display to the user the list packages that will be
uninstalled (ie. the transitive closure of all forward-dependencies).
If the user accepts the list, all the packages should be uninstalled,
and the client state should be let in a consistent state.
\subsubsection{Managing OPAM repository}
When the user wants to manage OPAM repositories, he should write:
\begin{verbatim}
$ opam remote -list
$ opam remote -rm $REPO
$ opam remote -add [-kind $KIND] $REPO $ADRESS
\end{verbatim}
\begin{itemize}
\item \verb+-list+ lists the current repositories by looking at
\verb+$opam/config+
\item \verb+-rm $REPO+ deletes \verb+$opam/repo/$REPO+ and removes
\verb+$REPO+ from the {\tt repositories} list in \verb+$opam/config+.
Then, for each package in \verb+$opam/repo/index+ it updates the link
between packages and repositories (ie. it either deletes packages or
symlink them to the new repository containing the package).
\item \verb+-add [-kind $KIND] $REPO $ADDRESS+ initializes
\verb+$REPO+ as described in \S\ref{opam-init}.
\end{itemize}
\subsubsection{Dependency solver}
\label{deps}
Dependency solving is a hard problem and we do not plan to start from
scratch implementing a new SAT solver. Thus our plan to integrate (as
a library) the Debian depency solver for CUDF files, which is written
in OCaml.
\begin{itemize}
\item the dependency solver should run on the client; and
\item the dependency solver should take as input a list of packages
(with some optional version information) the user wants to install,
upgrade and remove and it should return a consistent list of
packages (with version numbers) to install, upgrade, recompile and
remove.
\end{itemize}
\subsection{OPAM repository scripts}