README

README how to build TeX live for Debian packages
================================================

IMPORTANT NOTE: This file is NOT installed into any Debian package, it is
here only for documentation purpose how to generate the deb source packages
and how in the deb source packages the binary debs are build.

----------------------------------------------------
QUICK HOWTO TO BUILD A SOURCE PACKAGE

. Do NOT create a new .orig.tar.xz, ie only make a new Debian revision,
  or read the whole text
. Put the .orig.tar.xz into . (or ./src/)
. edit whatever you have to change, don't forget to change
  ./$sourcepackage/debian/changelog (it is the debian
  changelog and defines the version number!)
. call
	perl ./tpm2deb-source.pl make-deb-source $sourcepackage
  where $sourcepackage is one of texlive-{base,extra,doc,lang} or 'all'
. texlive-bin is a separate package, that is located parallel to
  the current location in texlive-bin is a completely normal
  Debian package without any meta-generation.


After this you should have a normal Debian source package
----------------------------------------------------

===============================
BUT HOW DOES IT WORK IN DETAIL?
===============================

Short answer: Read the Makefile, everything is there.


Long answer:

Contents:
1) Layout of the directory
2) Building a new .orig.tar.gz
3) Source package build time
4) Binary package build time
5) Special stuff
   5a) Maintainer scripts handling
   5b) Changelogs
   5c) other special files handling
   5d) rules.in, tpm.data, lintian.override
6) Config file syntax and semantic
7) License information procedure
8) Some additional remarks for special files

========================================================


1) Layout of the directory:
===========================

The 4 directories
	texlive-base -extra -lang
have the same structure like an unpacked source package, ie they contain a
	debian/
subdirectory and optionally other subdirectories.

In addition there is a directory
	all
(also with the same structure as an unpacked source package) which holds
stuff common for all source packages.

Furthermore there are some files in the top level:

tpm2deb-source.pl
	the central script for building a .orig.tar.gz and a source
	package

do-one
do-all
	convenience scripts that builds one package (do-one) including
	the .orig.tar if necessary. do-all builds all in parallel, moves
	the generated files to pool directory, runs a apt archive
	generation and checks for duplicate files

Makefile
	not strictly essential, but it shows what you really have to do
	to generate a source package. Don't forget to call it with the
	correct MASTER setting, e.g.
		MASTER=/checkout/of/tlnet make foobar
	if you have need access to it at the foobar command (mostly
	only at source package build time when you also create a new
	.orig.tar.gz).

TODO
	well, obvious

scripts/
	generate-lin-report.pl
		summarize all the .lintian.log files into one lintian.log
		file, used in Makefile target lintian and linda
	deep-debdiff.sh
		my version of debdiff, used in Makefile target deepdiff
	build-infra
		a script to create package files, releases, signature etc
	tex-filter-debdiff.py
		script for filtering debdiff outputs

docs/
	all outdated !!!!

	README.ftpmaster
		a README I wrote to Jörg for explaining a lot of things
		after the first reject of the packages. Maybe it helps
	tl4debtree.*
		the sources/pdf of the dependency tree pdf
		if you change something, please do not forget to update
		also the respective .uu file in
		addons-per-source/texlive-doc/debian/tl4deb-deptree.pdf.uu
	source-pkg.txt
		approximate sizes and contents of the source packages
	soon to come hopefully the tugboat article

src/	(maybe not present)
	this contains the last build .orig.tar.gz. If they are present
	new ones are not build. NOT UNDER control, so you wont see
	it in a check out, but you could create it and put the orig.tar.gz
	into it ;-)

tests/
	tests/scripts: scripts that are run for
		install-remove-install-purge etc tests
	tests/stable tests/testing tests/sid
		upgrade from the respective releases

etc/
	random stuff


2) Building a new .orig.tar.gz
==============================

The Debian TeXLive repository does not contain the actual sources,
only the files needed to create Debian packages from them.  You have
to get the sources separately. Since TL2012 we are building from
a tlnet distribution, that is a set of .tar.xz files and some
infrastructure. This is the primarly distribution of TeX Live
nowadays.

Please get a copy of tlnet from CTAN the ctan network

	http://mirror.ctan.org/systems/texlive/tlnet/

For hints how to download from there using rsync/wget please visit:

	https://tug.org/texlive/acquire-mirror.html

Then call in the texlive-new trunk repository:
	perl tpm2deb-source.pl make-orig-tar --master=/path/to/tlnet $srcpackage
where $srcpackage is one of texlive-{base,extra,lang}.

IMPORTANT: for texlive-extra one additional argument is necessary: The texlive-extra
source package contains an additional .orig tarball
	texlive-extra_$version.orig-tex4ht.tar.xz
which is generated from files either in the TeX Live repository, or the
texlive-bin Debian package. One needs to pass in the location of the tex4ht
java files to the script:
	perl tpm2deb-source.pl make-orig-tar \
		--master=/path/to/tlnet		\
		--tex4ht-source=/path/to/tex4ht/java/	\
		texlive-extra
In the normal TeX Live repository these files are available in
	REPO/Build/source/texk/tex4htk/java
In the texlive-bin Debian package the files are available in
	texlive-bin/texk/tex4htk/java


The above call does nothing else than unpacking the necessary archives from tlnet,
to the source package directory, and creates the orig.tar.gz file from it.

Note that the blacklist: statements (see below) are taking action here, too,
so that blacklisted files and tpms are NOT included in the .orig.tar.gz.


3) Source package build time
============================

Note that for a debian release <= 1 you first have to create a new
.orig.tar.gz, see 2) above. The .orig.tar.gz has to be available at
some place, lets say $SRC.

At source package build time the following actions are taken
	- unpack the .orig.tar.gz
	- copy all the files from ./all/
	- copy all the files from ./$srcpackage/
	- create the rules file from ./all/debian/rules.in
	- create the control file
	- create the lintian override files
	- build the Debian source package with dpkg-source
all this is done with the perl script
	tpm2deb-source.pl

If you want to build a single source package, say texlive-base, call
	perl tpm2deb-source.pl make-deb-source		\
		$sourcepackage

Make sure the .orig.tar.xz you generated in step 2) are located in
directory ../texlive-nonbin, from where the command is called.

As usual, $sourcepackage can be one of texlive-{base,extra,lang}
or 'all' to build all of them.


4) Binary package build time
============================

When building the binary packages you may treat all them as ordinary
source packages and may use the common Debian tools "debuild",
"dpkg-buildpackage" etc.

Besides the rest as found in rules(.in) the main action is taken by
	perl debian/tpm2deb-bin.pl $packages
which does the following for each binary package $pkg
	- copy all the files to debian/$pkg/u/s/texlive/texmf-dist/...
	  this honors the remap/blacklist/etc directives in the
	  config file (see below).
	  Main function here is the do_remap_and_copy function.
	- next the Execute statements from the tpm files are executed
	  This can be addMap, BuildFormat, BuildLanguage
	  For the three cases the respective files in /etc/texmf/updmap.d,
	  fmt.d, language.d are created and the list files in
	  /v/l/tex-common/.... (ie it takes the job of dh_installtex)
	- next work on the SpecialActions as defined in the config file
	  currently only install-info for .info files
	  in this case a debian/$binpkg.info file is created for the
	  dh_installinfo call later on in the debian/rules file
	- finally the maintainer scripts are created (see below 5a)
After this the debian/rules file continues and does the following
(current situation) for each $binpkg:
	- generate the license file
	- remove the tpm files
	- install lintian override, bug script and control files
	- if present, add the shipped debian/$binpkg.links.dist to
	  debian/$binpkg.links (in earlier revision I think to remember
	  that tpm2deb(-bin).pl created $binpkg.links, but this is not
	  done anymore AFAIS, so this whole stuff could be deleted)
	- create the README.Debian files from the package specific
	  and the general one.


5) Special stuff
================

5a) maintainer scripts handling
-------------------------------

sub make_maintainer in all/debian/tpm2deb-bin.pl

The invocations of various commands (update-*, build_format etc) are
automatically generated from the various Execute statements in the
tpm files. But you can add stuff *before* and *after* the
automatically generated code by adding scripts
	$srcpackage/debian/$binpackage.$maintainerscript.{pre,post}
so the final maintainer scripts look like this:

$package.postinst
	included $package.postinst.pre
	generated code for $package.postinst
	included $package.postinst.post
and so on.

5b) Changelogs
--------------
All changes are collected in CHANGES.packaging (which is also installed into
the doc directory). The relevant changes must be merged by hand into the
respective changelog files
	$srcpackage/debian/changelog
by calling dch <args> -c $srcpackage/debian/changelog

5c) Other special files handling
--------------------------------
Some other special files also contain automatically generated content, but
can be augmented with additional text *at the beginning* by dropping files
into addons-per-bin/:
	README.Debian:
		$srcpackage/debian/$binpackage.README
	$package.links:
		$srcpackage/debian/$binpackage.links.dist
	Licenses:
		$srcpackage/debian/$binpackage.Licenses.add
So the final README.Debian for $package contains first a present
$srcpackage/debian/$binpackage.README followed by the automatically
generated stuff, in this case followed by the content of
all/debian/README.Debian.

The final debian/$binpackage.links (input for dh_link) contains first the
content if present of $srcpackage/debian/$binpackage.links.dist followed by the
automatically generated list.

The final Licenses contains first all/debian/Licenses.header,
then if present the file $srcpacakge/debian/$binpackage.Licenses.add
followed by the automatically generated content
(generated by all/debian/generate-license-file).

5d) rules.in, lintian.override
-------------------------------
rules.in
	An eperl script from which the real debian/rules file is
	generated at source package build time. It is called with
		-d arch=[any|all]
		-d srcpackage=texlive-[bin|base|extra|doc|lang]
		-d binpackages="list of bin packages of the respective src pkg"


lintian.override
	Gives the lintian override file lines, either without a package
	name, than the current bin package name is used, or with one,
	then the line is copied directly.
	I haven't found a better place for this file for now.



6) Config file syntax and semantic
==================================
The config file for building the orig.tar.gz, the source package and the
bin package is located in
	all/debian/tpm2deb.cfg
	all/debian/cfg.d/*

(list is not up to date!!)

arch;<package>;<arch>
	defines that the source package <package> has architecture <arch>
	default is "all" for <arch>
source;<pkgname>
	defines that <pkgname> is a source package, so if make_deb_source
	is called with argument "all" then these packages are build
maintainer;<package>|*;maintainer
	defines the maintainer for <package> or all packages
uploaders;<package>|*;uploaders
	defines the uploaders for <package> or all packages
section;<package>|*;section
	defines the section for <package> or all packages
priority;<package>|*;priority
	defines the priority for <package> or all packages
standards;<package>|*;standard
	defines the standards release
texlive-base-version;version
	minimal texlive-base version all packages depend on
latest-version;<srcpackage>;version
	latest version the texlive-full package depend on
	need not be the real latest greatest release
name;<tl-name>;<debian-name>;<debian-source>
	define a debian package:
	<tl-name> is the collection, like collection-latexextra
	<debian-name> is the name of the binary pkg: texlive-latex-extra
	<debian-source> maps this package into the source package
title;<tpm>;<title>
	changes the title of a tpm which will be used in the descriptions
description;<binpackage>;<description>
	gives the description in the control file
	this can be multi lined, so all lines are concatenated
move;<tpmname>;<binpkg>
	move a tpm to another package (debian name for <binpkg>!)
<relation>;<pkgname>;text
	where <relation> is in depends, recommends, build-dep, suggests,
	provides, replaces, conflicts
blacklist;tpm;<tpmname>;<in-which-collection>
	blacklist a whole tpm <tpmname>. the last entry gives
	either * for all collections, or the name of a binary package
	in which the tpm is blacklisted
	blacklist is applied at ORIG build time and naturally at binary
	build time, so this way DFSG incompatible stuff can be excluded
	from the .orig.tar.gz
blacklist;file;<origfilename>
	blacklist a file <origfilename>. The file name refers to the
	file name in UPSTREAM source (as listed in tpm files).
	It is a perl regexp!!!!
	Otherwise works as above.
kill;<origfilename>
	included in .orig.tar.gz
	not installed
ignore;<origfilename>
	included in .orig.tar
	not installed
	BUT post actions done!
notinorig;<origfilename>
	not included in .orig.tar
	BUT installed
	needs to be build in a different way (currently only tex4ht.jar!)
special;<filename>;<tag>
	currently only allowed for <tag> is "install-info"
	which installs the file <filename> via pkg.info and dh_installinfo
extra;format;<formatname>;<binarypkg>;<flag>
	currently only extra is "format" (as given) which allows to add
	additional calls to format generation in addition to the ExecuteActions
	in the tpm files.
	<formatname> is the format to be build via fmtutil-sys --byname
	<binarypkg> is the package name in which it is executed
	<flag> = 0	build_format --byfmt $fmt
	       = 1	build_format_if_format_exists --byfmt $fmt
	       o.w.	build_format_if_file_exists --byfmt $fmt <flag>
execute;tpm;actionline
	add an additional Execute statement to a tpm
	does the same as an entry in the tpm file
mapping;<origfilename>;<action>;<arg>
	controls the installation of files

	Note that "add as a config file" is *not* intended to register
	each conf(ig)file.  Instead it means that the file was a
	conffile in TL 2005, and its *move* needs to be handled
	properly for a conffile.

	<origfilename> (perl regexp) is the file which should be installed
	<arg> normally contains file names with possibly contained
		backreferences (like $1) to sub-patterns in the <origfilename>
		Filenames can be either given as relative paths, in this
		case /usr/share/texmf-texlive is prepended, or as abolute
		pathnames.
	<action> = move, = config-move
		the orig file is moved to <arg>
		in the case of config-move <arg> is added as config file
		= copy, = config-copy
		the orig file is copied to <arg>
		in the case of config-copy <arg> is added as config file
		create a symlink from <origfilename> to <arg>
		= copy-move
		<arg> is "<confabsolutepath>,<otherabsolutepath>"
		the orig file is moved to <confabsolutepath> and
		copied to <otherabsolutepath>
		= link
		make <origfilename> a symlink to <arg>, but do not create <arg>
		= move-link
		combination of move and link:
		move <origfilename> to <arg> (see above) and in addition
		= add-link
		install <origfilename> normally and add a symlink <arg> to it
		= replace-link
		<arg> must be aaa%bbb where aaa is a symlink to bbb
		=   (empty)
		just forget the file
		the difference to blacklist is that special actions can still
		be carried out


7) License Information Procedure
================================

At bin package build time:
--------------------------
see debian/rules, the script generate-license-information is called.

For now ignore the remap/etc stuff from tpm2deb.cfg