Chapter 14

Michael Neumann edited this page May 14, 2018 · 19 revisions

Port Creation Walk-through

This page serves as an example on how to create a new ravenport.

Step 1: Select a good port name

The namebase of a port often comes directly from the distribution tarball, but not always. If the port exists in the FreeBSD ports collection, pkgsrc, or ArchLinux packages, consider using the same name these ports use.

After picking a candidate name, go to https://repology.org/ and find the search packages box under the Quick navigation heading. Use the search feature here get a list of repology entries matching the candidate name pattern and look for an exact match. Click on that match's link to see all the variations that map to the repology entry and confirm that the repology entry actually refers to the same software you are trying to port.

If you find your name isn't a good candidate, use Repology's search feature to identify what other repositories are using and attempt to maintain consistency.

In some cases, the commonly used name is not good (often based on a poor name for the distribution file). We can intentionally select a namebase that differs from others commonly used, but we may have to open an issue at repology to fix mapping later.

For our example, I enter "libgit" in the repology search box, with these results:

Metapackage Spread Newest
haskell:gitlib-libgit2 2 3.1.12
haskell:hlibgit2 2 0.18.0.162
haskell:libgit 2 0.3.12
libgit2 20 0.26.012
libgit2-25 1 0.25.1
libgit2-glib 14 0.26.010
libgitimport 1 0.0.1
mingw-w64-libgit2 1 0.26.0

Okay, so "libgit2" looks like the winner. After clicking on the libgit2 hyperlink, I confirm that every repository uses "libgit2" except AUR which uses "libgit2-git". I click on the "information" tab at the top of the page and confirm from the "Homepage" and "Downloads" section that libgit2 refers to the same software I'm trying to port. I leave this "information" page up to refer to it later.

Step 2: Create entry in ravensource directory

I started with the assumption that you have already acquired the ravensource git repository. If you haven't, go to https://github.com/jrmarino/ravensource and "fork" a copy to your github account. Then clone that forked repository to a directory of your choosing (I'm using ~/ravensource for the sake of this example).

The port location is derived from the namebase. To find the expected location for our new port, I use the ravenadm's location command:

> /raven/bin/ravenadm locate libgit2
Does not exist at /usr/raven/bucket_11/libgit2

This tells me the libgit2 port source files must be located at ~/ravensource/bucket_11/libgit2 so I create this directory.

> mkdir ~/ravensource/bucket_11/libgit2

Step 3: Create template

This is just a little time-saving step

> cd ~/ravensource/bucket_11/libgit2
> /raven/bin/ravenadm dev template save

I've created two empty directories and a specification file in the new directory:

> ls -al
total 20
drwxr-xr-x  4 marino  wheel  512 Sep 28 08:33 .
drwxr-xr-x  6 marino  wheel  512 Sep 28 08:32 ..
drwxr-xr-x  2 root    wheel  512 Sep 28 08:33 descriptions
drwxr-xr-x  2 root    wheel  512 Sep 28 08:33 manifests
-rw-r--r--  1 root    wheel  449 Sep 28 08:33 specification

Step 4: Try to define subpackages now

Any given port can have between one and an unlimited number of subpackages. Ports with more than one subpackage can have a metapackage created automatically by naming the first subpackage as "complete". So examples for the current conventions for subpackage selection are:

single
complete primary docs
complete primary examples docs
complete primary manpages
complete server client
complete static shared
single common

If the first subpackage is called "complete", a run dependency on all following subpackages will automatically be created. It is not necessary to provide descriptions or manifests for "complete" subpackages. These are also taken care of automatically.

If the subpackage "docs" is defined, the manifest and descriptions are taken care of if they are not present. All files in the ${LOCALBASE}/share/doc/${namebase} subdirectory of the stage directory will be automatically added to docs manifest, but only if no manifests/plist.doc* files exist.

If the subpackage "examples" is defined, the manifest and descriptions are taken care of if they are not present. All files in the ${LOCALBASE}/share/examples/${namebase} subdirectory of the stage directory will be automatically added to docs manifest, but only if no manifests/plist.examples* files exist.

It's not always possible to know if a port will have documentation and examples at this point, but you can look at existing works. For example, https://svnweb.freebsd.org/ports/head/devel/libgit2/pkg-plist shows no documentation or examples installed on FreeBSD, so "single" is a good starting definition for our "libgit2" port.

Step 5: Create subpackage descriptions

(If you haven't been able to finalize your subpackage definition yet, you can skip to Step 7 and come back to this step afterwards.)

The full name for a description file uses the format, "desc.[subpackage].[variant]", e.g. "desc.single.standard". However, if the last component is omitted, the same description file will be used for all variants. If both files are present, the former takes precedence for the given variant.

For example, say a port has two variants, "standard" and "loaded". Two files are present in the descriptions directory:

desc.single.loaded
desc.single

The "standard" variant will use "desc.single" file and the "loaded" variant will use the "desc.single.loaded" file.

A quick way to create this file is just copy and edit a freebsd version, e.g.

cp /usr/ports/devel/libgit2/pkg-descr descriptions/desc.single

If you do this, remove the "WWW:" and "AUTHORS:" line. Also, make sure the description is wrapped to column 75 (/usr/bin/fold -s -w75 [file] can help). Sometimes the FreeBSD descriptions are quite old, so check the software's home page to see if there's a better description available. Try to keep the description less than 25 lines, but this is not a hard requirement.

Step 6: Create subpackage packing manifests

If you don't have any idea what will be installed, you can skip this step and come back. When you test-build the port, all the packing errors will be listed on the log. You can actually use the log errors to create the packing list.

However, if you are converting a FreeBSD port, you can get a head start by copying FreeBSD's existing manifest:

> cp /usr/ports/devel/libgit2/pkg-plist manifests/plist.single

After you copy the file, open it and remove entries for documentation and examples. If these are present, you either have to remove them during post-install (not all documentation is worth installing as a package!) or you have to have a docs/examples subpackage, and plist.single will need to be renamed to plist.primary.

Take particular note on %% variables. All option variables, e.g. "%%NLS%%, will either have to be modified (e.g. "%%NLS-ON%%") or removed altogether. It depends on what you do with the specification file. You will probably have to tweak your manifest list several times during the course of port testing.

When the manifests are done, ensure the items are in the proper order with the dev sort command:

ravenadm dev sort

The above example assumes the current working directory is of the port being worked on. If that's not the case, the path can be added as an additional argument. The command will scan in all the lists in the manifest subdirectory and re-write them in a specific order that cannot be mimicked with the unix sort command.

Step 7: Fill out specification

The default template starts the specification file like this:

# DEF[PORTVERSION]=     1.00
# ----------------------------------------------------------------------------

NAMEBASE=               ...
VERSION=                ${PORTVERSION}
KEYWORDS=               ...
VARIANTS=               standard
SDESC[standard]=        ...
HOMEPAGE=               none
CONTACT=                Jay_Leno[jay@aarp.org]

DOWNLOAD_GROUPS=        main
SITES[main]=            http://www.example.com/
DISTFILE[1]=            x-${PORTVERSION}.tar.gz:main

SPKGS[standard]=        single

OPTIONS_AVAILABLE=      none
OPTIONS_STANDARD=       none

FPC_EQUIVALENT=         ...

Using information on the repology "information" page for libgit2 and perhaps also using /usr/ports/devel/libgit2/Makefile, I quickly update the file like this:

DEF[PORTVERSION]=       0.26.0
# ----------------------------------------------------------------------------

NAMEBASE=               libgit2
VERSION=                ${PORTVERSION}
KEYWORDS=               devel
VARIANTS=               standard
SDESC[standard]=        Library providing Git core methods
HOMEPAGE=               https://libgit2.github.com/
CONTACT=                John_Marino[draco@marino.st]

DOWNLOAD_GROUPS=        main
SITES[main]=            GITHUB/libgit2:libgit2:v${PORTVERSION}
DISTFILE[1]=            generated:main

SPKGS[standard]=        single

OPTIONS_AVAILABLE=      none
OPTIONS_STANDARD=       none

FPC_EQUIVALENT=         devel/libgit2

The "FPC_EQUIVALENT" field can be omitted if there is no equivalent in FreeBSD ports. Particularly note that I used repology's "Homepages" section to pick the most appropriate URL that isn't a redirect. I also heavily use the "Summaries" section to see if I can match commonly used descriptions, or at least be inspired by them.

Using FreeBSD's makefile as a guide, I quickly finish the rest of the specification file:

DEF[PORTVERSION]=       0.26.0
DEF[SOVERSION]=         ${PORTVERSION}
DEF[SOMAJOR]=           ${SOVERSION:R:E}
# ----------------------------------------------------------------------------

NAMEBASE=               libgit2
VERSION=                ${PORTVERSION}
KEYWORDS=               devel
VARIANTS=               standard
SDESC[standard]=        Library providing Git core methods
HOMEPAGE=               https://libgit2.github.com/
CONTACT=                John_Marino[draco@marino.st]

DOWNLOAD_GROUPS=        main
SITES[main]=            GITHUB/libgit2:libgit2:v${PORTVERSION}
DISTFILE[1]=            generated:main

SPKGS[standard]=        single

OPTIONS_AVAILABLE=      none
OPTIONS_STANDARD=       none

LICENSE=                CUSTOM1:single
LICENSE_NAME=           CUSTOM1:"libgit2 License - GPLv2 with linking exception"
LICENSE_FILE=           CUSTOM1:{{WRKSRC}}/COPYING
LICENSE_SCHEME=         solo

FPC_EQUIVALENT=         devel/libgit2

BUILDRUN_DEPENDS=       libssh2:single:standard
                        curl:primary:standard

USES=                   cmake pkgconfig python:build ssl zlib
CMAKE_ARGS=             -DUSE_SSH=ON
                        -DTHREADSAFE=ON
                        -DCURL=ON
PLIST_SUB=              SOVERSION=${SOVERSION}
                        SOMAJOR=${SOMAJOR}

Note that I decided to convert FreeBSD's default-on options into standard configurations (thus removing the options). In general, options should be implemented as variants. That is, if it were important to have a curl-free libgit2, I should define a variant port that builds libgit without curl.

Step 8: Generate distinfo file

If the specification file has no errors in it, we can now generate the distinfo file:

> /raven/bin/ravenadm dev distinfo

Copying /usr/obj/ravenports/SL09/port/distinfo to current directory

Let's check it out.

> cat distinfo
6a62393e0ceb37d02fe0d5707713f504e7acac9006ef33da1e88960bd78b6eac      4697156 libgit2-libgit2-0.26.0.tar.gz

If there are errors in the specification file, fix them and try again.

Step 9: Incorporate new port into configured conspiracy directory

Now compile the port:

> /raven/bin/ravenadm dev buildsheet . save
libgit2 buildsheet created at:
/usr/raven/bucket_11/libgit2

We have to regenerate a new conspiracy index for ravenadm. This command only needs to be done once:

> /raven/bin/ravenadm dev generate-index
Index successfully generated.
       Total ports : 1356
    Total variants : 2107
    Total packages : 2459
  Linear scan time : 00:00:01

Step 10: Test-build port

Let's test the port:

> /raven/bin/ravenadm test libgit2

Step 11: Iterate

If it fails to build, check the log:

vi  /mech/var/ravenports/primary/logs/logs/libgit2___standard.log

In this case, it did fail to build due to manifest file errors (I forgot to update the %% variables and change libdata/pkgconfig to lib/pkgconfig and one header is different from FreeBSD's version):

--------------------------------------------------------------------------------
--  Phase: check-plist
--------------------------------------------------------------------------------
====> Checking for package manifest issues
Directory raven/libdata/pkgconfig listed on single manifest is not present in the stag
e directory.
Orphaned file detected: lib/pkgconfig/libgit2.pc
Orphaned file detected: lib/libgit2.so.26
Orphaned file detected: lib/libgit2.so.0.26.0
Orphaned file detected: include/git2/worktree.h
File raven/lib/libgit2.so.%%PORTVERSION%% listed on single manifest is not present in
the stage directory.
File raven/libdata/pkgconfig/libgit2.pc listed on single manifest is not present in th
e stage directory.
File raven/lib/libgit2.so.%%SHLIB_VER%% listed on single manifest is not present in th
e stage directory.

After adjusting the manifest, the next test passed just fine.

The task is complete.  Final tally:
Initial queue size: 1
    packages built: 1
           ignored: 0
           skipped: 0
            failed: 0

Duration: 00:00:28
The build logs can be found at: /mech/var/ravenports/primary/logs/logs

I recheck the log anyway:

--------------------------------------------------------------------------------
--  Phase: deinstall
--------------------------------------------------------------------------------
=> Checking shared library dependencies
### WARNING ###  /raven/lib/libgit2.so is not stripped.  See Ravenporter's guide.
### WARNING ###  /raven/lib/libgit2.so.0.26.0 is not stripped.  See Ravenporter's guid
e.
### WARNING ###  /raven/lib/libgit2.so.26 is not stripped.  See Ravenporter's guide.
===> libgit2-single-standard-0.26.0 subpackage:
     NEEDED               libcurl.so.4
     NEEDED               libz.so.1
     NEEDED               libssl.so.43
     NEEDED               libcrypto.so.41
     NEEDED               libssh2.so.1
     NEEDED               librt.so.0
     NEEDED               libpthread.so.0
     NEEDED               libc.so.8

The target of the /raven/lib/libgit2.so symlink needs to be stripped, so I add this to the specification:

post-install:
        ${STRIP_CMD} ${STAGEDIR}${PREFIX}/lib/libgit2.so

Now everything really passes after recompiling and testing the port.

Step 12: Open a pull request

Once you are confident in the quality of the port and you want to contrib, push your change to your fork of ravensource and open a pull request against the official repository.

Previous Chapter: Updating home Next Chapter: Appendix
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.