about:workflow

Jonathan Perkin edited this page Jan 12, 2018 · 11 revisions

Contents

  1. Introduction
  2. Repositories
    1. pkgsrc
    2. pkgsrc-joyent
    3. pkgsrc-wip
  3. Repository Flows
  4. Branches
    1. pkgsrc-YYYYQQ
    2. backports
    3. ctf
    4. miscfix
    5. multiarch
    6. pbulk
    7. pbulkmulti
    8. release
  5. FAQ
    1. Find which repository a package came from

Introduction

Our pkgsrc releases are made up from a number of different repositories and branches, which can be confusing for newcomers. This document aims to explain the relationships clearly and make it easy for users to know where issues and pull requests should go.

Repositories

We have three main repositories, though two of them are themselves forks of upstream repositories:

pkgsrc

This is the primary repository, located at https://github.com/joyent/pkgsrc/, and contains the majority of packages. It is a downstream fork of the official pkgsrc repository.

pkgsrc-joyent

This is a secondary repository, located at https://github.com/joyent/pkgsrc-joyent/. This repository contains a number of packages that are not (yet) suitable for committing either to pkgsrc or pkgsrc-wip, mostly because they have Joyent-specific requirements.

pkgsrc-wip

This is another secondary repository, located at https://github.com/joyent/pkgsrc-wip/, and based on the pkgsrc-wip project. This repository contains packages that are considered as not yet ready for inclusion into the main pkgsrc repository, but we find enough merit in some of them to include them in our package repositories.

Repository Flows

The following diagram shows how changes flow through the different repositories.

┌──────────────────────────────────────┬───────────────────────────────────────┐
│                                      │                                       │
│         The NetBSD Foundation        │                 Joyent                │
│         =====================        │                 ======                │
│                                      │                                       │
│       Commit directly to pkgsrc      │  Commit directly to pkgsrc-joyent and │
│                                      │    local branches of pkgsrc and wip   │
│                                      │                                       │
│  Access to pkgsrc is limited to TNF  │  Access limited to Joyent employees   │
│   members (via sponsorship process)  │                                       │
│                                      │                                       │
│  ┌────────────────────────────────┐  │                                       │
│  │ cvs.netbsd.org:/cvsroot/pkgsrc │  │                                       │
│  └───────────────┬────────────────┘  │                                       │
│                 [1]                  │                                       │
│     ┌────────────v─────────────┐     │     ┌──────────────────────────┐      │
│     │ github.com/NetBSD/pkgsrc ├────[2]───>│ github.com/joyent/pkgsrc │      │
│     └──────────────────────────┘     │     └──────────────────────────┘      │
│                                      │                                       │
├──────────────────────────────────────┤                                       │
│                                      │                                       │
│       The pkgsrc-wip project         │                                       │
│       ======================         │  ┌─────────────────────────────────┐  │
│                                      │  │ github.com/joyent/pkgsrc-joyent │  │
│    Commit directly to pkgsrc-wip     │  └─────────────────────────────────┘  │
│                                      │                                       │
│    Access open to anyone who asks    │                                       │
│                                      │                                       │
│    ┌────────────────────────────┐    │    ┌──────────────────────────────┐   │
│    │ wip.pkgsrc.org:/pkgsrc-wip ├───[3]──>│ github.com/joyent/pkgsrc-wip │   │
│    └────────────────────────────┘    │    └──────────────────────────────┘   │
│                                      │                                       │
└──────────────────────────────────────┴───────────────────────────────────────┘
  1. Regular fossil-based full conversion from CVS to Git exported to GitHub
  2. Forked repository regularly updated from upstream
  3. Loose fork, updates from wip are merged into joyent/pkgsrc-wip manually

Branches

We have developed a number of patch sets that are not yet suitable for pushing upstream to pkgsrc. In order to keep track of them they are separated into feature branches and then merged together to form our release branch.

For our 2017Q4 release it looks something like this:

┌──────────────────────────────────────┬───────────────────────────────────────┐
│                                      │                                       │
│         The NetBSD Foundation        │                 Joyent                │
│         =====================        │                 ======                │
│                                      │                                       │
│     ┌──────────────────────────┐     │     ┌──────────────────────────┐      │
│     │ github.com/NetBSD/pkgsrc │     │     │ github.com/joyent/pkgsrc │      │
│     └────────────┬─────────────┘     │     └──────────────────────────┘      │
│                  v                   │                                       │
│     ┌──────────────────────────┐     │     ┌──────────────────────────┐      │
│     │       pkgsrc-2017Q4      ├──────────>│      pkgsrc-2017Q4       │      │
│     └──────────────────────────┘     │     └────────────┬─────────────┘      │
│                                      │                  v                    │
│                                      │ ┌──────────────────────────────────┐  │
│                                      │ │ joyent/feature/backports/2017Q4  │  │
│                                      │ │ joyent/feature/ctf/2017Q4        │  │
│                                      │ │ joyent/feature/miscfix/2017Q4    │  │
│                                      │ │ joyent/feature/multiarch/2017Q4  │  │
│                                      │ │ joyent/feature/pbulk/2017Q4      │  │
│                                      │ │ joyent/feature/pbulkmulti/2017Q4 │  │
│                                      │ └────────────────┬─────────────────┘  │
│                                      │                  v                    │
│                                      │     ┌──────────────────────────┐      │
│                                      │     │   joyent/release/2017Q4  │      │
│                                      │     └──────────────────────────┘      │
└──────────────────────────────────────┴───────────────────────────────────────┘

The branches are as follows:

pkgsrc-YYYYQQ

The pristine branch from upstream. This is managed by the releng-pkgsrc@pkgsrc.org team who backport changes from pkgsrc trunk as requested by developers. This process is documented at https://www.netbsd.org/developers/releng/pullups.html

We pull this branch as-is into our own pkgsrc-YYYYQQ branch, and that is then used as the basis for our feature branches documented below.

backports

A branch where we perform our own pullups of changes from pkgsrc trunk that might not be suitable for general pkgsrc use, or are done after releng-pkgsrc have stopped maintaining their branch.

ctf

Our CTF work for pkgsrc-2017Q4 that enables CTF debugging support in supported packages.

miscfix

A general dumping ground for changes that aren't big enough for their own branch.

multiarch

Our branch to add multiarch support to our packages, see http://www.perkin.org.uk/posts/multiarch-package-support-in-smartos.html for more details.

pbulk

A set of changes to enhance pbulk, the software that we use to bulk build our package sets.

pbulkmulti

Changes to enable more multi-package builds, for example building p5-DBD-MySQL against every version of MySQL, MariaDB, and Percona that are supported in pkgsrc instead of just the default.

release

This is a combined merge of all of the feature branches, plus the addition of our pkgsrc-joyent and pkgsrc-wip submodules. This is the branch that is used to build the release packages.

FAQ

Find which repository a package came from

You can use pkg_info to show where a package came from, and thus which repository to log any issues or pull requests against. For example:

$ pkg_info -Q PKGPATH runit rust zoneinit
wip/runit
lang/rust
joyent/zoneinit

If it starts with joyent/ then pkgsrc-joyent, if wip/ then pkgsrc-wip, otherwise pkgsrc

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.