Skip to content

KLayout Package Manager (Salt)

Jump to bottom
Matthias Köfferlein edited this page Jul 29, 2017 · 6 revisions

Introduction

Let there be work, bread, water and salt for all.

Nelson Mandela

Why "Salt"? Because without salt, food is often a little tasteless. By adding salt, you make things interesting.

So "Salt" is KLayout's package manager which allows selecting and installing packages from a global repository. Packages make KLayout more tasty. Packages (the "grains") may cover a variety of features:

  • Ruby or Python macros
  • DRC runsets
  • Technologies
  • Fonts for the Basic.TEXT PCell
  • Static layout libraries
  • PCell libraries
  • Code libraries for Ruby and Python
  • Binary extensions

Packages can depend on other packages - these are installed automatically if a package requires them and they are not installed yet.

Packages are identified by name. A package name needs to be unique in the package universe. You can use a prefixed name like "www.mydomain.org/nameofpackage" to create a non-ambiguous name. The choice of the prefix is entirely up to you. You can use a domain name that is owned by yourself for example.

Packages also come with version information, so KLayout can check for updates and install them if required. KLayout will assume strict upward compatibility. This specifically applies to packages that other packages are depending on (such as code libraries). If you need to change them in a non-backward compatible way, you'd need to provide a new package with a different name.

Packages come with some meta data such as authoring information, an optional icon and screen shot image, license information and more. The more information you provide, the more useful a package will become.

The key component for the package repository is the "Salt.Mine" package repository service (sami.klayout.org). It is a web service that maintains a package list. It does not host the packages, but stores links to the actual hosting site. In order to author a package, you need to upload the package to one of the supported host sites and register your package on the Salt.Mine page. Registration is a simple process and the only information required is the link to your host site and a mail account for confirmation.

Currently, GitHub is supported as a host site. As a package author you'll need a GitHub account and open a project for your package there. GitHub acts both as a code repository for your development and as a deployment platform. KLayout can download packages directly from your GitHub project's master or a release tag.

The DNA

Technically, packages are folders stored below the "salt" folder within KLayout's home path. Prefixed packages are kept in sub-folders named after the prefix. Each package directory contains a description file called "grain.xml" and the files that make up the package. For Ruby macros, for example, the package folder will contain a "macros" sub-folder. Basically, packages do not require to be of a specific kind - they can provide technology information, DRC scripts and macros in the same package. In this example, the will be a "tech", "drc" and "macros" sub-folder with the corresponding files. KLayout will scan the package folders for useful files and load them automatically.

Packages are published on the Salt.Mine server. This is a web service that delivers a complete list of packages plus a little meta data such as current version, the icon and a brief description. KLayout will download this list from "http://sami.klayout.org/repository.xml" by default. You can set the KLAYOUT_SALT_MINE environment variable to a different URL which makes KLayout use another repository service, i.e. one inside your own organisation. This service can be any HTTP server that delivers a package list in the same format than the Salt.Mine package service.

When installing a package, KLayout will simply download the files from the URL given in the package list. KLayout employs the WebDAV protocol to download the files. This protocol is spoken by Subversion and GitHub with the subversion bridge. The latter requires a simple translation of the original Git URL's to obtain the subversion equivalent.

For package development you utilize KLayout to initialize and edit the files inside the package folder or populate the folder manually. You will find a tutorial about how to connect to GitHub for development or deployment below.

Publishing Packages on GitHub

Master is Released approach

The simple approach is "master is released": in this case, you use the master branch on GitHub to keep the released version. This means, you are doing work on Git branches. Everything that gets pushed to GitHub's remote master becomes available for installation for the users.

The advantage of this approach is that new versions get automatically distributed as long as the version number is incremented. A package is registered once, but there is no need to re-register new package version. The disadvantage of this approach is less control over what is actually released: there is no more checkpoint between a push to master and distribution. And there is no way back - you cannot withdraw the master. You can only push a new release that reverts updates if they are broken.

This is the recipe:

  1. Create the new package in Package Manager:

    • Go to "Current Packages" page
    • Choose "Create Package" from the bottom
    • Edit the package details with the pen icon above the package details panel
    • Use ".." as the documentation URL to refer to GitHub's main page as the documentation. Or use an explicit URL to any other documentation place you control.

  2. In GitHub create a new repository.

  3. Connect the salt package to the GIT repo. Example: for package mypackage and GitHub repository myself/mypackage use these commands in the shell:

    • cd ~/.klayout/salt/mypackage
    • git init .
    • git remote add origin https://github.com/myself/mypackage.git
    • git pull origin master
    • Add and commit your files with:
      • git add *
      • git commit
    • git push origin master

  4. Register the new package at Salt.Mine (http://sami.klayout.org/register). Salt.Mine is a link service that provides a list of packages. It does not hold the actual package content. Some package attributes like description texts and icons will be kept by the registration service, but the actual download location needs to be provided by you.

    Currently, only GitHub is supported as a source of package downloads. To register a GitHub repository for download, enter the GitHub URL on the registration page Use the subversion compatible repository paths. In this example the path is:

    https://github.com/myself/mypackage.git/trunk

    ("trunk" means "master" in SVN terminology).

    You also need to supply a mail address that will be used to manage the package in Salt.Mine. Please use a "real", sustaining mail account (no single-use accounts). The mail address will be used when you need to remove or update a package. Please check the mail address before you submit the registration.

    Use "view package repository" to check the link.

    Press "Submit", check your mail and follow the confirmation link to finish the registration. Registration links are valid for a few days only.

  5. On updates:

    • Make sure you increment the version to notify users of your changes. Changes without a version update may not become visible as updates.
    • Commit your changes to GitHub's master
    • Allow for one hour maximum to let Salt.Mine synchronize itself with your master.

Tagged releases

If you plan to create bigger packages or you want community contributions, you should consider deploying your packages using GitHub's release feature. Technically, a GitHub release is a tag. You can register this tag in Salt.Mine to refer to a specific release. This recipe is valid as well if you use release branches.

Every new release requires you to re-register the package manually with the new URL.

The recipe is basically the same than for the "master is released" approach. But

  1. You can use "../.." as the documentation URL to lead users to your GitHub main page for documentation. It needs to go up two directories since we are going to use "tags/x" as the relative version path.

  2. After pushing your changes to master, do a release with GitHub's release feature. Create a new release with a new version. Technically this will produce a new tag on GitHub with the URL suffix "tags/x" for version "x" (i.e. "tags/1.0" for version "1.0"). It is recommended to follow the semantic version scheme suggested by GitHub.

  3. After the tag has been created, go to the Salt Mine package registration page (http://sami.klayout.org/register) and enter the new package URL. As the URL use

    https://github.com/myself/mypackage.git/tags/x

    for version "x".

    Enter the mail address you used for the initial release. Check the "Accept terms of use" and click on the "Submit" button.

    You will receive a mail with the confirmation link. Follow this link to activate your new release.

Automatic updates

The Salt.Mine service will check the packages regularly (once a day) and update the attributes of the packages if required. Packages that are no longer available will disappear from the list and will be deleted when they are unavailable for more than 3 days.

Contributing to package development

You may want to contribute code to a package hosted on GitHub or any other Git server. In this case, don't install the package, but clone the repository into KLayout's "salt" folder. This folder is found in KLayout's home folder. On Linux that folder is found in ~/.klayout/salt, on Windows the path is usually C:\Users\yourself\KLayout\salt.

You can also use any other place too - for example ~/klayout-dev/salt - and add this location (without the "salt") to the KLAYOUT_PATH environment variable. On Linux:

export KLAYOUT_PATH ~/klayout-dev

Contact for support

If you need support with the package registration, please contact the salt.mine admin through contact@klayout.de.

Clone this wiki locally