🍝 PureScript package manager and build tool powered by Dhall and Spacchetti package-sets
Branch: master
Clone or download

README.md

spago

Build Status

(IPA: /ˈspaɑo/)

PureScript package manager and build tool powered by Dhall and Spacchetti package-sets.

spacchetti logo

What does all of this mean?

spago aims to tie together the UX of developing a PureScript project.
In this Pursuit (see what I did there) it is heavily inspired by Rust's Cargo and Haskell's Stack, and builds on top of ideas from existing PureScript infrastructure and tooling, as psc-package, pulp and purp.

Installation

Right, so how can I get this thing?

The recommended installation methods are:

  • npm install -g purescript-spago (Linux and macOS only - see the latest releases on npm here)
  • Download the binary from the latest GitHub release
  • Compile from source by cloning this repo and running stack install

Note #1: support for Windows is still basic, and we're sorry for this - the reason is that no current maintainer runs it.
If you'd like to help with this that's awesome! Get in touch by taking a look at the open issues and eventually opening one :)

Note #2: we assume you already installed the PureScript compiler. If not, get it with npm install -g purescript, or the recommended method for your OS.

Quickstart

Let's set up a new project!

$ mkdir purescript-unicorns
$ cd purescript-unicorns
$ spago init

This last command will create a bunch of files:

.
β”œβ”€β”€ packages.dhall
β”œβ”€β”€ spago.dhall
β”œβ”€β”€ src
β”‚Β Β  └── Main.purs
└── test
    └── Main.purs

Convention note: spago expects your source files to be in src/ and your test files in test/.
It is possible to include additional source paths when running some commands, like build, test or repl.

Let's take a look at the two Dhall configuration files that spago requires:

  • packages.dhall: this file is meant to contain the totality of the packages available to your project (that is, any package you might want to import).
    In practical terms, it pulls in a Spacchetti package-set as a base, and you are then able to add any package that might not be in the package set, or override esisting ones.
  • spago.dhall: this is your project configuration. It includes the above package-set, the list of your dependencies, and any other project-wide setting that spago will use for builds.

Switching from psc-package

Do you have an existing psc-package project and want to switch to spago?

No problem! If you run spago init, we'll port your existing psc-package.json configuration into a new spago.dhall 😎

Note: spago won't otherwise touch your psc-package.json file, so you'll have to remove it yourself.

You'll note that most of the psc-package commands are the same in spago, so porting your existing build is just a matter of search-and-replace most of the times.

Configuration file format

It's indeed useful to know what's the format (or more precisely, the Dhall type) of the files that spago expects. Let's define them in Dhall:

-- The basic building block is a Package:
let Package =
  { dependencies : List Text  -- the list of dependencies of the Package
  , repo = Text               -- the address of the git repo the Package is at
  , version = Text            -- git tag
  }

-- The type of `packages.dhall` is a Record from a PackageName to a Package
-- We're kind of stretching Dhall syntax here when defining this, but let's
-- say that its type is something like this:
let PackageSet =
  { console : Package
  , effect : Package
  ...                  -- and so on, for all the packages in the package-set
  }

-- The type of the `spago.dhall` configuration is then the following:
let Config =
  { name : Text               -- the name of our project
  , dependencies : List Text  -- the list of dependencies of our app
  , packages : PackageSet     -- this is the type we just defined above
  }

Commands

For an overview of the available commands, run:

$ spago --help

You will see several subcommands (e.g. build, test); you can ask for help about them by invoking the command with --help, e.g.:

$ spago build --help

This will give a detailed view of the command, and list any command-specific (vs global) flags.

Package management

We initialized a project and saw how to configure dependencies and packages, the next step is fetching its dependencies.

If we run:

$ spago install

..then spago will download all the dependencies listed in spago.dhall (and store them in the .spago folder).

Listing available packages

It is sometimes useful to know which packages are contained in our package set (e.g. to see which version we're using, or to search for packages).

You can get a complete list of the packages your packages.dhall imports (together with their versions and URLs) by running:

$ spago list-packages

By passing the --filter flag you can restrict the list to direct or transitive dependencies:

# Direct dependencies, i.e. only the ones listed in spago.dhall
$ spago list-packages --filter=direct

# Transitive dependencies, i.e. all the dependencies of your dependencies
$ spago list-packages -f transitive

Adding a dependency

You can add dependencies from your package-set by running:

$ spago install my-new-package another-package

Adding and overriding dependencies in the Package Set

Let's say I'm a user of the simple-json package. Now, let's say I stumble upon a bug in there, but thankfully I figure how to fix it. So I clone it locally and add my fix.
Now if I want to test this version in my current project, how can I tell spago to do it?

We have a overrides record in packages.dhall just for that!
And in this case we override the repo key with the local path of the package.
It might look like this:

let overrides =
      { simple-json =
            upstream.simple-json // { repo = "../purescript-simple-json" }
      }

Note that if we list-packages, we'll see that it is now included as a local package:

$ spago list-packages
...
signal                v10.1.0   Remote "https://github.com/bodil/purescript-signal.git"
sijidou               v0.1.0    Remote "https://github.com/justinwoo/purescript-sijidou.git"
simple-json           v4.4.0    Local "../purescript-simple-json"
simple-json-generics  v0.1.0    Remote "https://github.com/justinwoo/purescript-simple-json-generics.git"
smolder               v11.0.1   Remote "https://github.com/bodil/purescript-smolder.git"
...

And since local packages are just included in the build, if we add it to the dependencies in spago.dhall and then do spago install, it will not be downloaded:

$ spago install
Installing 42 dependencies.
...
Installing "refs"
Installing "identity"
Skipping package "simple-json", using local path: "../purescript-simple-json"
Installing "control"
Installing "enums"
...

Let's now say that we test that our fix works, and we are ready to Pull Request the fix.
So we push our fork and open the PR, but while we wait for the fix to land on the next package-set release, we still want to use the fix in our production build.

In this case, we can just change the override to point to some branch of our fork, like this:

let overrides =
    { simple-json =
          upstream.simple-json
       // { repo = "https://github.com/my-user/purescript-simple-json.git"
          , version = "my-branch-with-the-fix"
          }
    }

Note: currently only "branches" and "tags" work as a version, and tags are recommended over branches (as for example if you push new commits to a branch, spago won't pick them up unless you delete the .spago folder).
Commit hashes are not supported yet, but hopefully will be at some point.

If a package is not in the upstream package-set, you can add it in a similar way, by changing the additions record in the packages.dhall file.
E.g. if we want to add the facebook package:

let additions =
  { facebook =
      mkPackage
        [ "console"
        , "aff"
        , "prelude"
        , "foreign"
        , "foreign-generic"
        , "errors"
        , "effect"
        ]
        "https://github.com/Unisay/purescript-facebook.git"
        "v0.3.0"
  }

The mkPackage function should be already included in your packages.dhall, and it will expect as input a list of dependencies, the location of the package, and the tag you wish to use.

Of course this works also in the case of adding local packages. In this case you won't care about the value of the "version" (since it won't be used), so you can put arbitrary values in there.

And of course if the package you're adding has a spago.dhall file you can just import it and pull the dependencies from there, instead of typing down the list of dependencies!

Example:

let additions =
  { foobar =
      mkPackage
        (../foobar/spago.dhall).dependencies
        "../foobar"
        "local-fix-whatever"
  }

Verifying your additions and overrides

"But wait", you might say, "how do I know that my override doesn't break the package-set?"

This is a fair question, and you can verify that your fix didn't break the rest of the package-set by running the verify command.

E.g. if you patched the foreign package, and added it as a local package to your package-set, you can check that you didn't break its dependants (also called "reverse dependencies") by running:

$ spago verify foreign

Once you check that the packages you added verify correctly, we would of course very much love if you could pull request it to the Upstream package-set, spacchetti ❀️🍝

Upgrading the Package Set

The version of the package-set you depend on is fixed in the packages.dhall file (look for the upstream var).

You can upgrade to the latest version of Spacchetti with the spacchetti-upgrade command, that will automatically find out the latest version, download it, and write the new url and hashes in the packages.dhall file for you.

Running it would look something like this:

$ spago spacchetti-upgrade
Found the most recent tag for "spacchetti": "0.12.2-20190210"
Package-set upgraded to latest tag "0.12.2-20190210"
Fetching the new one and generating hashes.. (this might take some time)
Done. Updating the local package-set file..

Caching the Package Set

It is important to have the hashes set in your packages.dhall, like this:

...

let mkPackage =
      https://raw.githubusercontent.com/spacchetti/spacchetti/0.12.2-20190210/src/mkPackage.dhall sha256:0b197efa1d397ace6eb46b243ff2d73a3da5638d8d0ac8473e8e4a8fc528cf57

let upstream =
      https://raw.githubusercontent.com/spacchetti/spacchetti/0.12.2-20190210/src/packages.dhall sha256:1bee3f7608ca0f87a88b4b8807cb6722ab9ce3386b68325fbfa71d7211c1cf51

...

The reason why it's so important is that (apart from the safety guarantees) when your imports are protected by a hash they will be cached, considerably speeding up all the config-related operations.

You can freeze the imports in your package set by running:

$ spago freeze

Building, bundling and testing a project

We can build the project and its dependencies by running:

$ spago build

This is just a thin layer above the PureScript compiler command purs compile.
The build will produce very many JavaScript files in the output/ folder. These are CommonJS modules, and you can just require() them e.g. on Node.

It's also possible to include custom source paths when building (src and test are always included):

$ spago build --path 'another_source/**/*.purs'

Note: the wrapper on the compiler is so thin that you can pass options to purs. E.g. if you wish to output your files in some other place than output/, you can run

$ spago build -- -o myOutput/

Anyways, the above will create a whole lot of files, but you might want to get just a single, executable file. You'd then use the following:

# You can specify the main module and the target file, or these defaults will be used
$ spago bundle --main Main --to index.js
Bundle succeeded and output file to index.js

# We can then run it with node:
$ node .

However, you might want to build a module that has been β€œdead code eliminated” if you plan to make a single module of your PS exports, which can then be required from JS.

Gotcha covered:

# You can specify the main module and the target file, or these defaults will be used
$ spago make-module --main Main --to index.js
Bundling first...
Bundle succeeded and output file to index.js
Make module succeeded and output file to index.js

$ node -e "console.log(require('./index).main)"
[Function]

More information on when you might want to use the different kinds of build can be found at this FAQ entry.

You can also test your project with spago:

# Test.Main is the default here, but you can override it as usual
$ spago test --main Test.Main
Build succeeded.
You should add some tests.
Tests succeeded.

And last but not least, you can spawn a PureScript repl!
As with the build and test commands, you can add custom source paths to load, and pass options to the underlying purs repl by just putting them after --.
E.g. the following opens a repl on localhost:3200:

$ spago repl -- --port 3200

FAQ

Hey wait we have a perfectly functional pulp right?

Yees, however:

  • pulp is a build tool, so you'll still have to use it with bower or psc-package.
  • If you go for bower, you're missing out on package-sets (that is: packages versions that are known to be working together, saving you the headache of fitting package versions together all the time).
  • If you use psc-package, you have the problem of not having the ability of overriding packages versions when needed, leading everyone to make their own package-set, which then goes unmaintained, etc.
    Of course you can use Spacchetti to solve this issue, but this is exactly what we're doing here: integrating all the workflow in a single tool, spago, instead of having to use pulp, psc-package, purp, etc.

I miss bower link!

Take a look at the section on editing the package-set for details on how to add or replace packages with local ones.

I added a new package to the packages.dhall, but spago is not installing it. Why?

Adding a package to the package-set just includes it in the set of possible packages you can depend on. However if you wish spago to install it you should then add it to the dependencies list in your spago.dhall.

So if I use spago make-module this thing will compile all my js deps in the file?

No. We only take care of PureScript land. In particular, make-module will do the most we can do on the PureScript side of things (dead code elimination), but will leave the requires still in.
To fill them in you should use the proper js tool of the day, at the time of writing ParcelJS looks like a good option.

If you wish to see an example of a project building with spago + parcel, a simple starting point is the TodoMVC app with react-basic. You can see in its package.json that a "production build" is just spago build && parcel build index.html.
If you open its index.js you'll see that it does a require('./output/Todo.App'): the files in output are generated by spago build, and then the parcel build resolves all the requires and bundles all these js files in.

Though this is not the only way to include the built js - for a slimmer build or for importing some PureScript component in another js build we might want to use the output of make-module.

For an example of this in a "production setting" you can take a look at affresco.
It is a PureScript monorepo of React-based components and apps.
The gist of it is that the PureScript apps in the repo are built with spago build (look in the package.json for it), but all the React components can be imported from JS apps as well, given that proper modules are built out of the PS sources.
This is where spago make-module is used: the build-purs.rb builds a bundle out of every single React component in each component's folder - e.g. let's say we make-module from the ksf-login component and output it in the index.js of the component's folder; we can then yarn install the single component (note it contains a package.json), and require it as a separate npm package with require('@affresco/ksf-login').

Why can't spago also install my npm dependencies?

A common scenario is that you'd like to use things like react-basic, or want to depend on JS libraries like ThreeJS. In any case, you end up depending on some NPM package.

And it would be really nice if spago would take care of installing all of these dependencies, so we don't have to worry about running npm besides it, right?

While these scenarios are common, they are also really hard to support. In fact, it might be that a certain NPM package in your transitive dependencies would only support the browser, or only node. Should spago warn about that?
And if yes, where should we get all of this info?

Another big problem is that the JS backend is not the only backend around. For example, PureScript has a C backend and an Erlang backend among the others.
These backends are going to use different package managers for their native dependencies, and while it's feasible for spago to support the backends themselves, supporting also all the possible native package managers (and doing things like building package-sets for their dependencies versions) is not a scalable approach.

So this is the reason why if you or one of your dependencies need to depend on some "native" packages, you should run the appropriate package-manager for that (e.g. npm).
For examples on how to do it, see the previous FAQ entry.

I still want to use psc-package, can this help me in some way?

Yes! We can help you setup your psc-package-project to use the Spacchetti package-set.

We have two commands for it:

  • psc-package-local-setup: this command creates a packages.dhall file in your project, that points to the most recent Spacchetti package-set, and lets you override and add arbitrary packages.
    See the Spacchetti docs about this here.

  • psc-package-insdhall: do the Ins-Dhall-ation of the local project setup: that is, generates a local package-set for psc-package from your packages.dhall, and points your psc-package.json to it.

    Functionally this is equivalent to running:

    NAME='local'
    TARGET=.psc-package/$NAME/.set/packages.json
    mkdir -p .psc-package/$NAME/.set
    dhall-to-json --pretty <<< './packages.dhall' > $TARGET
    echo wrote packages.json to $TARGET