Deploying an application that contains several packages

[matsrietdijk]

First: thanks for your open source work, Halcyon, HoH, Cannot, ..., they're all amazing pieces of work.

We are working on a CMS in Haskell (to be publicly announced in the coming week), that we would like to deploy with HoH (and provide a tutorial for others to deploy theirs with HoH).

The CMS recommends to break up features into separate packages, therefor a git repo of a project that uses this CMS contains a subdirectory for each package.

For instance: https://github.com/lambdacms/lambdacms.org

The "base" subfolder is contains the master site, it builds the actual binary that should be hosted. The other subdirectories contain dependencies of the "base" app.

Is it possible to install an app like this with HoH? Or do you have suggestions on how we can make it easier for HoH to install an app like this?

Thanks.

[mietek]

Thanks for the kind words. Currently, Halcyon doesn’t support apps which aren’t located at the top level of the source directory. This is tracked as mietek/halcyon#11.

I’ve now opened a small PR, fixing a few minor packaging issues in your project (lambdacms/lambdacms.org#1).

With these changes in place, I’ve successfully built your project on a regular Ubuntu 14.04 machine, then deployed it to Heroku. It’s now running on lambdacms.herokuapp.com.

In order to build your app, I first split the project into three repositories, one per package:

• https://github.com/mietek/lambdacms.org-base
• https://github.com/mietek/lambdacms.org-page
• https://github.com/mietek/lambdacms.org-tutorial

Next, I added a number of Halcyon magic files to the base package (https://github.com/mietek/lambdacms.org-base/commit/185bdbd7e71c9d96b2db87ba5d93cee320138c3c):

$ git show --name-only HEAD
commit 185bdbd7e71c9d96b2db87ba5d93cee320138c3c
Author: Miëtek Bak <mietek@bak.io>
Date:   Thu Jan 29 16:29:25 2015 +0000

    Package with Halcyon

.halcyon/constraints
.halcyon/extra-data-files
.halcyon/extra-os-packages
.halcyon/sandbox-extra-apps
.halcyon/sandbox-extra-apps-constraints/alex-3.1.4.constraints
.halcyon/sandbox-extra-apps-constraints/happy-1.19.5.constraints
.halcyon/sandbox-extra-configure-flags
.halcyon/sandbox-extra-os-packages
.halcyon/sandbox-sources

Each of these files is described in the Halcyon reference. Most importantly, the HALCYON_SANDBOX_SOURCES option is used to point to the other two packages:

$ cat .halcyon/sandbox-sources 
https://github.com/mietek/lambdacms.org-page
https://github.com/mietek/lambdacms.org-tutorial

Finally, to deploy your app to Heroku, I configured it in the manner required by Yesod, by manually extracting portions of the Heroku Postgres DATABASE_URL, which is set by heroku addons:add heroku-postgresql:hobby-dev:

$ heroku config | grep -E '^(PG|APPROOT)'
APPROOT:                      https://lambdacms.herokuapp.com
PGDATABASE:                   d8n9…
PGHOST:                       ec2-….compute-1.amazonaws.com
PGPASS:                       IJJb…
PGUSER:                       kjrj…

When deploying to Heroku, Yesod automatically finds the static directory in the current working directory. When deploying elsewhere, it’ll also be necessary to point STATIC_DIR into the Cabal run-time data files directory:

$ cat /app/.halcyon/lambdacms-dot-org-0.0.0/data-dir
/app/share/x86_64-linux-ghc-7.8.4/lambdacms-dot-org-0.0.0

$ ls /app/share/x86_64-linux-ghc-7.8.4/lambdacms-dot-org-0.0.0
static

The three-package split isn’t ideal for a number of reasons. Join #haskell-deployment on the freenode IRC network if you’d like to discuss this with less latency.

[matsrietdijk]

Thanks for this answer and the fixes. We are going to use this multi-repo setup for now.

[cies]

Thanks @mietek for the elaborate reply, and the PR. As @matsrietdijk said we'll move to 1-repo-per-cabal-file and include a HoH-deploy tutorial for the initial release.

I'm still interested why you think multi-repos are not ideal. I can come up with some reasons, but I also see reasons in favor of them.

[mietek]

Oops, just noticed APPROOT was set incorrectly on the live instance (lambdacms.herokuapp.com). Images and styles work now.

[mietek]

The three-package split interacts with Halcyon in a complex fashion. It would be better to explain this in chat, once you have gone through the Halcyon tutorial.

[mietek]

In short:

• Halcyon works around various Cabal issues by (almost) never reinstalling packages into an existing sandbox
• Halcyon determines which sandbox is needed by hashing the version constraints of your app’s dependencies
• Halcyon saves time by building a particular sandbox once, then restoring it from an archive

Your app is lambdacms.org-base, and its dependencies include lambdacms.org-page and lambdacms.org-tutorial, which are referenced via HALCYON_SANDBOX_SOURCES. If you change their code, Halcyon won’t notice and will keep using the existing sandbox, unless you either bump the version numbers, or force a sandbox rebuild with HALCYON_SANDBOX_REBUILD. Both options aren’t ideal for iterative development.

[mietek]

Another option is to keep all of the code in a single repo, with your app at the top level, your app’s dependencies as subdirectories, and HALCYON_SANDBOX_SOURCES referencing subdirectory names. Then, if you change the code of your app’s dependencies, Cabal will notice at the time the app is built, and reinstall them into the sandbox. This still isn’t ideal, because the reinstall can fail, but any failure could be mitigated by forcing a sandbox rebuild.

To sum up, the three-package split adds a lot of complexity, and I haven’t tested such use cases much. I’ll be happy to help you work through any issues, but I can’t guarantee everything will go smoothly.

[cies]

Once again thanks for the resourceful reply. We'll reconsider it, in the light of the release we are planning to do in the coming days.