Fix ci build and cache

.github/workflows/test.yml

@@ -1,9 +1,9 @@
 name: "Tests"
 env:
-  CACHE_NAME: marlowe-temp
+  CACHE_NAME: marlowe-playground
   ALLOWED_URIS: "https://github.com https://api.github.com"
-  TRUSTED_PUBLIC_KEYS: "hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= marlowe-temp.cachix.org-1:1gPjVFpu4QjaAT3tRurCioX+BC23V7mjvFwpP5bV0Ec="
-  SUBSTITUTERS: "https://hydra.iohk.io https://iohk.cachix.org https://cache.nixos.org/ https://marlowe-temp.cachix.org"
+  TRUSTED_PUBLIC_KEYS: "hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= marlowe-playground.cachix.org-1:8TmdbVgcB4QzTmuzLxNdaCxdc8ZVI9S8eeeXsY9stxo="
+  SUBSTITUTERS: "https://hydra.iohk.io https://iohk.cachix.org https://cache.nixos.org/ https://marlowe-playground.cachix.org"
 on:
   push:
     branches-ignore:
@@ -14,6 +14,7 @@ on:
       - shlevy
   workflow_call:
 jobs:
+
   nix-code-formatting:
     strategy:
       matrix:
@@ -35,7 +36,30 @@ jobs:
           name: ${{ env.CACHE_NAME }}
           authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
       - run: nix build .#test-nix-fmt .#test-prettier .#test-purs-tidy .#test-png .#test-shell .#test-stylish-haskell
+  build-and-cache:
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v3
+      - uses: cachix/install-nix-action@v16
+        with:
+          nix_path: nixpkgs=channel:nixos-unstable
+          install_url: https://releases.nixos.org/nix/nix-2.8.1/install
+          extra_nix_config: |
+            allowed-uris = ${{ env.ALLOWED_URIS }}
+            trusted-public-keys = ${{ env.TRUSTED_PUBLIC_KEYS }}
+            substituters = ${{ env.SUBSTITUTERS }}
+            experimental-features = nix-command flakes
+      - uses: cachix/cachix-action@v12
+        with:
+          name: ${{ env.CACHE_NAME }}
+          authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"
+      - run: nix build .#generated-purescript .#marlowe-playground-server:exe:marlowe-playground-server
+
   nix-code-generation:
+    needs: build-and-cache
     strategy:
       matrix:
         os: [ubuntu-latest]

.gitignore

@@ -99,7 +99,6 @@ gh-pages
 *.db
 *.db-shm
 *.db-wal
-hie.yaml
 .pre-commit-config.yaml
 secrets/*/.gpg-id
 ghcid.txt

CONTRIBUTING.adoc

@@ -8,7 +8,7 @@ Make sure you have set up the link:./README{outfilesuffix}#prerequisites[prerequ
 
 === How to get a shell environment with tools
 
-You can get an environment for developing the entire project using `nix-shell` in the root directory.
+You can get an environment for developing the entire project using `nix develop` in the root directory.
 This includes a variety of useful tools:
 
 * The right version of GHC with all the external Haskell dependencies in its package database.
@@ -18,23 +18,23 @@ This includes a variety of useful tools:
 * `purescript-language-server`
 * ... and more
 
-Have a look in `shell.nix` to see what's there.
+Have a look in `flake.nix` to see what's there.
 
 We rely heavily on this approach to ensure that everyone has consistent versions of the tools that we use.
 Please do make use of this, since problems due to mismatched versions of tools are particularly annoying to fix!
 
-NOTE: You may want to consider using https://github.com/target/lorri[lorri] as a convenient alternative to running `nix-shell` directly.
 
 === How to use helper scripts to fix some common issues
 
 The shell comes with some tools for fixing various simple problems that the CI will complain about. Specifically:
 
+- `fix-nixfmt` will re-format nix files
+- `fix-prettier` will re-format JS, CSS and HTML files.
 - `fix-stylish-haskell` will re-format all the Haskell sources correctly.
 - `fix-purs-tidy` will re-format all the Purescript sources correctly.
 - `fix-png-optimization` will optimize all PNGs in the repository.
-- `updateMaterialized` will update the materialized Nix files (see xref:update-generated[later]).
 
-If you're not using `nix-shell` all the time and you want to run one of these, you can use `nix-shell --run`.
+If you're not inside the nix shell all the time and you want to run one of these, you can use `nix run .#fix-purs-tidy` instead.
 
 === How to build the code during development
 
@@ -43,7 +43,6 @@ From here you can build the project packages directly with `cabal`.
 
 NOTE: You may need to run `cabal update` so that `cabal` knows about the index state xref:update-index-state[we have pinned].
 
-For `stack`, you may want to use the Nix integration and point it at `shell.nix`, which will make sure you at least get the right GHC.
 
 [WARNING]
 ====
@@ -60,44 +59,11 @@ this can at least potentially cause problems or cause you to be missing
 bug workarounds.
 ====
 
-=== How to build the code with profiling
-
-VIf you launch `nix-shell` with the `enableHaskellProfiling` argument set to true, you will get a shell where all the dependencies have been built with profiling.
-
-Like this: `nix-shell --arg enableHaskellProfiling true`.
-
-[WARNING]
-====
-The shell with profiling dependencies is not currently cached, so this will result in you rebuilding all of our dependencies with profiling on your machine.
-This will take a *long* time.
-====
-
-Once you have a shell with profiling libraries for our dependencies, add `profiling: true` to `cabal.project.local`, which will tell cabal that you want profiling (in particular, that will cause it to build *our* libraries with profiling).
-Alternatively, you can pass the `--enable-profiling` option to `cabal` on an ad-hoc basis, but adding the option to `cabal.project.local` will make it apply to everything, which is probably what you want when you're doing profiling work.
-
-At this point you need to configure which cost centres you want GHC to insert.
-The https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/profiling.html[GHC user guide] explains this very well.
-A typical way of doing this is to add `-fprof-auto` to either the `ghc-options` in the `.cabal` file for the project, or in an `OPTIONS_GHC` pragma in the module you care about.
-
-[WARNING]
-====
-Do *not* set the `-prof` option yourself!
-This will enable profiling libraries unconditionally, which interferes with what `cabal` wants.
-Setting `profiling: true` already sorts this out properly.
-====
-
-Then you can use the RTS `-p` option to dump a profile e.g. `cabal run plc ... -- +RTS -p`.
-
-There are various tools for visualizing the resulting profile, e.g. https://hackage.haskell.org/package/ghc-prof-flamegraph.
 
 === How to setup `haskell-language-server`
 
 The `nix-shell` environment has a `haskell-language-server` binary for the right version of GHC.
 
-IMPORTANT: this binary is called `haskell-language-server`, rather than `haskell-language-server-wrapper`, which is what some of the editor integrations expect.
-
-We don't have a `hie.yaml`, the implicit cradle support in HLS seems to work fine these days.
-
 === How to setup `purescript-language-server`
 The `nix-shell` environment has a `purescript-language-server` executable.
 Follow the instructions for your editor to configure it:
@@ -108,18 +74,6 @@ NOTE: you must run your editor from the nix shell, and not from, say, an
 application launcher like spotlight or dmenu for your editor to find it in
 its PATH.
 
-[[update-generated]]
-=== How to update the generated Haskell package set
-
-Some of the Nix code that builds all the Haskell packages and their dependencies is generated automatically.
-However, to avoid doing too much work all the time, we have checked the generated output in.
-
-IMPORTANT: These files needs to be regenerated if you change any dependencies in cabal files.
-But the CI will tell you if you've failed to do so.
-
-You can regenerate the files by running `updateMaterialized` (provided by `nix-shell`) from the repository root.
-
-This will also update the `plan-256` shas for the extra Haskell tools, if you have a problem iwth that.
 
 === How to add a new Haskell package
 
@@ -147,17 +101,14 @@ For the moment you have to do this by hand, using the following command to get t
 [[update-nix-pins]]
 === How to update our pinned Nix dependencies
 
-We pin versions of some git repositories that are used by Nix, for example `nixpkgs`.
+We use nix flakes with a flake.lock file. You can update an input by using
 
-We use Nix flakes to manage these dependencies, even though we do not
-actually use flakes for normal Nix builds. To manage the dependencies,
-see https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake.html#flake-inputs[the Nix flake inputs documentation]
-and https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-flake-lock.html[the Nix flake lock command].
+----
+nix flake lock --update-input <input>
+----
 
-Specifically, you will probabl want to say `nix flake lock --update-input <input-name>`. 
 Do *not* use `nix flake update`, as that will update all the inputs, which we typically don't want to do.
 
-Inside of the project nix-shell, a version of Nix with flakes is available under the alias `nix-flakes`.
 
 [[update-index-state]]
 === How to update the Hackage index state

README.adoc

@@ -18,7 +18,6 @@ Related projects:
 ====
 The rest of this README is focussed on people who want to develop or contribute to Marlowe.
 
-For people who want to *use* Marlowe, please consult the <<user-documentation>>.
 ====
 
 [[cache-warning]]
@@ -39,8 +38,6 @@ If you find yourself building GHC, STOP and fix the cache.
 
 Issues can be filed in the https://github.com/input-output-hk/marlowe-playground/issues[GitHub Issue tracker].
 
-However, note that this is pre-release software, so we will not usually be providing support.
-
 [[how-to-develop]]
 === How to develop and contribute to the project
 
@@ -78,7 +75,7 @@ If you use Nix, these tools are provided for you via `shell.nix`, and you do *no
 [[building-with-nix]]
 ==== How to build the Haskell packages and other artifacts with Nix
 
-Run `nix build -f default.nix marlowe.haskell.packages.marlowe.components.library` from the root to build the Marlowe library.
+Run `nix build marlowe-playground-server:exe:marlowe-playground-server` from the root to build the Playground backend service.
 
 See <<nix-build-attributes>> to find out what other attributes you can build.
 
@@ -93,9 +90,8 @@ The best way is to do this is inside a `nix-shell`.
 For fresh development setups, you also need to run `cabal update`.
 ====
 
-Run `cabal build marlowe` from the root to build the Marlowe library.
+Run `cabal build all` from the root to build all artifacts.
 
-See the link:./cabal.project[cabal project file] to see the other packages that you can build with `cabal`.
 
 === Deployment
 
@@ -124,8 +120,9 @@ To set up the cache:
 . On non-NixOS, edit `/etc/nix/nix.conf` and add the following lines:
 +
 ----
-substituters        = https://hydra.iohk.io https://iohk.cachix.org https://cache.nixos.org/
-trusted-public-keys = hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
+experimental-features = nix-command flakes
+substituters        = "https://cache.nixos.org" "https://cache.iog.io https://marlowe-playground.cachix.org"
+trusted-public-keys = "hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= marlowe-playground.cachix.org-1:8TmdbVgcB4QzTmuzLxNdaCxdc8ZVI9S8eeeXsY9stxo="
 ----
 +
 [NOTE]
@@ -136,9 +133,15 @@ You must be a https://nixos.org/nix/manual/#ssec-multi-user[trusted user] to do
 . On NixOS, set the following NixOS options:
 +
 ----
-nix = {
-  binaryCaches          = [ "https://hydra.iohk.io" "https://iohk.cachix.org" ];
-  binaryCachePublicKeys = [ "hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=" "iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo=" ];
+nix.settings = {
+  experimental-features = [ "nix-command" "flakes" ];
+  substituters = [
+    "https://cache.nixos.org"
+    "https://cache.iog.io"
+  ];
+  trusted-public-keys = [
+    "hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ="
+  ];
 };
 ----
 
@@ -164,19 +167,26 @@ sudo launchctl start org.nixos.nix-daemon
 [[nix-build-attributes]]
 === Which attributes to use to build different artifacts
 
-link:./default.nix[`default.nix`] defines a package set with attributes for all the artifacts you can build from this repository.
+link:./flake.nix[`flake.nix`] defines the packages that you can build from this repository.
 These can be built using `nix build`.
 For example:
 
 ----
-nix build -f default.nix docs.site
+nix build .#marlowe-playground-generate-purs
+----
+
+.Other builds
+To see other packages that can be built or run, execute
+
+----
+nix flake show
 ----
 
-.Example attributes
-* Project packages: defined inside `marlowe.haskell.packages`
-** e.g. `marlowe.haskell.packages.marlowe.components.library`
+[NOTE]
+====
+You might need to comment other build systems inside flake.nix
+====
 
-There are other attributes defined in link:./default.nix[`default.nix`].
 
 == Licensing
 

flake.nix

@@ -83,6 +83,11 @@
               final.haskell-nix.cabalProject' {
                 src = ./.;
                 compiler-nix-name = "ghc8107";
+                modules = [
+                  {
+                    packages.plutus-script-utils.ghcOptions = [ "-Wwarn" "-Wno-unused-packages" ];
+                  }
+                ];
                 shell.tools = {
                   cabal = { };
                   # hlint = {};

hie.yaml

@@ -0,0 +1,2 @@
+cradle:
+    cabal: